JP2002342098A - Management device, data processing system, management method and program for making computer perform management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform the deadline management of distributed batch processings in a distributed batch system. SOLUTION: Clients 3A-3N transmit a batch processing request to a management server 1 and the manager 11 of the management server 1 inquires of processing servers 2A-2N whether or not the batch processing can be completed by a deadline. The batch job scheduling demons 22A-22N of the respective processing servers judge whether or not the processing can be performed by the deadline and reply the manager 11. The manager 11 selects one of the processing servers capable of performing the processing by the deadline and the clients 3A-3N feed the batch processing to the batch service 23 of the selected processing server.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed batch system for enabling deadline management in a distributed system comprising a plurality of servers for providing a plurality of batch services and a plurality of clients for requesting batch jobs. It relates to job scheduling technology.

[0002]

2. Description of the Related Art In a conventional batch system, at the stage of designing, a static load distribution for allocating a CPU resource, a memory resource, a disk resource, and the like to be executed is performed. In a distributed batch system such as Japanese Patent Application No. 9-146660 “Dynamic load distribution processing method and system for batch jobs in a distributed system environment”, dynamic load distribution on a distributed system is possible. There is a problem that the deadline management of batch processing cannot be performed on the system.

[0003]

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems and enables deadline management in a distributed batch system.

[0004]

A management apparatus according to the present invention manages a plurality of data processing apparatuses that perform predetermined data processing based on a data processing request, and performs a data processing for each data processing request. A communication unit that receives, from at least one or more data processing devices, prediction completion time information indicating a prediction completion time of data processing for each data processing request; A designation unit that designates a designated data processing device for each data processing request based on the information, and causes the designated data processing device to perform data processing based on the data processing request.

The communication unit is capable of completing term-limited data processing based on a term-limited data processing request in which at least one data processing unit has a term of completion by at least one data processing device before the completion term. Receiving the completion possible information that notifies the completion of the deadline processing, the designation unit designates any one of the data processing devices that have transmitted the completion complete information as a designated data processing device that performs the time-limited data processing. Features.

[0006] The communication unit executes the time-limited data processing when at least one or more data processing apparatuses stop executing at least one or more data processing scheduled to be executed by each data processing apparatus. Receiving the conditional completion possible information notifying that the completion can be completed before the completion deadline, and the designation unit sets any one of the data processing devices that transmitted the conditional completion feasibility information to the data processing device according to the deadline. It is designated as a designated data processing device for performing attached data processing.

The communication unit receives, from at least one or more data processing devices, conditional completion possible information including information on at least one or more data to be canceled to be suspended in each data processing device. The specifying unit selects any one of the data processing devices that transmitted the conditional completion possible information as a candidate data processing device, and sets the candidate data processing device to cancel any other target data process. It is determined whether or not the candidate data processing apparatus can be transferred to any of the other data processing apparatuses. It is specified as

The communication unit receives, from at least one or more data processing devices, conditional completion possible information indicating a data processing amount of the data processing to be canceled, and the designation unit receives the conditional completion possible information. And selecting the candidate data processing device based on the data processing amount of the stop target data processing shown in (1).

[0009] The communication unit receives, from at least one or more data processing devices, conditional completion enable information indicating the number of data processing to be canceled, and the designation unit indicates the conditional completion enable information indicated in the conditional completion enable information. The candidate data processing apparatus is selected based on the number of the data processing to be aborted.

[0010] The communication unit receives, from any one of the data processing devices other than the candidate data processing device, candidate prediction completion time information for notifying the prediction completion time of the data processing to be stopped by the candidate data processing device, The designating unit designates, based on the candidate prediction completion time information, one of the data processing devices that has transmitted the candidate prediction completion time information, as a migration destination of the cancellation target data processing of the candidate data processing device. It is characterized by the following.

[0011] The communication unit receives, from at least one or more data processing devices, conditional completion possible information indicating at least one or more stop target data processes and a completion deadline of each stop target data process. The designation unit is a candidate for any one of the data processing devices that transmitted the conditional completion possible information indicating at least one or more stop target data processes and the completion deadline of each stop target data process, as the candidate. Selected as a data processing device, the communication unit,
From any one of the data processing devices other than the candidate data processing device, candidate completion possible information for notifying that each stop target data process of the candidate data processing device can be completed before the completion deadline of each stop target data Upon receiving, the specification unit specifies any of the data processing devices that have transmitted the candidate completion possibility information as a migration destination of the data processing to be canceled by the candidate data processing device.

[0012] The communication unit may be configured to execute a conditional completion from a plurality of data processing devices, wherein a plurality of data processing units to be canceled and a common completion time period commonly defined for the plurality of data units to be canceled are indicated. Receiving the possible information, and the specifying unit transmits any one of the data processing apparatuses that transmitted the conditional completion possible information indicating a plurality of data to be suspended and a common completion date for the plurality of data to be suspended to the candidate data. Selected as a processing device, the communication unit can complete a plurality of stop target data processes of the candidate data processing device by a data processing device other than the candidate data processing device before a common completion deadline. Receiving the candidate completion possible information, and specifying the one of the data processing devices among the data processing apparatuses that transmitted the candidate completion possible information. The characterized in that it specifies the destination of the plurality of stop target data processing of the candidate data processing apparatus.

[0013] The management device is connected to at least one or more data processing request devices for transmitting a data processing request, and the communication unit receives a data processing request from one of the data processing request devices, A request for calculating a prediction completion time of data processing based on the received data processing request is transmitted to the processing device.

The communication unit receives a data processing request from one of the data processing devices and performs a data processing based on a data processing request received from a data processing device other than the data processing device that transmitted the data processing request. And transmitting a calculation request for the prediction completion time.

[0015] The management device further calculates a data processing unit for performing predetermined data processing based on the data processing request, and a prediction completion time of the data processing by the data processing unit.
A completion time prediction unit that outputs the calculated prediction completion time to the designation unit as own device prediction completion time information, wherein the designation unit acquires the own device prediction completion time information from the completion time prediction unit, The designated data processing device is specified from the plurality of data processing devices and the management device based on the acquired own device prediction completion time information and the prediction completion time information received by the communication unit. .

The management device may further include a data processing unit for performing predetermined data processing based on the data processing request, and whether the data processing unit can complete the time-limited data processing before the completion time limit. And determining whether the data processing unit can complete the data processing with time limit before the completion time limit in accordance with the determination result. The designation unit acquires the own device completion possible information from the completion possibility determination unit, and based on the acquired own device completion possibility information and the completion possibility information received by the communication unit, The designated data processing device is designated from among the data processing device and the management device.

A data processing system according to the present invention includes a plurality of data processing devices for performing predetermined data processing based on a data processing request, and a data for managing the plurality of data processing devices and performing data processing for each data processing request. A data processing system comprising: a management device that designates a processing device as a designated data processing device, wherein the management device performs, with respect to the plurality of data processing devices, data processing with a time limit for completing data processing. A completion permission inquiry is sent to inquire whether the time-limited data processing based on the request can be completed before the completion time limit.Each data processing device transmits a completion completion prediction of the time-limited data processing based on the completion permission inquiry. Measure the time and complete the time-limited data processing before the completion time limit based on the measured predicted completion time It determines whether it is possible to,
When the time-limited data processing can be completed before the completion time limit, including the predicted completion time, the completion possible information for notifying that the time-limited data processing can be completed before the completion time limit is included. The management device transmits to the management device, and based on the predicted completion time included in the completion possibility information, the management device sends one of the data processing devices that transmitted the completion possibility information to the time-limited data processing device. Is specified as a specified data processing device that performs

Each of the data processing devices stops executing at least one or more data processes scheduled to be executed by each of the data processing devices when the data processing with a time limit cannot be completed before the completion time limit. Then, it is determined whether the time-limited data processing can be completed before the completion time limit, and if the execution of at least one or more data processing scheduled to be executed is stopped, the time-limited data processing is performed. If it is possible to complete before the completion deadline, including information on at least one or more stop target data processing to be canceled, if the stop of the execution of the stop target data processing, the time-limited data processing The management device transmits conditional completion possible information notifying that the completion can be completed before the completion deadline to the management device, and the management device Whether any of the data processing devices that have transmitted the function information is selected as a candidate data processing device, and whether or not the cancellation target data processing of the candidate data processing device can be transferred to any of the other data processing devices And when the data processing to be stopped of the candidate data processing apparatus can be transferred to any other data processing apparatus, the candidate data processing apparatus is designated as the designated data processing apparatus.

Each data processing device is connected to at least one or more data processing request devices for transmitting a data processing request, and each data processing device has a data processing completion time limit determined by one of the data processing request devices. Received the time-limited data processing request, and determines whether the time-limited data processing based on the received time-limited data processing request can be completed before the completion time limit. When the processing cannot be completed, the time-limited data processing request is transferred to the management device, and the management device sends the time-limited data processing request to a data processing device other than the data processing device that transferred the time-limited data processing request. Sends a completion permission inquiry asking whether the data processing with time limit can be completed before the completion time limit. It is characterized in.

Each data processing device is connected to at least one or more data processing request devices for transmitting a data processing request, and each data processing device has a data processing completion time limit determined by one of the data processing request devices. Received the time-limited data processing request, and determines whether the time-limited data processing based on the received time-limited data processing request can be completed before the completion time limit. When the processing cannot be completed, if the execution of at least one or more data processing scheduled to be executed by each data processing apparatus is stopped, the time-limited data processing can be completed before the completion time limit. Judge whether or not the execution of at least one or more data processing scheduled to be executed is completed and the time-limited data processing is completed. If it can be completed before the deadline, a data processing request is sent to the management device requesting that at least one or more data to be aborted to be aborted is completed before a predetermined completion deadline. And
The management device receives a data processing request of the data processing to be stopped from each data processing device, and transmits the data processing request of the data processing to be stopped to a data processing device other than the data processing device that transmits the data processing request of the data to be stopped. It is characterized by transmitting a completion possibility inquiry for inquiring whether data processing can be completed before the completion deadline.

A management method according to the present invention manages a plurality of data processing devices that perform predetermined data processing based on a data processing request, and designates a data processing device that performs data processing for each data processing request as a designated data processing device. A communication step of receiving, from at least one or more data processing devices, prediction completion time information indicating a prediction completion time of data processing for each data processing request, and performing data processing based on the prediction completion time information. Specifying a designated data processing device for each request, and causing the designated data processing device to perform data processing based on the data processing request.

In the communication step, the term-limited data processing based on the term-limited data processing request in which the term of completion of the data processing is determined can be completed before the term of completion by at least one data processing device. Receiving the completion possible information, and specifying the one of the data processing devices that transmitted the completion possible information as a specified data processing device that performs the time-limited data processing. Features.

In the communication step, the execution of the time-limited data processing is stopped when at least one or more data processing apparatuses stop executing at least one or more data processing scheduled to be executed by each data processing apparatus. Receiving the conditional completion possible information notifying that the completion can be completed before the completion deadline, and the specifying step causes the one of the data processing devices that transmitted the conditional completion feasibility information to the data processing device to perform the deadline. It is designated as a designated data processing device for performing attached data processing.

[0024] The communication step includes receiving, from at least one or more data processing apparatuses, conditional completion possible information including information on at least one or more data processing units to be aborted in each data processing apparatus. The specifying step selects any one of the data processing devices that transmitted the conditional completion possible information as a candidate data processing device, and sets the candidate data processing device to cancel any other target data process. It is determined whether or not the candidate data processing apparatus can be transferred to any of the other data processing apparatuses. It is specified as

A program for causing a computer to execute the management method according to the present invention manages a plurality of data processing devices that perform predetermined data processing based on a data processing request, and executes a data processing that performs data processing for each data processing request. A program for causing a computer to execute a management method of designating a device as a designated data processing device, wherein at least one or more data processing devices provide predicted completion time information indicating a predicted completion time of data processing for each data processing request. A management method comprising: a communication step of receiving; and a designation step of, based on the prediction completion time information, designating a designated data processing device for each data processing request and causing the designated data processing device to perform data processing based on the data processing request. It is a program to be executed by a computer.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 The configuration of the distributed batch system according to the present embodiment will be described with reference to FIG. In the figure, 1 is a management server as a management device, 2 is a processing server as a data processing device, and 3 is a client as a data processing request device. In FIG. 1, the server 2
Means that 2A to 2N exist, and the client 3
3A to 3N exist. Note that the management server 1 and the processing servers 2A to 2N may be simply referred to as servers. The management server 1 specifies a processing server to perform batch processing for each batch processing request transmitted from the clients 3A to 3N from among the processing servers 2A to 2N. Note that the configuration of the management server 1 and the processing server 2 are almost the same, except that the manager 11 is arranged in the management server 1. Therefore, the management server 1 can also perform batch processing as a processing server.

Next, the internal configuration of the management server 1 will be described. A communication unit 10 communicates with the clients 3A to 3N and the processing servers 2A to 2N.
The communication unit 10 transmits and receives a batch processing request and its response to and from the clients 3A to 3N, for example.
It sends and receives an estimate request and a response to the estimate request, which will be described later, between the processing servers 2A to 2N. Reference numeral 11 denotes a manager as a designating unit, which designates, from among the processing servers 2A to 2N, a processing server to perform batch processing for each batch processing request transmitted from the clients 3A to 3N. The manager 11 transmits an estimation request to the processing servers 2A to 2N through the communication unit 10, receives an estimation response through the communication unit 10, and specifies a processing server based on the received estimation response. Reference numeral 12 denotes a batch job scheduling daemon that functions as a completion time prediction unit or a completion availability determination unit. The batch job scheduling daemon 12 calculates the completion time of the batch processing (the own apparatus predicted completion time) in response to the estimation request from the manager 11, and returns the calculated completion time of the batch processing as an estimation response. Reference numeral 13 denotes a batch service as a data processing unit, which performs a predetermined batch process based on a batch process request.

The management server 1 is a computer having a CPU such as a microprocessor and recording means such as a magnetic disk and a semiconductor memory.
A computer program describing the functions of the manager 11, the batch job scheduling daemon 12, and the batch service 13 is recorded, and the CPU reads out the computer programs and executes the contents of the programs to realize the functions of the management server 1. ing.

Next, the internal configuration of the processing servers 2A to 2N will be described. Reference numerals 20A to 20N denote communication units, which receive an estimation request from the communication unit 10 of the management server 1 and transmit an estimation reply to the communication unit 10 of the management server 1. Reference numerals 22A to 22N denote batch job scheduling daemons, which perform the same operations as the batch job scheduling daemon 12 of the management server 1. That is, in response to the estimation request requested by the management server 1, the completion time of the batch processing (estimated completion time) is calculated, and the calculated completion time of the batch processing is returned as an estimation response. Also, the batch job scheduling daemon 22
A to 22N function as a completion time prediction unit or a completion possibility determination unit. 23A to 23N are batch services, and perform predetermined batch processing based on a batch processing request, similarly to the batch service 13 of the management server 1. Further, the batch services 23A to 23N function as data processing units. In FIG. 1, one batch service is arranged in each of the management server 1 and the processing servers 2A to 2N, but a plurality of batch services may be provided in each server.

Next, clients 3A to 3N, manager 11, batch job scheduling daemon 12,
22A-22N, batch service 13, 23A-23N
Will be described with reference to FIG. The client requesting the batch processing transmits the batch processing request to the manager 11 with the deadline and the work amount of the batch processing added (S21). Here, it is assumed that the client 3K transmits a batch processing request. Note that a batch processing request indicating a deadline is called a data processing request with a term. The manager 11 is a batch job scheduling daemon 1 on each of the processing servers 2A to 2N and the management server 1.
2, 22A to 22N to send an estimate request (S2
2). Batch job scheduling daemon 12,2
2A to 22N calculate the completion time of the requested batch process (predicted completion time) from the schedule status of each batch service on the own server, the deadline added to the request, and the workload, and can complete the deadline within the deadline If so, the completion time is sent to the manager 11, and if not possible, an estimate response indicating that processing is impossible is sent to the manager 11 (S23). The manager 11 receives a response from each batch job scheduling daemon and selects the one with the earliest completion time from among the responses.
Here, the batch job scheduling daemon 22
It is assumed that L is selected. Then, the manager 11 transmits a scheduling request to the selected batch job scheduling daemon 22L (S24). If there is no server that can process, the server sends a response indicating that processing is impossible to the client 3K. The batch job scheduling daemon 22L performs scheduling for the corresponding batch service 23L (S25), and transmits the completion of the scheduling to the manager 11 (S2).
6). The manager 11 transmits the selected server as an answer to the client (S27). The client submits the batch to the answered server (S2
8). The batch service 23L executes the input batch process in accordance with the scheduling from the batch job scheduling daemon, and transmits a batch execution result to the client 3K (S29).

A flowchart of the manager 11 according to the present embodiment will be described with reference to FIG. In S101, the manager 11 receives a batch processing request from any of the clients 3A to 3N via the communication unit 10. In S102, an estimate request is transmitted to each batch job scheduling daemon 12, 22A to 22N. In step S103, reception of a response is waited. If there is no response even after the timeout period has elapsed, the process proceeds to step S106, and if there is a response, the process proceeds to step S104. In step S104, the estimation response is received. In step S105, if all the responses have been received, the process proceeds to step S106.
Return to 103. In S106, if there is a processing server that can process by the deadline, the process proceeds to S107; otherwise, the process proceeds to S111. In S107, the processing server with the earliest completion time is determined as the processing server that performs the batch processing in response to the batch processing request from the clients 3A to 3N from among the processing servers that can be processed before the deadline, and in S108, in S107 The scheduling request is transmitted to the batch job scheduling daemon on the determined processing server. In S109, the completion of scheduling is received from the batch job scheduling daemon, and the server name determined in S107 is transmitted as an answer to the client. In S106, if there is no processing server capable of processing before the deadline, S1
In step 11, the processing failure is transmitted to the client as a response.

With the above configuration, the present embodiment enables deadline management on a distributed batch system.

In the above description, the management server 1 has the batch job scheduling daemon 12 and the batch service 13, and the management server 1 itself requests the batch processing in response to the batch processing request. The batch job scheduling daemon 12 and the batch service 13 need not be provided. In this case, the management server 1 only manages the processing servers 2A to 2N.

Embodiment 2 The configuration of the distributed batch system according to the second embodiment is the same as that of FIG. The operation of the present embodiment is almost the same as the operation of the first embodiment shown in FIG. 2, but the processing of S111 is different. The different operation will be described with reference to FIG. First, the manager 11 executes the batch job scheduling daemons 12, 22A to 22N.
A conditional estimation request is transmitted to the client (S31). Here, the conditional estimation request means that each batch job scheduling daemon 12,2 returns a conditional estimation response when it is determined that the batch processing based on the batch processing request cannot be completed by the deadline.
2A to 22N. Each of the batch job scheduling daemons 12, 22A to 22N deadline the requested process unless one of the batch processes already scheduled is executed (that is, if the execution of the batch process is stopped). If the processing is completed by this time, the work amount (data processing amount) of the batch processing (data processing to be canceled) to be canceled and the deadline of the batch processing to be canceled are transmitted to the manager 11 as a conditional estimation response. (S32). The manager 11 selects the batch job scheduling daemon with the least amount of work from the received conditional estimate responses. In FIG. 4, it is assumed that the batch job scheduling daemon 22L has been selected. Note that the selected batch job scheduling daemon is called a candidate data processing device. The manager 11 will cancel the batch process indicated in the selected conditional estimate answer,
The estimation request is transmitted to each batch job scheduling daemon other than the batch job scheduling daemon 22L (S33). Each batch job scheduling daemon other than the batch job scheduling daemon 22L transmits an estimate reply to the manager 11 (S34). The manager 11 selects the batch job scheduling daemon with the earliest completion time from the estimated response. In FIG. 4, for example, the batch job scheduling daemon 22M is selected. Next, a batch transfer request is transmitted to the selected batch job scheduling daemon 22M (S35). Upon receiving the batch transfer request, the batch job scheduling daemon 22M transfers the stop target batch process from the batch job scheduling daemon 22L on the processing server 2L that was to execute the stop target batch process. 11 is transmitted to S11 (S3).
6). When the transfer of each batch process is completed, Manager 1
1 transmits a scheduling request to the batch job scheduling daemon 22L (S24).

The operation of the manager according to this embodiment is as follows.
Parts different from the first embodiment will be described with reference to FIG. If there is no server that can be processed in S106 in FIG. 3, the process proceeds to S201 in FIG. 5, and in S201, a conditional estimation request is transmitted to each batch job scheduling daemon 12, 22A to 22N. In step S202, reception of a response is waited. If there is no response even after the timeout period has elapsed, the process proceeds to step S205.
Proceed to 3. In S203, a conditional estimation response is received. In S204, if all the responses have been received, the process proceeds to S205, and if not, the process returns to S202.
In step S205, if the batch process to be canceled is not performed, the process proceeds to step S206 if there is a processing server that can perform the batch process before the deadline. Otherwise, the process proceeds to step S215.
In S206, the batch process to be canceled with the least amount of work is determined as the transfer batch process. In S207, an estimate request is transmitted to each batch job scheduling daemon other than the batch job scheduling daemon of the processing server 2 which was to execute the migration batch processing determined in S206. In step S208, reception of a response is waited. If there is no response after the timeout period has elapsed, the process proceeds to step S211. If there is a response, the process proceeds to step S209.
In step S209, the estimation response is received. In step S210, if all the responses have been received, the process proceeds to step S211. If not, the process returns to step S208. In S211,
If there is a processing server capable of performing the transfer batch processing by the deadline, the process proceeds to S212; otherwise, the process proceeds to S216.
Proceed to. In S212, a server having the earliest completion time is determined as a migration destination from a processing server capable of performing the migration batch processing by the deadline.
A batch processing shift request is transmitted to the batch job scheduling daemon on the server determined in step 12. S214
, The completion of batch processing transfer from the batch job scheduling daemon is received, and the process proceeds to S108 in FIG. If there is no server that can be processed in S205 and S211, a response indicating that processing is not possible is transmitted to the client in S215 and S216.

With the above operation, the server can be used efficiently. In FIG. 5, S21
In step 6, a response indicating that processing is not possible is transmitted to the client, but the processing may return to step S206, select another batch processing, and continue processing. As described above, the stop target batch process with the smallest data amount is selected in S206, but since the deadline of the selected stop target batch process is tight, there is no server that can process before the deadline. Case is conceivable. Therefore, even in the case of No in S211, the process returns to S206, selects another batch process to be canceled which has a slightly larger data amount but still has a deadline, and selects a new batch process to be canceled. On the other hand, it is conceivable to perform the processing after S207.

Embodiment 3 The configuration of the distributed batch system according to the third embodiment is the same as that of FIG. The operation of the present embodiment is almost the same as the operation of the second embodiment shown in FIG. 4, but the contents of the conditional estimation answer are different. Although the conditional estimation response in the second embodiment indicates the workload and deadline of the batch processing requiring one shift, in the present embodiment, the batch job scheduling daemon executes a plurality of batch processes. If the requested process can be completed by the deadline if not executed, the number of batch processes to be canceled, the amount of work for each batch process to be canceled, and the deadline for each batch process to be canceled It is transmitted to the manager 11 as a conditional estimate response.

The operation of the manager 11 according to the present embodiment will be described with reference to FIG. If there is a processing server that can process in S205 of FIG. 5, the process proceeds to S301. In step S301, the server that has the least number of batch processes to be canceled and needs to be migrated, which is indicated in the conditional estimation response, is determined as a server to be migrated in batch processing. In S302,
The processes of S303 to S308 are performed for each of the batch processes to be canceled of the server determined in S301, and when the processes for all the batch processes to be canceled are completed, the process proceeds to S309.
In S303, an estimate request is transmitted to each batch job scheduling daemon other than the batch job scheduling daemon that runs on the server determined in S301. In step S304, the process waits for reception of a response. If there is no response even after the timeout period has elapsed, the process proceeds to step S307, and if there is a response, the process proceeds to step S305. In step S305, the estimation response is received. In step S306, if all the responses have been received, the process proceeds to step S307, and if not, the process returns to step S304. In step S307, if there is a server that can process the server process to be canceled of the server determined in step S301 before the deadline, the process proceeds to step S308. Otherwise, the process proceeds to step S312. In S308, the server with the earliest completion time is determined, and the process returns to S302. When the migration destination server is determined for all the batch processes to be migrated, the process proceeds to S
Move from S302 to S309. In S309, S30
The processes in S310 to S311 are performed for each batch process in the conditional estimation answer selected in 1 and when the processes for all batch processes are completed, the process proceeds to S108 in FIG. S3
In step 10, a batch processing shift request is transmitted to the batch job scheduling daemon operating on the server determined in step S308, and in step S311, batch processing shift completion from the batch job scheduling daemon is received. If there is no server that can be processed in S307, a response indicating that processing is not possible is transmitted to the client in S312.

With the above operation, the server can be used efficiently. Further, in the present embodiment, a response indicating that processing is not possible is transmitted to the client in S312, but the processing may return to S301, select another server, and continue processing. As described above, in S301, the server having the smallest number of batch processes to be canceled is determined. However, since the deadline of the selected batch process to be canceled is tight, there is a server that can be processed before the deadline. There may be cases where they do not. For this reason, even if No in S307, S301
Returning to step S <b> 303, it is possible to newly select another stop target batch process that has a large deadline but has a margin of deadline, and perform the processing of step S <b> 303 and thereafter for the newly selected stop target batch process. Conceivable.

Embodiment 4 FIG. The configuration of the distributed batch system according to the fourth embodiment is the same as that of FIG. The operation of the present embodiment is almost the same as the operation of the second embodiment shown in FIG. 4, but the contents of the conditional estimation answer are different. In the conditional estimation response according to the second embodiment, the workload and the deadline of the batch processing requiring one shift are indicated, but in the present embodiment, the batch job scheduling daemon is continuously scheduled. If the requested process can be completed by the deadline unless one batch process group consisting of multiple batch processes is executed, the total work volume of the entire batch process group The deadline is transmitted to the manager 11 as a conditional estimate response.

The flow chart of the manager in this embodiment is the same as that in FIG. 3 of the second embodiment.

With the above operation, it is possible to use the server more efficiently than in the second embodiment, and it is possible to reduce the number of processes required for migration as compared with the third embodiment.

Embodiment 5 FIG. The configuration of the distributed batch system according to the fifth embodiment is the same as that of FIG. The operation of the present embodiment is almost the same as the operation of the second embodiment shown in FIG. 4, but the contents of the conditional estimation answer are different. The conditional estimation answer in the second embodiment indicates the workload and deadline of one batch process that needs to be migrated, but in the present embodiment, the batch job scheduling daemon
If the requested process can be completed by the deadline unless multiple batch processing groups consisting of multiple batch processes that are scheduled in succession are executed, the amount of work of the entire batch processing group can be reduced. total,
A deadline common to a plurality of batch processing groups is transmitted to the manager 11 as a conditional estimate response.

The flowchart of the manager 11 in the present embodiment is the same as that in FIG. 5 of the third embodiment.

With the above operation, the server can be used more efficiently than in the second embodiment.

Embodiment 6 FIG. The configuration of the distributed batch system according to the sixth embodiment will be described with reference to FIG. 7, a management server 1, processing servers 2A to 2N, and clients 3A to 3N are the same as those in FIG. In FIG.
Although the clients 3A to 3N have transmitted the batch processing request to the management server 1, in FIG.
A to 3N are different in that a batch processing request can be transmitted to any of the management server 1 and the processing servers 2A to 2N.

The operation of the distributed batch system according to this embodiment will be described with reference to FIGS. Here, it is assumed that the client 3K (not shown) transmits a batch processing request to the processing server 2L (not shown). The client 3K that requests batch processing is the processing server 2
A batch processing request is sent to the L batch job scheduling daemon 22L with the deadline and the workload of the batch processing added (S21). When the requested batch process can be processed within the deadline on the own server, the batch job scheduling daemon 22L operates as shown in FIG. 8, performs scheduling for the batch service 23L (S25), and returns the own server name. (S27). The client is the answering server 2L
, And a batch is input (S28). The batch service 23L is a batch job scheduling daemon 22
In accordance with the scheduling from L (S25), the input batch processing is executed, and the batch execution result is transmitted to the client 3K (S29).

On the other hand, the batch job scheduling daemon 22L operates as shown in FIG. 9 when the requested batch processing cannot be processed within the deadline on its own server. The batch job scheduling daemon 22L transmits a batch processing request to the manager 11 (S4).
1). The manager 11 is a batch job scheduling daemon 12, 22A- on each of the servers 1, 2A-2K, 2M-2N other than the server 2L that has sent the batch processing request.
22K, and sends an estimate request to 22M to 22N (S
22). The batch job scheduling daemon that receives the estimate request, schedule status of each batch service on its own server, deadline added to the request,
Calculate the completion time of the requested batch process from the amount of work,
If it can be completed within the deadline, the completion time is shown.
1 (S23). The manager 11 receives a response from each batch job scheduling daemon and selects the one with the earliest completion time from among the responses. here,
It is assumed that the batch job scheduling daemon 22M is selected. Next, the manager 11 transmits a scheduling request to the selected batch job scheduling daemon 22M (S24). On the other hand, if there is no server that can process, the server sends a response indicating that processing is impossible to the batch job scheduling daemon 22L (S27). Batch job scheduling daemon 2
The 2M performs scheduling on the corresponding batch service 23M (S25), and sends a notification of scheduling completion to the manager 11 (S26). The manager 11 sends the selected server as an answer to the batch job scheduling daemon 22L. The batch job scheduling daemon 22L sends the response received from the manager 11 to the client 3K (S27).
The client 3K submits a batch to the server 2M to which the response has been made (S28). The batch service 23M executes the input batch processing in accordance with the scheduling (S25) from the batch job scheduling daemon 22M, and transmits a batch execution result to the client (S29).

The batch job scheduling daemon 22L may operate as shown in FIG. 10 when the requested batch processing cannot be processed within the deadline on its own server. Batch job scheduling daemon 22
L indicates a work load of the batch process and a batch transfer indicating the deadline when the requested process can be completed by the deadline unless one of the scheduled batch processes is executed. The request is transmitted to the manager 11 (S35). Manager 11
Are batch job scheduling daemons 12, 22A to 22K and 22M to 22 on the servers 1, 2A to 2K and 2M to 2N other than the server 2L that has transmitted the batch transfer request.
Then, an estimation request is transmitted to N (S22). Each batch job scheduling daemon 12, 22A to 22K, 2
2M to 22N calculate the completion status of the requested batch process from the schedule status of each batch service on the own server, the deadline added to the request, and the workload, and if the completion is possible within the deadline, the completion time If it is not possible, the manager 11
(S23). The manager 11 manages each batch job scheduling daemon 12, 22A to 22K, 2
Responses from 2M to 22N are received, and the one with the earliest completion time is selected from the responses. Here, the batch job scheduling daemon 22M is selected. Next, the manager 11 transmits a batch transfer request to the selected batch job scheduling daemon 22M (S24). Upon receiving the batch transfer request, the batch job scheduling daemon 22M transfers the batch processing from the batch job scheduling daemon 22L, and when the transfer is completed, sends a batch transfer completion to the manager 11 (S36). The completion of batch transfer is transmitted to the scheduling daemon 22L (S46). The batch job scheduling daemon 22L performs scheduling for the corresponding batch service 23L (S25), and sends a response to the client 3K (S27). Client 3K
Submits a batch to the answered server (S2
8). The batch service 23L executes the input batch process in accordance with the scheduling (S25) from the batch job scheduling daemon, and transmits the batch execution result to the client (S29).

Although the operations of the management device and the data processing system according to the present invention have been described in the first to sixth embodiments, the management method according to the present invention can be implemented by the operation procedure shown in the first to sixth embodiments. Can be realized.

Here, the features of the present invention described above are summarized as follows. In a distributed batch job scheduling method according to the present invention, in a distributed batch system including a plurality of servers providing a plurality of batch services and a plurality of clients requesting a batch job, a batch for scheduling a batch service on each server is provided. One of the job scheduling daemon and the server is provided with a manager for coordinating between the client and the batch job scheduling daemon, and performs deadline management of batch processing.

The distributed batch job scheduling method is characterized in that when there is no server capable of processing a batch, scheduling is performed by transferring one batch process to another server.

The distributed batch job scheduling method is characterized in that when there is no server capable of processing a batch, scheduling is performed by transferring a plurality of batch processes to another server.

In the distributed batch job scheduling method, when there is no server capable of processing batches, scheduling is performed by transferring a plurality of batch processes that are continuously scheduled to another server as one batch process. It is characterized by.

In the distributed batch job scheduling method, when there is no server capable of processing batches, a plurality of batch processes scheduled continuously are treated as one batch process, and the plurality of batch processes are transferred to another server. Thus, scheduling is performed.

In the distributed batch job scheduling method according to the present invention, in a distributed batch system including a plurality of servers providing a plurality of batch services and a plurality of clients requesting a batch job, scheduling of the batch service is performed on each server. One of the batch job scheduling daemon and server that performs
Finally, a manager for adjusting the batch job scheduling daemon is provided, and the client performs a batch processing deadline management by transmitting a batch processing request to a batch job scheduling daemon on an arbitrary server.

In the distributed batch job scheduling method, the batch job scheduling daemon issues a batch processing request to the manager when a batch cannot be processed on the server that has transmitted the batch processing request, thereby achieving the deadline of batch processing. It is characterized by performing management.

In the distributed batch job scheduling method, when a batch cannot be processed on the server that has transmitted the batch processing request, the batch job scheduling daemon issues a batch transfer request to the manager, thereby causing the batch processing deadline. It is characterized by performing line management.

[0059]

As described above, according to the present invention, the management server inquires each processing server whether batch processing can be completed by the deadline, and from among the processing servers capable of processing by the deadline. Since a processing server that executes batch processing is selected, deadline management in a distributed batch system can be effectively performed.

According to the present invention, even if there is no processing server capable of processing before the deadline,
The management server shifts the batch processing between the processing servers based on the conditional estimation answer, and as a result, any one of the processing servers can complete the processing by the deadline. For this reason, the present invention can efficiently utilize the resources of each processing server.

According to the present invention, the management server selects and shifts the batch processing with the least amount of work.
The destination of the batch processing can be determined early.

Further, according to the present invention, since the management server selects and transfers the least number of batch processes, the destination of the batch process can be determined early.

Further, according to the present invention, the management server treats a plurality of batch processes as a group, so that the batch process transfer process can be made more efficient.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a distributed batch system.

FIG. 2 is a diagram illustrating an operation of the distributed batch system according to the first embodiment.

FIG. 3 is a flowchart illustrating an operation of a manager of the management server according to the first embodiment;

FIG. 4 is a diagram illustrating an operation of the distributed batch system according to the second embodiment.

FIG. 5 is a flowchart illustrating the operation of a manager of the management server according to the second embodiment.

FIG. 6 is a flowchart illustrating an operation of a manager of the management server according to the third embodiment.

FIG. 7 is a diagram showing a configuration of a distributed batch system.

FIG. 8 illustrates an operation of the distributed batch system according to the sixth embodiment.

FIG. 9 illustrates an operation of the distributed batch system according to the sixth embodiment.

FIG. 10 is a diagram illustrating an operation of the distributed batch system according to the sixth embodiment.

[Explanation of symbols]

1 management server, 2 processing servers, 3 clients,
10 communication unit, 11 manager, 12 batch job scheduling daemon, 13 batch service, 20
Communication unit, 22 batch job scheduling daemon, 23 batch service.

Claims (22)

[Claims] 1. A management device that manages a plurality of data processing devices that perform predetermined data processing based on a data processing request, and specifies a data processing device that performs data processing for each data processing request as a designated data processing device. A communication unit that receives, from at least one data processing device, prediction completion time information indicating a prediction completion time of data processing for each data processing request; and a designated data processing for each data processing request based on the prediction completion time information. A designation unit for designating a device and causing a designated data processing device to perform data processing based on a data processing request. 2. The communication unit according to claim 2, wherein at least one or more data processing devices complete term-limited data processing based on a term-limited data processing request in which a term of completion of data processing is determined before the completion term. Receiving the completion possible information notifying that the completion is possible, the designation unit designates any one of the data processing devices that transmitted the completion complete information as the designated data processing device that performs the time-limited data processing The management device according to claim 1, wherein: 3. The time-limited data processing when the communication unit stops executing at least one or more data processes scheduled to be executed by each of the data processing devices from at least one or more data processing devices. Receiving the conditional completion possible information notifying that the completion can be completed before the completion deadline, and the specifying unit transmits any one of the data processing apparatuses among the data processing apparatuses that transmitted the conditional completion feasibility information. The management apparatus according to claim 2, wherein the management apparatus is designated as a designated data processing apparatus that performs the time-limited data processing. 4. The conditional completion completion information including information on at least one or more data processing units to be aborted in each of the data processing devices from at least one or more data processing devices. The designation unit selects any one of the data processing devices that transmitted the conditional completion possible information as a candidate data processing device, and selects other data processing units to be canceled by the candidate data processing device. Judge whether it is possible to shift to any of the data processing devices, and if the data processing to be stopped of the candidate data processing device can be shifted to any other data processing device, the candidate data processing device 4. The apparatus according to claim 3, wherein the apparatus is designated as a processing apparatus.
Management apparatus according to. 5. The communication unit receives, from at least one or more data processing devices, conditional completion enable information indicating a data processing amount of a data process to be canceled, and the designation unit receives the conditional completion. 5. The candidate data processing device is selected based on a data processing amount of the data to be canceled indicated in the possible information.
Management apparatus according to. 6. The communication unit receives, from at least one or more data processing devices, conditional completion enable information indicating the number of data processes to be aborted, and the designation unit receives the conditional completion enable information. 5. The management device according to claim 4, wherein the candidate data processing device is selected based on the number of the data processing to be stopped indicated in (b). 7. The communication unit receives, from one of the data processing devices other than the candidate data processing device, candidate prediction completion time information for notifying the prediction completion time of the stop target data processing of the candidate data processing device. The designation unit is configured to, based on the candidate prediction completion time information, set any one of the data processing devices that transmitted the candidate prediction completion time information to a migration destination of the candidate data processing device to be canceled. The management device according to claim 4, wherein the management device is designated. 8. The communication unit receives, from at least one or more data processing devices, conditional completion possible information indicating at least one or more stop target data processes and a completion deadline of each stop target data process. The designation unit may include any one of the data processing devices that transmitted the conditional completion enable information indicating at least one or more stop target data processes and the completion deadline of each stop target data process. Select as the candidate data processing device, the communication unit, from any one of the data processing device other than the candidate data processing device, each stop target data processing of the candidate data processing device before the completion deadline of each stop target data Receiving the candidate completion possible information for notifying that the candidate completion can be completed, the designation unit includes any one of the data processing apparatuses that transmitted the candidate completion possible information The management apparatus according to claim 4, wherein the data processing apparatus is designated as a migration destination of the data processing to be canceled of the candidate data processing apparatus. 9. The communication unit according to claim 1, wherein at least one or more data processing devices indicate a plurality of stop target data processes and a common completion time limit commonly defined for the plurality of stop target data processes. The designation unit receives any of the data processing devices that transmitted conditional completion possible information indicating a plurality of stop target data processes and a common completion deadline for the plurality of stop target data processes. The communication unit may be selected as a candidate data processing device, and the communication unit may complete a plurality of stop target data processes of the candidate data processing device by a data processing device other than the candidate data processing device before a common completion deadline. Receiving the candidate completion possible information for notifying that the candidate completion possible information is transmitted, and 5. The management apparatus according to claim 4, wherein the apparatus is designated as a migration destination of a plurality of data to be canceled of the candidate data processing apparatus. 10. The management device is connected to at least one or more data processing request devices that transmit a data processing request, and the communication unit receives a data processing request from one of the data processing request devices, 2. The management apparatus according to claim 1, wherein a request for calculating a prediction completion time of data processing based on the received data processing request is transmitted to each data processing apparatus. 11. The communication unit receives a data processing request from one of the data processing devices, and based on a data processing request received from a data processing device other than the data processing device that transmitted the data processing request. The management device according to claim 1, wherein the management device transmits a calculation request for a prediction completion time of the data processing. 12. The management apparatus further calculates a data processing unit that performs predetermined data processing based on a data processing request, calculates a prediction completion time of data processing by the data processing unit, and automatically calculates the calculated prediction completion time. A completion time prediction unit that outputs to the designation unit as device prediction completion time information, wherein the designation unit acquires the own device prediction completion time information from the completion time prediction unit, and acquires the acquired own device prediction completion time. The management according to claim 1, wherein the designated data processing device is designated from among the plurality of data processing devices and the management device based on information and the prediction completion time information received by the communication unit. apparatus. 13. The management device, further comprising: a data processing unit that performs predetermined data processing based on a data processing request; and whether the data processing unit can complete the time-limited data processing before the completion time limit. Whether or not the data processing unit outputs self-apparatus completion enable information to the designation unit to notify that the time-limited data processing can be completed before the completion time limit according to the determination result. Having a determination unit, the designation unit acquires the own device completion possibility information from the completion possibility determination unit, based on the acquired own device completion possibility information and the completion possibility information received by the communication unit, 3. The designated data processing device is designated from among the plurality of data processing devices and the management device.
Management apparatus according to. 14. A designated data processing device comprising: a plurality of data processing devices for performing predetermined data processing based on a data processing request; and a data processing device for managing the plurality of data processing devices and performing data processing for each data processing request. A data processing system comprising: a management device that designates the data processing device as a data processing system, the management device comprising: Transmits a completion permission / inquiry inquiry as to whether or not the completion can be completed before the completion deadline.Each data processing device measures and estimates the predicted completion time of the time-limited data processing based on the completion permission / inhibition inquiry. Based on the predicted completion time, it is determined whether the time-limited data processing can be completed before the completion time limit. In the case where the time-limited data processing can be completed before the completion time limit, the completion including the predicted completion time and notifying that the time-limited data processing can be completed before the completion time limit can be completed. Transmitting the information to the management device, and the management device transmits one of the data processing devices that transmitted the completion possibility information to the management device based on the predicted completion time included in the completion possibility information with the time limit. A data processing system, which is designated as a designated data processing device for performing data processing. 15. When each data processing device cannot complete the time-limited data processing before the completion time limit, each data processing device executes at least one or more data processes scheduled to be executed by each data processing device. If it is determined that the time-limited data processing can be completed before the completion time limit, the execution of at least one or more data processing scheduled to be performed is stopped. If the processing can be completed before the completion deadline, the information including at least one or more stop target data processing to be canceled and including the information on the stop target data processing, if the execution of the stop target data processing is stopped, the time-limited data The management device transmits conditional completion possible information notifying that the process can be completed before the completion deadline, and the management device Select one of the data processing devices that have transmitted the completion possibility information as a candidate data processing device, and determine whether or not the candidate data processing device's suspension target data processing can be transferred to any other data processing device Determining whether the candidate data processing device can be transferred to any of the other data processing devices by the candidate data processing device, and specifying the candidate data processing device as the designated data processing device. 15. The data processing device according to 14. 16. Each data processing device is connected to at least one or more data processing request devices for transmitting a data processing request, and each data processing device has a data processing completion deadline from one of the data processing request devices. Receiving a data processing request with a fixed time limit, determining whether it is possible to complete the data processing with a time limit based on the received data processing request with a time limit before the completion time limit, and When the time-limited data processing cannot be completed, the time-limited data processing request is transferred to the management device; and the management device is a data processing device other than the data processing device that has transferred the time-limited data processing request. In response to this, a completion availability inquiry asking whether or not the time-limited data processing can be completed before the completion time limit is issued. Data processing system according to claim 14, characterized in that the signal. 17. Each data processing device is connected to at least one or more data processing request devices for transmitting a data processing request, and each data processing device has a data processing completion deadline from one of the data processing request devices. Receiving a data processing request with a fixed time limit, determining whether it is possible to complete the data processing with a time limit based on the received data processing request with a time limit before the completion time limit, and When the data processing with a term cannot be completed, if the execution of at least one or more data processing scheduled by each data processing device is stopped, the data processing with a term is completed before the completion time limit. Judge whether or not the processing of at least one or more scheduled data processing is stopped, and When the completion can be completed before the completion deadline, a data processing request requesting that the at least one or more stop target data processing to be canceled is completed before a predetermined completion deadline to the management device. Transmitting, the management device receives a data processing request of the stop target data processing from each data processing device, and for a data processing device other than the data processing device that has transmitted the data processing request of the stop target data processing, 15. The data processing system according to claim 14, wherein a completion permission inquiry is transmitted to inquire whether the suspension target data processing can be completed before the completion deadline. 18. A management method for managing a plurality of data processing devices that perform predetermined data processing based on a data processing request and designating a data processing device that performs data processing for each data processing request as a designated data processing device. A communication step of receiving, from at least one data processing device, prediction completion time information indicating a prediction completion time of data processing for each data processing request; and a designated data processing for each data processing request based on the prediction completion time information. And a designation step of designating a device and causing the designated data processing device to perform data processing based on the data processing request. 19. The communication step, wherein at least one or more data processing devices complete term-limited data processing based on a term-limited data processing request in which a term of completion of data processing is determined before the completion term. Receiving the completion possible information notifying that the completion possible information is transmitted, and the specifying step specifies one of the data processing apparatuses transmitting the completion possible information as the specified data processing apparatus that performs the time-limited data processing The management method according to claim 18, wherein: 20. The time-limited data processing when the execution of at least one or more data processing scheduled to be executed by each of the at least one data processing apparatus is stopped by the at least one or more data processing apparatuses. Receiving the conditional completion possible information notifying that the completion can be completed before the completion deadline, and the specifying step includes any one of the data processing apparatuses that transmitted the conditional completion feasibility information. 20. The management method according to claim 19, wherein the management method is designated as a designated data processing device that performs the time-limited data processing. 21. The communication step, wherein at least one or more data processing devices execute conditional completion possible information including information on at least one or more stop target data processes whose execution is to be stopped in each data processing device. And the specifying step selects any one of the data processing devices that have transmitted the conditional completion possible information as a candidate data processing device, and selects another candidate data processing of the candidate data processing device. Judge whether it is possible to shift to any one of the data processing devices, and when the data processing to be stopped of the candidate data processing device can be shifted to any other data processing device, the candidate data processing device 21. The management method according to claim 20, wherein the management method is designated as a processing device. 22. A computer which manages a plurality of data processing apparatuses which perform predetermined data processing based on a data processing request and designates a data processing apparatus which performs data processing for each data processing request as a designated data processing apparatus. A program for executing, based on the prediction completion time information, a communication step of receiving, from at least one or more data processing devices, prediction completion time information indicating a prediction completion time of data processing for each data processing request. A program for causing a computer to execute a management method having a designation step of designating a designated data processing device for each data processing request and causing the designated data processing device to perform data processing based on the data processing request.