JP2009151376A - Distribution processing method, calculator management device, and distribution processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To optimize allocation of calculators to jobs, without worsening calculation efficiency. <P>SOLUTION: The jobs are stored at first, the number of calculators for calculating the stored first job at the shortest time is drawn out thereafter as the optimum number, when conducting distribution processing in a distribution processing system 1 provided with the plurality of calculators 20. Then, the number of waiting calculators 20 under a job waiting state is determined as to the propriety of satisfying the optimum number or more. The optimum number of calculators is selected out of the calculators 20 under the job waiting state, when the number of waiting calculators is the optimum number or more, and the first job is distribution-processed by the selected calculators 20. A processed result of the first job is output based on a distribution-processed result. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a distributed processing method, a computer management apparatus, and a distributed processing system that cause a plurality of computers connected via a network to distribute and execute calculations.

  Distributed computing has been adopted in a wide range of fields because large-scale calculation results can be obtained at high speed. There are two main processing methods when a job that is a calculation request from a user is given.

  In the sequential processing method, which is one of them, the server distributes jobs in the order of acceptance using all computers, and starts calculating the next job when processing of one job is completed. That is, in the sequential processing method, only one job is calculated at a time. In this case, if there are large and small variations in the job scale, inequality between the waiting time and the processing time becomes a problem, such as a small job waiting for the calculation of a large job.

In order to solve this problem, there is a parallel calculation method in which computers are allocated in accordance with the job scale and the calculations are performed in parallel. For example, Patent Document 1 below discloses a distributed processing system that realizes job distribution in consideration of a job completion deadline. This distributed processing system includes a plurality of computers that share operation information, predicts the execution completion time of a job input to one computer, and requests other computers to execute the job according to the prediction result.
JP 2003-256222 A

  However, in the parallel processing method, if the computer allocation is not designed appropriately in accordance with the job size and occurrence frequency characteristics, the operation rate of the computer deteriorates and the calculation efficiency deteriorates. For example, if the assignment of computers to large and small jobs is fixed, each computer will function effectively if large and small jobs are given equally, but if the number of large and small jobs is unbalanced, the calculation efficiency will deteriorate. End up. For example, when a large job is continuously given, a situation occurs in which a computer assigned to a small job does not operate. On the other hand, if the assignment of jobs to computers is variable, even if the number of large and small jobs is unbalanced, all computers will always operate for any job, so the calculation efficiency will not deteriorate. However, creating a program that can change the assignment of computers during distributed processing is very complicated.

  The present invention has been made to solve the above problems, and provides a distributed processing method, a computer management apparatus, and a distributed processing system capable of optimizing the allocation of computers to jobs without reducing the calculation efficiency. The purpose is to do.

  The distributed processing method of the present invention is a distributed processing method in a distributed processing system including a plurality of computers, and includes a storage step for storing a job, and the number of computers capable of calculating the job stored in the storage step in the shortest time. Derivation step that derives as the optimal number of units, a determination step that determines whether the number of waiting computers, which is the number of computers waiting for a job, is greater than or equal to the optimal number derived in the derivation step, A selection step for selecting the optimal number of computers from the computers in a job waiting state when it is determined that there is a processing step for distributing the jobs stored in the storage step to the computers selected in the selection step; Based on the result of distributed processing in the processing step, stored in the storage step Characterized in that it comprises an output step of outputting the processing result of the job, the.

  The computer management apparatus of the present invention is a computer management apparatus capable of communicating with a plurality of computers, and includes a storage means for storing jobs, and the number of computers capable of calculating the jobs stored in the storage means in the shortest time. Deriving means for deriving as the optimal number, determining means for determining whether or not the number of waiting computers as the number of computers in the job waiting state is equal to or larger than the optimal number derived by the deriving means, If it is determined that there is a selection unit that selects the optimal number of computers from the computers in a job waiting state, and a processing unit that distributes the job stored in the storage unit to the computer selected by the selection unit; Output means for outputting the processing result of the job stored in the storage means based on the result of the distributed processing by the processing means.

  Further, the distributed processing system of the present invention is a distributed processing system having a plurality of computers, and the optimum number of storage means for storing jobs and the number of computers capable of calculating the jobs stored in the storage means in the shortest time. Deriving means derived as follows, determining means for determining whether the number of waiting computers as the number of computers waiting for a job is equal to or larger than the optimum number derived by the deriving means, and determining that the waiting number is greater than or equal to the optimum number A selection unit that selects the optimal number of computers from the computers in the job waiting state, a processing unit that distributes the jobs stored in the storage unit in the computer selected by the selection unit; Output means for outputting the processing result of the job stored in the storage means on the basis of the result of the distributed processing according to the above.

  According to such a distributed processing method, a computer management apparatus, and a distributed processing system, when more than the optimal number of computers capable of calculating a job in the shortest time is in the job waiting state, the computer in the job waiting state The optimal number of computers is selected from the above, and jobs are distributedly processed by the selected computers. This makes it possible to optimize the assignment of computers to jobs without affecting the computers that are executing the calculation (without reducing the calculation efficiency).

  In the distributed processing method of the present invention, in the determination step, when the number of waiting machines is less than the optimum number, the number of computers waiting for a job is stored within the addition time of the arrival time and the shortest time until the number of computers waiting for the job reaches the optimum number. It is determined whether or not the job stored in the step can be calculated by the computer waiting for the job. In the selection step, if it is determined in the determination step that the job can be calculated by the computer waiting for the job within the addition time, It is preferable to select all the computers waiting for the job.

  Further, in the computer management apparatus of the present invention, when the waiting number is less than the optimum number, the determination means is within the addition time of the time until the number of computers waiting for the job reaches the optimum number and the shortest time, When it is determined whether or not the job stored in the storage means can be calculated by the computer waiting for the job, and the selection means determines that the determination means can calculate the job by the computer waiting for the job within the addition time It is preferable to select all the computers waiting for the job.

  In this case, if the number of computers waiting for a job is less than the optimum number, whether or not the job can be processed within the addition time of the shortest time until the number of computers currently waiting for a job reaches the optimum number Is further determined. If it is determined that the processing is possible, all the computers in the current waiting state are selected and the distributed processing is executed without waiting until the number of computers in the job waiting state reaches the optimum number. This makes it possible to complete job processing more quickly by using the computer more effectively.

  In the distributed processing method of the present invention, in the storage step, one or more other jobs are further stored, and in the determination step, if it is determined that the job cannot be calculated by the computer in the job waiting state within the addition time, In the selection step, if it is determined that there is another job that can be calculated in the determination step, the computer waiting for the job is selected. When all are selected and it is determined in the determination step that there is another job that can be calculated in the determination step, it is preferable that the different jobs that can be calculated are distributedly processed by the computer selected in the selection step.

  In the computer management apparatus of the present invention, the storage means further stores one or more other jobs, and the determination means arrives when it is determined that the job cannot be calculated by the computer in the job waiting state within the addition time. It is determined whether there is another job that can be calculated by the computer waiting for the job within the time, and if the selection unit determines that there is another job that can be calculated by the determination unit, the computer waiting for the job When it is determined that there is another job that can be calculated by the determination unit, it is preferable that the computer selected by the selection unit distributes the other job that can be calculated.

  In this case, if it is determined that one computer that is currently waiting for a job cannot be processed within the addition time, it is further determined whether there is another job that can be processed within the arrival time. When such another job exists, all computers that are currently waiting are selected, and distributed processing for the other job is executed. This makes it possible to shorten the job execution time as a whole by using the computer more effectively.

  In the distributed processing method of the present invention, using the function defined by the coefficient indicating the job calculation scale and the number of computers, the number of jobs stored in the storing step is calculated for the number of computers equal to or less than the total number of computers. It is preferable to further include a calculation step for calculating a calculation time in the case where the calculation time is calculated, and in the derivation step, the number corresponding to the shortest of the plurality of calculation times calculated in the calculation step is derived as the optimum number.

  In the computer management apparatus of the present invention, the number of jobs stored in the storage means is stored for each number of computers equal to or less than the total number of computers, using a function defined by a coefficient indicating the job calculation scale and the number of computers. It is preferable to further include calculation means for calculating the calculation time in the case of calculating in (4), and the derivation means derives the number corresponding to the shortest of the plurality of calculation times calculated by the calculation means as the optimum number.

  In this case, the calculation time is calculated using a function defined by the coefficient indicating the job calculation scale and the number of computers. This calculation is performed for each case where 1 to a total number of computers are used. Then, the number corresponding to the shortest calculation time among the plurality of calculated times is derived as the optimum number. Since the job calculation time depends on the job calculation scale and the number of computers, the optimum number can be derived more accurately by using a function defined by the calculation scale and the number of computers.

  According to such a distributed processing method, computer management apparatus, and distributed processing system, a computer that processes a job is based on a comparison between the number of computers that can calculate a job in the shortest time and the number of computers in a job waiting state. Since it is selected, it is possible to optimize the assignment of computers to jobs without reducing the calculation efficiency.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

  First, the distributed processing system 1 according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram showing the overall configuration of the distributed processing system 1. FIG. 2 is a diagram showing a hardware configuration of the computer management apparatus 10 shown in FIG. FIG. 3 is a diagram showing the configuration of queue management information. FIG. 4 is a diagram showing a configuration of computer information.

  The distributed processing system 1 is a computer system capable of processing a plurality of jobs in parallel, and includes a computer management apparatus 10 and N computers 20. The computer management apparatus 10 and each computer 20 can communicate with each other via a network (not shown) (for example, the Internet or a local area network (LAN)).

  The computer management apparatus 10 assigns a job to each computer 20 to execute distributed processing, and generates and outputs a job processing result based on the result of the distributed processing. For this purpose, the computer management apparatus 10 includes an input unit 11, a queue management unit (storage unit) 12, a computer management unit 13, a calculation time prediction unit (calculation unit) 14, and a distributed processing start unit (derivation unit, determination unit, selection unit). , Processing means) 15, result receiving section 16 and output section (output means) 17.

  As shown in FIG. 2, the computer management apparatus 10 includes a CPU 101 that executes an operating system, application programs, and the like, a main storage unit 102 that includes a ROM and a RAM, an auxiliary storage unit 103 that includes a hard disk, a network card, and the like Communication control unit 104, an input interface (I / F) 105 such as a keyboard and a mouse, and an output interface (I / F) 106 such as a monitor and a printer. Each function shown in FIG. 1 reads predetermined software on the CPU 101 and the main storage unit 102 shown in FIG. 2 and operates the communication control unit 104 under the control of the CPU 101, This is realized by reading and writing data in the storage unit 103.

  The input unit 11 is a unit that receives a job input as a user calculation request and outputs the job to the queue management unit 12. The input unit 11 may directly accept a job input by a user, or may accept a job input from another information processing apparatus (not shown).

  The queue management unit 12 is means for storing (registering) a job input from the input unit 11 using a queue. The queue management unit 12 holds queue management information indicating the queue status (status of each job).

  As shown in FIG. 3, the queue management information includes a job number for identifying a job, a job reception date and time, a calculation status indicating a job status, a calculation start date and time, a calculation condition, and a calculation time prediction table. Job numbers are assigned in ascending order from the job with the oldest stored date. The calculation status is indicated, for example, as “calculation completed”, “calculating (in x units)”, “waiting for calculation”, or the like. The calculation condition is a parameter necessary for job calculation. The calculation time prediction table is configured with the predicted calculation time of the job calculated for each number of computers (for each of 1 to N). For example, the calculation time when the job of job number 2 in FIG. 3 is calculated by one computer 20 is predicted to be x (seconds), and the calculation time when the job is calculated by N computers 20 is x ′. '(Second) is predicted.

  When a job is input from the input unit 11, the queue management unit 12 stores the job and also generates and stores queue management information for the job. Thereafter, the queue management unit 12 updates the queue management information based on the job information input from the distributed processing start unit 15 or the result reception unit 16. When job information is input from the distributed processing start unit 15, the queue management unit 12 updates the calculation status of the corresponding job to “calculating in x units (x = 1 to N)” and the calculation start date and time. . When job information is input from the result receiving unit 16, the queue management unit 12 updates the calculation status of the corresponding job to “computation completed”.

  The calculation time prediction unit 14 is a means for calculating (predicting) the calculation time of the job based on the contents of the job stored in the queue management unit 12. The calculation time prediction unit 14 calculates the calculation time for each number of computers equal to or less than the total number of computers 20 in the distributed processing system 1.

For example, the calculation time prediction unit 14 calculates the calculation time using a prediction function represented by the following formula (1). In Equation (1), x is a coefficient indicating the job calculation scale (calculation scale coefficient), and n is the number of computers used for calculation (distributed number). That is, the prediction function is defined by the calculation scale factor and the number of distributed units.
f (x, n) = x / n + 0.1nx (1)

  The calculation scale factor x is determined based on calculation conditions, and a different value is set depending on the type of job. For example, in a system for estimating the service area of a mobile phone (estimation system), when calculating the radio wave reception intensity at a certain point, the calculation scale factor can be expressed by the number of points to be calculated.

  The calculation time prediction unit 14 can also use a prediction function f (x1, x2, n) including two types of calculation scale factors. For example, the calculation time prediction unit 14 can predict the calculation time of the estimation system using two types of calculation scale factors, that is, the number of points to be calculated and the number of base stations.

  The right side of the prediction function f (x1, x2, n) can be determined using multivariate analysis. Therefore, the prediction function is determined by a parameter that is a calculation condition of a system to be applied (for example, the estimation system described above) and multivariate analysis. The right side of the above formula (1) is also determined by multivariate analysis.

Further, the calculation time prediction unit 14 may use a calculation time function different from the prediction function such as Expression (1). The calculation time prediction unit 14 can use the calculation time function when the relationship between the calculation condition and the calculation time is known in advance in the application target system. For example, in a system to be applied, a certain time a is necessarily required to solve one event (problem), and when the event is distributed to n computers, the calculation time becomes 1 / n. However, it has been found that one-tenth of the calculation time takes as much as the number of distributed units n as a loss of distributed processing. In this case, the calculation time prediction unit 14 can calculate the calculation time using a calculation time function represented by the following formula (2). In addition, x 'in Formula (2) is the number of events (a kind of calculation scale factor), and n is a distributed number. That is, the calculation time function is defined by the number of events to be solved and the number of distributed units.
f (x ′, n) = a (x ′ / n + 0.1nx ′) (2)

  When the queue management unit 12 stores a job, the calculation time prediction unit 14 acquires information about the job (for example, calculation conditions and a calculation scale) from the queue management unit 12, and calculates the calculation time for the job from the distributed number 1 Calculate for each of N. Then, the calculation time prediction unit 14 stores the calculation result in the calculation time prediction table of the queue management information.

  The computer management unit 13 is a means for storing the state of each of the N computers 20. The computer management unit 13 holds computer information as shown in FIG. 4 and updates the computer information as necessary.

  The computer information is set for each computer 20 and includes a computer number, a calculation state, and other information. The computer number is an identifier of the computer 20. The calculation state is information indicating current information of the computer 20. For example, values such as “under calculation” indicating that the current job is being processed and “waiting for calculation (waiting for job)” indicating that the current job is not being processed are set as the calculation state. Examples of the other information include an IP address as shown in FIG. 4, but other information (for example, a machine name or a model name) may be set as the other information.

  The computer management unit 13 updates the computer information based on the information of the computer 20 input from the distributed processing start unit 15 or the result reception unit 16. When computer information is input from the distributed processing start unit 15, the queue management unit 12 updates the calculation state of the corresponding computer 20 to “under calculation”, and when computer information is input from the result receiving unit 16, The calculation state of the computer 20 is updated to “waiting for calculation”.

  The distributed processing start unit 15 is a unit that divides a job stored in the queue management unit 12 and transmits the divided job (hereinafter referred to as “divided job”) to a computer in a job waiting state.

  First, the distributed processing start unit 15 refers to the computer information stored in the computer management unit 13 and determines whether there is a computer 20 in a job waiting state. At this time, if such a computer 20 does not exist, the distributed processing start unit 15 does nothing until the calculation is completed in any of the computers 20.

On the other hand, when there is a computer 20 in the job waiting state, the distributed processing start unit 15 sets the job registered first among unprocessed jobs stored in the queue management unit 12 (hereinafter “first job”). The number of computers 20 that can be calculated in the shortest time is derived as the optimum number. For example, when the queue management information is in the state shown in FIG. 3, the distributed processing start unit 15 refers to the calculation time prediction table of the job having the job number 4, and selects the number with the minimum predicted calculation time as the optimum number. . In the following description, the optimal prediction computation time corresponding to the optimum number calculating time (shortest time) that T _o.

Incidentally, it may number corresponding to the optimum calculation time T _o is more derived. In this case, the distributed processing starting unit 15, the optimum number of smallest number among the number of units expected to calculation in optimization calculation time T _o. For example, if the calculation time is the shortest when a certain job is calculated by four or five computers, the distributed processing start unit 15 sets the optimum number to four.

  Subsequently, the distributed processing start unit 15 determines whether or not the waiting number, which is the number of computers 20 in the job waiting state, is equal to or greater than the optimum number. In the computer information shown in FIG. 4, computer numbers 3 and 4 whose calculation state is “waiting for calculation” correspond to computers in the job waiting state. When the waiting number is equal to or greater than the optimal number, the distributed processing start unit 15 selects the optimal number of computers 20 from the computers 20 in the job waiting state. Subsequently, the distributed processing start unit 15 divides the earliest job in order to cause the selected computer 20 to distribute the earliest job, and transmits the divided job to each of the selected computers 20.

On the other hand, when the number of waiting machines is less than the optimum number, the distributed processing start unit 15 calculates the time required for the number of computers waiting for a job to reach the optimum number (hereinafter referred to as “arrival time T _a ”). _a and the optimal calculation time for adding the (shortest time) _{T o.} Distributed processing start unit 15 can derive an arrival time T _a by calculating the time until the process end for one or more jobs. For example, if the calculation start time and the current time of a job are 10:00 and 10:02, respectively, and the processing time of the job is 180 seconds, the computer executing the job waits for calculation after 60 seconds. It turns out that it will be in a state. Optionally, distributed processing start unit 15 calculates an arrival time T _a based on the processing time of a plurality of jobs.

Subsequently, the distributed processing start unit 15 determines whether the time (hereinafter referred to as “second calculation time T _s ”) when the first job is calculated using all the computers waiting for the job is equal to or shorter than the addition time (T _a + T _o ). Determine whether or not.

Here, if T _s ≦ T _a + T _o , the distributed processing start unit 15 determines that the earliest job can be calculated by the computer in the job waiting state within the addition time (T _a + T _o ), and the job waiting state All the computers 20 are selected. Subsequently, the distributed processing start unit 15 divides the earliest job in order to cause the selected computer 20 to distribute the earliest job, and transmits the divided job to each of the selected computers 20.

On the other hand, T _s> if a _T a _{+ T o,} distributed processing starting unit 15 there is another job that can be calculated by the computer 20 of the job waiting in the arrival time T _a (unprocessed job other than the earliest job) It is determined whether or not to do. Specifically, the distributed processing start unit 15 calculates a time (second calculation time T _t ) when such a different job is calculated using all the computers 20 in the job waiting state, and T _t It determines ≦ _{T a} or not. The distributed processing start unit 15 executes this determination processing in order from another job with the oldest reception date.

Here, when T _t ≦ T _a for a certain other job, the distributed processing start unit 15 selects all the computers 20 in the job waiting state. Subsequently, the distributed processing start unit 15 divides the other job in order to cause the selected computer 20 to distribute the different job, and transmits the divided job to each of the selected computers 20. On the other hand, if T _t > T _a for all other jobs, the distributed processing start unit 15 does nothing.

  When the distributed processing start unit 15 transmits the divided job to the selected computer 20, the distributed processing start unit 15 outputs information on the job (first job or another job) to be distributed to the queue management unit 12 and transmits the divided job transmission destination. Is output to the computer management unit 13.

  The result receiving unit 16 is a means for generating a job processing result based on the calculation result of the divided job transmitted from the computer 20, that is, the result of the distributed processing. When the result receiving unit 16 receives all the calculation results of the divided jobs for a certain job, the result receiving unit 16 generates a job calculation result from the calculation result and outputs the job calculation result to the output unit 17. When the result receiving unit 16 receives the calculation result of the divided job, the result receiving unit 16 transmits the information of the computer 20 that is the transmission source of the calculation result to the computer managing unit 13. Further, when all the divided jobs for a certain job are received, the result receiving unit 16 outputs the job information to the queue management unit 12.

  The output unit 17 is means for outputting the calculation result input from the result receiving unit 16 to an output device such as a monitor or a printer.

  The computer 20 processes the job received from the computer management apparatus 10 and transmits the processing result to the computer management apparatus 10. For this purpose, the computer 20 includes a job receiving unit 21, a job processing unit 22, and a result transmitting unit 23. The hardware configuration of the computer 20 is the same as that shown in FIG.

  The job receiving unit 21 is means for receiving a divided job from the computer management apparatus 10. The job receiving unit 21 outputs the received divided job to the job processing unit 22.

  The job processing unit 22 is means for processing the divided job. The job processing unit 22 performs a predetermined calculation based on the input divided job and generates a calculation result. The job processing unit 22 outputs the generated calculation result to the result transmission unit 23.

  The result transmission unit 23 is a unit that transmits the calculation result of the divided job input from the job processing unit 22 to the computer management apparatus 10.

  Next, the operation of the distributed processing system shown in FIG. 1 will be described with reference to FIGS. 5 and 6, and the distributed processing method according to the present embodiment will be described. 5 and 6 are flowcharts for explaining the processing of the distributed processing system.

  First, in the computer management apparatus 10, the input unit 11 receives an input of a job, and the queue management unit 12 registers the received job (step S11, storage step). Subsequently, the calculation time prediction unit 14 calculates the calculation time of the registered job and generates a calculation time prediction table (step S12, calculation step). Specifically, the calculation time prediction unit 14 uses a predetermined function (for example, the above formula (1) or formula (2)) for each number of computers with the number of connected computers 20 as an upper limit. And the calculation result is recorded in the calculation time prediction table in the queue management information.

  Subsequently, the distributed processing start unit 15 determines whether there is a computer 20 in a job waiting state (step S13). At this time, if such a computer 20 does not exist (step S13; NO), the process ends. On the other hand, if there is a computer 20 in the job waiting state (step S13; YES), the distributed processing start unit 15 derives the optimum number of the earliest jobs based on the calculation time prediction table of the earliest job (step S14, derivation). Step). Subsequently, the distributed processing start unit 15 determines whether or not the number of computers 20 in the job waiting state (waiting number) is equal to or greater than the derived optimum number (step S15, determination step), and the number of waiting units is equal to or greater than the optimum number. If there is (step S15; YES), the optimal number of computers are selected from the computers waiting for the job (step S16, selection step).

  Subsequently, the distributed processing start unit 15 divides the earliest job and starts the distributed processing by transmitting the divided job to each of the selected computers 20 (step S17, processing step). At this time, the queue management unit 12 updates the queue management information of the earliest job, and the computer management unit 13 updates the computer information of the selected computer 20 (step S18). As a result, it is stored that the earliest job is being processed and that calculations are being performed in the optimal number of computers 20.

  Thereafter, the divided job is calculated in each selected computer 20 (step S19). Subsequently, the result receiving unit 16 of the computer management apparatus 10 acquires the calculation result of the divided job, and generates the calculation result of the earliest job based on the calculation result (step S20, output step). At this time, the queue management unit 12 updates the queue management information of the earliest job, and the computer management unit 13 updates the computer information of the computer 20 that has executed the calculation (step S21). As a result, it is stored that the earliest job has been processed and that the optimal number of computers 20 have transitioned to the job waiting state. Subsequently, the output unit 17 outputs the calculation result generated by the result receiving unit 16 (step S22, output step).

  If there is another job waiting for calculation after the distributed processing of the earliest job is executed (step S23; YES), the processing of steps S14 to S22 is executed for the new earliest job. .

  On the other hand, when the number of computers 20 waiting for a job is less than the optimum number in step S15 (step S15; NO), the processing shown in FIG. 6 is executed.

  In this case, first, the distributed processing start unit 15 calculates the time (arrival time) during which the number of computers in the job waiting state reaches the optimum number (step S31). Subsequently, the distributed processing start unit 15 acquires the time (second calculation time) when the earliest job is calculated using all the computers in the job waiting state from the queue management information (step S32). Then, the distributed processing start unit 15 determines whether or not the second calculation time is equal to or shorter than the addition time of the arrival time and the optimal calculation time (step S33, determination step). At this time, if the second calculation time is equal to or shorter than the addition time (step S33; YES), the distributed processing start unit 15 determines that the first job can be calculated by the computer 20 in the job waiting state within the addition time, and the job All the waiting computers 20 are selected (step S34, selection step). Thereafter, the processes after step S17 are executed for the earliest job.

  On the other hand, when the second calculation time is longer than the addition time (step S34; NO), the distributed processing start unit 15 determines whether there is another job that can be calculated by the computer 20 in the job waiting state within the arrival time. (Step S35, determination step). Specifically, the distributed processing start unit 15 calculates the second calculation time for an unprocessed job other than the earliest job, and executes the determination by comparing the second calculation time with the arrival time. At this time, if there is another job that can be calculated (step S35; YES), the distributed processing starting unit 15 selects all the computers 20 in the job waiting state (step S34, selection step), and thereafter The process after step S17 is performed. On the other hand, if such another job does not exist (step S35; NO), the process ends.

  Next, a specific example of the distributed processing method will be described with reference to FIGS. FIG. 7 shows a specific example of queue management information. FIG. 8 is a diagram showing a specific example of computer information. FIG. 9 is a diagram showing the operating status of five computers in time series. 7 and 8, the description of information not directly related to the explanation is omitted. On the other hand, in FIG. 7, the calculation scale factor is shown for explanation.

  In the following description, it is assumed that the distributed processing system 1 has five computers (computers C1 to C5). The calculation time prediction table of FIG. 7 stores the prediction calculation time (minutes) calculated using the prediction function of the above equation (1). Note that the predicted calculation time and the actual calculation time are the same.

  As shown in FIG. 9, at the time of 9:00 when the job J1 is registered, all of the computers C1 to C5 are in a job waiting state. On the other hand, referring to FIG. 7, the optimum number of jobs J1 is three. Accordingly, the distributed processing start unit 15 selects three computers (computers C1 to C3 in the example of FIG. 9), and causes these computers to execute job J1.

  Thereafter, job J2 is registered at 11:00. The optimum number of jobs J2 is 3, but there are only two computers (computers C4 and C5) waiting for jobs. Therefore, the distributed processing start unit 15 acquires the optimum calculation time (190 minutes) and the second calculation time (210 minutes) of the job J2, and calculates the arrival time as 380−120 = 260 (minutes). At this time, since the second calculation time (210 minutes) is shorter than the sum of the optimal calculation time and the arrival time (450 minutes), the distributed processing start unit 15 selects the computers C4 and C5, and jobs J2 are sent to these computers. Let's calculate.

  Thereafter, job J3 is registered at 12:00. At this time, there is no computer waiting for a job, so the distributed processing starting unit 15 does nothing. Thereafter, the same applies when job J4 is registered at 12:10.

  Thereafter, the calculation of the job J2 is completed at 14:30, and the queue management information and the computer information are updated as shown in FIGS. That is, the calculation status of job J2 is updated to “completed”, and the calculation statuses of computers C4 and C5 are updated to “waiting for calculation”.

  In response to this update, the distributed processing start unit 15 starts processing for the next unprocessed job. First, the distributed processing start unit 15 determines whether to calculate job J3, which is an unprocessed job with the oldest reception time. Since the optimum number of jobs J3 is three, but there are only two computers waiting for the job, the distributed processing start unit 15 determines the optimum calculation time (507 minutes) and second calculation time (560 minutes) of job J3, Based on the arrival time 380−330 = 50 (minutes), it is determined that the second calculation time is longer than the sum of the optimum calculation time and the arrival time (557 minutes).

  Therefore, the distributed processing start unit 15 determines whether or not the job J4, which is an unprocessed job whose reception time is the second oldest, can be processed. Since the second calculation time for job J4 is 42 minutes and the arrival time is 50 minutes, the distributed processing start unit 15 determines that job J4 is to be processed, selects computers C4 and C5, and assigns job J4 to these computers. Let it be calculated.

  Thereafter, the calculation of job J4 is completed at 15:12, and the queue management information of job J4 is updated. In response to this update, the distributed processing start unit 15 determines whether or not to calculate the job J3. Since the arrival time at this point is 380-372 = 8 (minutes), the distributed processing start unit 15 sets the second calculation time (560 minutes) to the sum of the optimum calculation time (507 minutes) and the arrival time (515 minutes). The job J3 is not calculated. At this time, since there is no job waiting to be processed, the distributed processing start unit 15 does nothing until the job being calculated ends.

  Thereafter, the calculation of job J1 is completed at 15:20, and the queue management information of job J1 is updated. At this time, since the computers 20 for the optimum number of jobs J3 can be secured, the distributed processing start unit 15 starts calculating the job J3. The calculation of job J3 ends at 23:47. Therefore, when the jobs J1 to J4 are processed using the distributed processing system 1, all processing is completed in 887 minutes.

  In contrast, when sequential processing is adopted and all of the jobs J1 to J4 are calculated by distributed processing using five computers, as shown in FIG. 10, it takes 1232 minutes to complete all processing. End up. Therefore, by adopting parallel processing by the distributed processing system 1, the calculation time is reduced by 28%.

  As described above, according to the present embodiment, when more than the optimal number of computers 20 are in a job waiting state, the optimal number of computers 20 are selected from those computers 20, and the earliest job is selected by the selected computer 20. Distributed processing. This makes it possible to optimize the assignment of the computer 20 to the earliest job without affecting the computer 20 that is executing the calculation (without reducing the calculation efficiency).

  Further, according to this embodiment, if the number of computers 20 in the job waiting state is less than the optimum number, it is further determined whether or not the earliest job can be processed within the addition time of the arrival time and the optimum calculation time. . If it is determined that processing is possible, all the computers 20 in the job waiting state are selected and distributed processing is executed. As a result, the processing of the earliest job can be completed more quickly by using the computer 20 more effectively.

  Further, according to the present embodiment, when the computer 20 in the job waiting state determines that the earliest job cannot be processed within the addition time, it is further determined whether there is another job that can be processed within the arrival time. Determined. When such another job exists, all the computers 20 in the job waiting state are selected, and distributed processing for the other job is executed. This makes it possible to shorten the job execution time as a whole by using the computer 20 more effectively.

  Further, according to the present embodiment, the calculation time is calculated using a function (for example, the above formula (1) or formula (2)) defined by the coefficient indicating the job calculation scale and the number of computers. This calculation is performed for each case where the computer 20 to be used is 1 to N. Then, the number corresponding to the shortest calculation time among the plurality of calculated times is derived as the optimum number. Since the job calculation time depends on the job calculation scale and the number of computers 20, the optimum number can be derived more accurately by using a function defined by the calculation scale and the number of computers.

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be modified in various ways as described below without departing from the scope of the invention.

In the embodiment described above, the following three types of determination processing are executed in the distributed processing start unit 15.
(1) Whether there are more than the optimal number of computers 20 in the job waiting state (2) Whether the earliest job can be executed by the computer 20 in the job waiting state within the addition time of the optimal calculation time and the arrival time (3) arrival Can another job other than the earliest job be executed by the computer 20 in the job waiting state within the time? However, the distributed processing start unit 15 may execute only a part of the above three types of determination processing.

  For example, only the determination (1) may be executed, or only the determinations (1) and (2) may be executed. When only the determination of (1) is executed, the distributed processing start unit 15 ends the process when the number of computers 20 waiting for a job is less than the optimum number. In the case where only the determinations (1) and (2) are executed, if it is determined that the earliest job cannot be executed within the addition time, the distributed processing start unit 15 ends the process.

  It is assumed that the distributed processing start unit 15 performs only the determination (1). At this time, if the jobs J1 to J4 shown in FIG. 7 are calculated using the computers C1 to C5 shown in FIG. 8, all the processes can be completed in 1115 minutes as shown in FIG. That is, the calculation time is reduced by 9% compared to the sequential processing shown in FIG.

  In the above embodiment, the distributed processing system 1 preferentially calculates a job with an old reception date and time, but the job selection method is not limited to this. For example, the distributed processing system 1 may preferentially calculate a job with a new reception date and time, or may select a job to be calculated based on the priority set for each job.

1 is a diagram illustrating an overall configuration of a distributed processing system according to an embodiment. It is a figure which shows the hardware constitutions of the computer management apparatus shown in FIG. It is a figure which shows the structure of queue management information. It is a figure which shows the structure of computer information. It is a flowchart explaining the process of the distributed processing system shown in FIG. It is a flowchart explaining the process of the distributed processing system shown in FIG. It is a figure which shows the specific example of queue management information. It is a figure which shows the specific example of computer information. It is a figure which shows the specific example of the operating condition of a computer at the time of using the distributed processing system shown in FIG. It is a figure which shows the specific example of the operating condition of a computer shown for a comparison. It is a figure which shows the specific example of the operating condition of a computer at the time of using the distributed processing system which concerns on a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Distributed processing system, 10 ... Computer management apparatus, 11 ... Input part, 12 ... Queue management part (storage means), 13 ... Computer management part, 14 ... Calculation time prediction part (calculation means), 15 ... Distributed processing start part (Derivation means, determination means, selection means, processing means), 16 ... result receiving section, 17 ... output section (output means), 20 ... computer, 21 ... job receiving section, 22 ... job processing section, 23 ... result transmitting section

Claims (9)

A distributed processing method in a distributed processing system including a plurality of computers,
A storage step for storing the job;
A derivation step for deriving the number of computers capable of calculating the job stored in the storage step in the shortest time as an optimum number;
A determination step of determining whether or not the number of computers waiting in a job waiting state is equal to or greater than the optimal number derived in the derivation step;
A selection step of selecting the optimal number of computers from the computers in the job waiting state when it is determined in the determination step that the waiting number is equal to or greater than the optimal number;
A processing step in which the computer selected in the selection step distributes the job stored in the storage step;
An output step for outputting the processing result of the job stored in the storage step based on the result of the distributed processing in the processing step;
A distributed processing method comprising: In the determination step, when the number of waiting machines is less than the optimum number, the number of computers waiting for the job is within the addition time of the arrival time and the shortest time until the number of computers waiting for the job reaches the optimum number. Determining whether the stored job can be calculated by the computer waiting for the job;
In the selection step, when it is determined in the determination step that the job can be calculated by the computer in the job waiting state within the addition time, all the computers in the job waiting state are selected.
The distributed processing method according to claim 1. In the storing step, one or more other jobs are further stored,
In the determination step, if it is determined that the job cannot be calculated by the computer waiting for the job within the addition time, whether there is another job that can be calculated by the computer waiting for the job within the arrival time. Determine whether or not
In the selection step, when it is determined in the determination step that there is another job that can be calculated, all the computers in the job waiting state are selected,
In the processing step, when it is determined in the determination step that the other job that can be calculated exists, the computer selected in the selection step causes the other job that can be calculated to be distributed.
The distributed processing method according to claim 2. Using a function defined by the coefficient indicating the job calculation scale and the number of computers, the calculation time when the number of jobs stored in the storage step is calculated for the number of computers equal to or less than the total number of computers is calculated. A calculation step for calculating,
In the derivation step, the number corresponding to the shortest of the plurality of calculation times calculated in the calculation step is derived as the optimum number.
The distributed processing method as described in any one of Claims 1-3 characterized by the above-mentioned. A computer management device capable of communicating with a plurality of computers,
Storage means for storing jobs;
Derivation means for deriving the number of computers capable of calculating the job stored in the storage means in the shortest time as the optimum number;
Determining means for determining whether the number of computers waiting in the job waiting state is equal to or greater than the optimum number derived by the deriving means;
A selection means for selecting the optimal number of computers from the computers in the job waiting state when the determination means determines that the waiting number is equal to or greater than the optimal number;
Processing means for distributing the jobs stored in the storage means to the computer selected by the selection means;
An output unit that outputs a processing result of the job stored in the storage unit based on a result of the distributed processing by the processing unit;
A computer management apparatus comprising: When the waiting number is less than the optimum number, the determining means stores the number of computers in the job waiting state in the storage means within the addition time of the time until the optimum number of computers reaches the optimum number. Whether the calculated job can be calculated by the computer waiting for the job,
The selection unit selects all the computers in the job waiting state when the determination unit determines that the job can be calculated by the computer in the job waiting state within the addition time;
The computer management apparatus according to claim 5. The storage means further stores one or more other jobs;
If the determination unit determines that the job waiting state computer cannot calculate the job within the addition time, whether there is another job that can be calculated by the job waiting state computer within the arrival time. Determine whether or not
The selection means selects all the computers in the job waiting state when the determination means determines that there is another job that can be calculated,
When the processing unit determines that the other job that can be calculated exists by the determination unit, the computer selected by the selection unit performs distributed processing on the other job that can be calculated.
The computer management apparatus according to claim 6. Using a function defined by the coefficient indicating the job calculation scale and the number of computers, the calculation time when the number of jobs stored in the storage means is calculated for each number of computers below the total number of computers is calculated. A calculation means for calculating;
The derivation means derives the number corresponding to the shortest of the plurality of calculation times calculated by the calculation means as the optimum number;
The computer management apparatus according to claim 5, wherein the computer management apparatus is a computer management apparatus. A distributed processing system comprising a plurality of computers,
Storage means for storing jobs;
Derivation means for deriving the number of computers capable of calculating the job stored in the storage means in the shortest time as the optimum number;
Determining means for determining whether the number of computers waiting in the job waiting state is equal to or greater than the optimum number derived by the deriving means;
A selection means for selecting the optimal number of computers from the computers in the job waiting state when the determination means determines that the waiting number is equal to or greater than the optimal number;
Processing means for distributing the jobs stored in the storage means to the computer selected by the selection means;
An output unit that outputs a processing result of the job stored in the storage unit based on a result of the distributed processing by the processing unit;
A distributed processing system comprising:

Images