JP2014197340A - Distributed processing system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To independently determine a job to be processed by own job processing means on the basis of the number of jobs of each processing definition information stored in a job queue in the case of allowing job processing means to perform the distributed processing of jobs to which processing definition information is designated.SOLUTION: Each job processing part 50 includes: distribution processing part 51 for distributing a flow to each job processing part 50 by executing distribution processing which is common to all the job processing parts 50; and a job execution part 53 for executing a job to which the flow distributed to its own job processing part is designated. The distribution processing part 51 is configured to, when there exists a flow in which the number of standby jobs stored in a job queue 16 satisfies increase conditions, increase the number of the job processing parts 50 as the distribution designations of the flow. An auto-scaler 15 is configured to, when the number of standby jobs satisfies the increase conditions even after the flow is distributed to all the job processing parts 50, perform scale out.

Description

  The present invention relates to a distributed processing system and a program.

  For example, in Patent Document 1, in order to distribute the load applied to a plurality of processing servers that process an event, a technique in which a context-dependent distribution device determines a processing server that processes the event according to a distribution rule according to the content of the event. Has been proposed.

JP 2006-309701 A JP 2007-306138 A JP 2008-040718 A Special table 2011-510367 gazette

  According to the present invention, when a job for which processing definition information is specified is distributed to the job processing means, the job processed by the own job processing means is independent based on the number of jobs for each processing definition information accumulated in the job queue. The purpose is to make a decision.

  The distributed processing system according to the present invention includes a registration unit for registering a job in which processing definition information defining processing to be performed on data to be processed is specified in a job queue, and a job stored in the job queue. One or a plurality of job processing means for processing only jobs for which processing definition information assigned to them is specified, and each job processing means executes a distribution process common to all the job processing means. A distribution processing unit that distributes the process definition information to each job processing unit, and a job execution unit that executes a job in which the process definition information distributed to the job processing unit is specified by the distribution processing unit. The distribution processing unit stores the process definition information specified for the job in accordance with the number of jobs accumulated in the job queue aggregated for each process definition information. Increasing or decreasing the number of said job processing means for the partial destination characterized.

  An adjustment unit that adjusts the number of job processing units, and the adjustment unit includes a predetermined number of jobs for which the process definition information assigned to all the job processing units by the distribution processing unit is designated; The number of job processing means is increased when a predetermined first job processing means increasing condition is satisfied.

  Further, the distribution processing unit, when the number of jobs accumulated in the job queue aggregated for each process definition information satisfies a predetermined increase condition, the process definition information specified for the job The number of job processing means to be distributed is increased.

  In addition, the distribution processing unit is configured to reduce the number of jobs stored in the job queue and specified in the process definition information with a plurality of job processing means as distribution destinations in advance. If the above condition is satisfied, the number of job processing means to which the process definition information specified for the job is assigned is reduced.

  Further, the adjusting means increases the number of job processing means even when the number of jobs accumulated in the job queue satisfies a predetermined second job processing means increase condition. To do.

  The adjustment unit may reduce the number of job processing units when the number of jobs stored in the job queue satisfies a predetermined job processing unit reduction condition.

  The program according to the present invention includes a registration unit for registering a job in which a process definition information for defining a process to be performed on data to be processed is specified in a job queue, and a job stored in the job queue. Of these, each job processing unit executes a distribution process common to all the job processing units. The job processing unit functions as one or a plurality of job processing units that process only a job for which processing definition information assigned to itself is specified. A distribution processing unit that distributes the process definition information to each job processing unit, and a job execution unit that executes a job in which the process definition information distributed to the job processing unit is specified by the distribution processing unit, The distribution processing unit is assigned to the job according to the number of jobs accumulated in the job queue that is aggregated for each process definition information. And wherein the increasing or decreasing the number of said job processing means for the assignment destination management definition information.

  According to the first aspect of the present invention, when the job processing means distributes the job for which the process definition information is specified, the own job processing means is based on the number of jobs for each process definition information accumulated in the job queue. Can independently determine the job to be processed.

  According to the second aspect of the present invention, the number of job processing means can be increased only after the process definition information is distributed to all job processing means.

  According to the third aspect of the present invention, when the load on the job processing means to which the process definition information is distributed increases, the number of job processing means to which the process definition information is assigned can be increased. .

  According to the fourth aspect of the present invention, when the load on the job processing means to which the process definition information is distributed decreases, the number of job processing means to which the process definition information is assigned can be reduced. .

  According to the fifth aspect of the present invention, when the load on all job processing means increases, the number of job processing means can be increased.

  According to the sixth aspect of the present invention, the number of job processing means can be reduced when the load on all job processing means is reduced.

  According to the seventh aspect of the present invention, when the job processing means distributes the job for which the process definition information is specified, the own job processing means is based on the number of jobs for each process definition information accumulated in the job queue. Can independently determine the job to be processed.

1 is an overall configuration diagram showing an embodiment of a distributed processing system according to the present invention. It is the figure which showed an example of the hardware constitutions of the job management server and job execution server in this Embodiment. It is the figure which showed an example of the block configuration by the side of the cloud among the distributed processing systems in this Embodiment. It is a figure showing an example of data composition of information about a job registered in a job queue in this embodiment. It is the figure which showed the example of a data structure of the job process part table set to the job process part table memory | storage part in this Embodiment. It is the figure which showed the data structural example of the flow multiplicity table set to the flow multiplicity table memory | storage part in this Embodiment. 5 is a flowchart illustrating job execution processing performed by a job processing unit according to the present embodiment. It is the figure which showed the procedure which maps a job processing part and a flow on the circumference based on the consistent hash method in this Embodiment, and the distribution state of the flow with respect to a job processing part. 5 is a flowchart showing processing on the job management server side in the present embodiment. It is the figure which showed the order which maps the replication of a flow and a job process part on the circumference based on the consistent hash method with the increase in the number of jobs in this Embodiment. It is the figure which showed the relationship between the job and job processing part in this Embodiment. It is the figure which showed the order which deletes the replication of the flow mapped on the circumference based on the consistent hash method, and the job process part according to the reduction | decrease in the number of waiting jobs in this Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing an embodiment of a distributed processing system according to the present invention. FIG. 1 shows an image forming apparatus 1 provided as an example of an apparatus that requests job execution, and a job management server 10 and a job execution server 5 that constitute a part of the cloud 2. The image forming apparatus 1 is an apparatus with a built-in computer equipped with a scan function, a copy function, a network function, and the like, and requests execution of a job by submitting the job to the cloud 2 in accordance with a user operation instruction. The job management server 10 receives a job execution request sent to the cloud 2 and manages the received job. The job execution server 5 executes the job received by the job management server 10.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the job management server 10 according to the present embodiment. In this embodiment, the server computer forming the job management server 10 can be realized by a general-purpose hardware configuration that has existed in the past. That is, the computer is configured by connecting the CPU 61, the ROM 62, the RAM 63, the HDD controller 65 to which the hard disk drive (HDD) 64 is connected, and the network controller 66 provided as a communication means to the internal bus 67 as shown in FIG. The Although not shown, an input / output controller for connecting a mouse and a keyboard as input means and a display as a display device may be connected to the internal bus 67.

  Since the job execution server 5 is also the same server computer as the job management server 10, the hardware of the job execution server 5 may be configured in the same way as the job management server 10.

  FIG. 3 is a diagram illustrating an example of a block configuration on the cloud 2 side in the distributed processing system according to the present embodiment. 3, the job management server 10 includes a job reception unit 11, a job management unit 12, a job processing unit monitoring unit 13, a job queue monitoring unit 14, an autoscaler 15, a job queue 16, a job processing unit table storage unit 17, and a flow. A multiplicity table storage unit 18 is provided. The job reception unit 11 receives a submitted job. The job received by the job receiving unit 11 includes data to be processed, a flow as process definition information in which a series of processes to be performed on the data are defined, and identification information (user ID) of the user who has requested execution of the job ) Is included at least. The flow includes a series of processing, such as fetching from a folder, scanning of original data such as document scanning, processing such as image processing, OCR processing, format conversion, and distribution processing of processing results such as fax, printing, and e-mail. The process flow and the contents of each process are defined. The job management unit 12 is provided as a registration unit. The job management unit 12 receives a job received by the job reception unit 11, adds the identification information (job ID) of the job and the reception date / time when the job is received, and registers the job in the job queue 16. . The job processing unit monitoring unit 13 monitors whether each job processing unit 50 is operating. The job queue monitoring unit 14 monitors the number of jobs accumulated in the job queue 16. The autoscaler 15 is provided as an adjusting unit and adjusts the number of job processing units 50.

  FIG. 4 is a diagram illustrating a data configuration example of information regarding jobs registered in the job queue 16 according to the present embodiment. The job-related information includes, for each job, identification information (job ID) of the job, identification information of the user who requested execution of the job (user ID), and identification information of the flow specified for the job (flow ID) , The reception date and time when the job is received and a document that is one form of data designated as the processing target of the job. The document may be information specifying the storage location of the document data, not the document data itself.

  FIG. 5 is a diagram showing a data configuration example of the job processing unit table set in the job processing unit table storage unit 17 in the present embodiment. The job processing unit table is updated by the job processing unit monitoring unit 13. In the job processing unit table, the identification information (job processing unit ID) of the job processing unit 50, the IP address of the job processing unit 50, and the last operation notification time are set in association with each other for each job processing unit 50. . The latest operation notification time is set to the latest time when it can be confirmed that the job processing unit 50 is operating.

  FIG. 6 is a diagram showing a data configuration example of the flow multiplicity table set in the flow multiplicity table storage unit 18 in the present embodiment. The flow multiplicity table is updated by the job queue monitoring unit 14. In the flow multiplicity table, identification information (flow ID) and multiplicity of the flow are set in association with each flow. In the multiplicity, the number of job processing units 50 that execute the flow is set. In the present embodiment, one flow is basically shared by one job processing unit 50, so 1 is set as an initial value.

  Each component 11 to 15 in the job management server 10 is realized by a cooperative operation of a computer that forms the job management server 10 and a program that operates on the CPU 61 mounted on the computer. The storage units 16 to 18 are realized by the HDD 64 or the RAM 63 mounted on the job management server 10. Alternatively, other storage means in the cloud 2 may be used via a network.

  In the present embodiment, the function provided by the job management server 10 is illustrated as being provided by a single server computer. However, the present invention is not limited to this, and the functions are provided distributed to a plurality of server computers. It may be configured.

  The job processing unit 50 in the present embodiment is realized by a virtual computer. One or a plurality of virtual computers are constructed in one job execution server 5. Each job processing unit 50 is provided as a job processing unit, and processes only jobs for which a flow assigned to itself is specified among jobs registered in the job queue 16. There may be one or more job processing units 50, and the number is increased or decreased by the autoscaler 15.

  The job processing unit 50 includes a distribution processing unit 51, a job acquisition unit 52, a job execution unit 53, and a control unit 54. The distribution processing unit 51 distributes the flow to each job processing unit 50 by executing a distribution process common to all the job processing units 50. The job acquisition unit 52 acquires a job as a processing candidate from the job management server 10. The job execution unit 53 executes a job in which the flow assigned to the own job processing unit 50 by the distribution processing unit 51 is designated. The control unit 54 performs execution control in the job processing unit 50 in cooperation with the components 51 to 53.

  The components 51 to 54 in the job execution server 5 are realized by a cooperative operation of a computer that forms the job execution server 5 and a program that operates on the CPU 61 mounted on the computer.

  The program used in this embodiment can be provided not only by communication means but also by storing in a computer-readable recording medium such as a CD-ROM or DVD-ROM. The program provided from the communication means or the recording medium is installed in the computer, and various processes are realized by the CPU of the computer sequentially executing the program.

  Incidentally, the job processing unit 50 according to the present embodiment is configured to be able to execute all types of flows. Therefore, since any job can be executed even if it is distributed to which job processing unit 50, autoscaling can be easily performed.

  However, while the job is being executed, information related to the flow specified for the job is held in the RAM 63, and therefore the RAM 63 is tight when the number of jobs having different types of flows to be executed increases. For example, even if the same 10 jobs are executed in parallel, the amount of memory used in the job processing unit 50 depends on whether the same flow is specified or whether all the different flows are specified. Will be different. This is particularly noticeable when jobs that do not know when to finish, such as jobs waiting for resumption, such as dialogue, are executed in parallel in different types of flows.

  Therefore, in the present embodiment, regarding job distribution processing, job processing units are not distributed based on the load based on the number of jobs being executed, but are distributed to the job processing units 50 according to the type of flow. 50 resources are used efficiently.

  Note that the job processing unit 50 processes a job, and a series of processing is performed on the data to be processed in accordance with the flow specified for the job. As described above, in this embodiment, jobs are distributed to the job processing units 50 according to the type of flow. For convenience of explanation, the job processing unit 50 simply selects “execute a job according to a specified flow” as “flow”. May be expressed as "execute". In the present embodiment, “processing” and “execution” for a job are almost synonymous.

  In this embodiment, job distribution processing is realized while efficiently using resources of the job processing unit 50 by distributing jobs to the job processing unit 50 according to the type of flow to be executed as described above. In the present embodiment, the flow is basically distributed so that one flow is processed by only one job processing unit 50. However, when the specified flow is biased, the processing is concentrated on the job processing unit 50 that is responsible for executing the flow with the increased number of jobs.

  Therefore, in the present embodiment, the job queue monitoring unit 14 totals the jobs accumulated in the job queue 16 for each flow type, and the distribution processing unit 51 determines the flow according to the total number of jobs. It is characterized by increasing or decreasing the number of job processing units 50 as distribution destinations. Specifically, when the number of jobs for each flow satisfies a predetermined increase condition, the number of job processing units 50 to which the flow is allocated is increased, while the job processing unit After increasing the number of 50, if the number of jobs for each flow satisfies a predetermined increase condition, the number of job processing units 50 to which the flow is allocated is decreased. ing. Each condition for increasing or decreasing the number of job processing units 50 as distribution destinations will be described later.

  Next, the operation in this embodiment will be described. First, processing that is performed periodically will be described.

  First, a predetermined number of job processing units 50 are prepared. In the present embodiment, three job processing units JP1 to JP3 are prepared. The job processing unit monitoring unit 13 inquires each job processing unit 50 as to whether or not it is regularly operating. Each job processing unit 50 notifies that it is operating by responding to this inquiry. Upon receiving the notification from the job processing unit 50, the job processing unit monitoring unit 13 updates the last operation notification time in the job processing unit table corresponding to the notified job processing unit 50 with the current time. Note that the last operation notification time corresponding to the job processing unit 50 that has not responded remains unupdated. For example, when the job processing unit monitoring unit 13 inquires every 5 minutes, if a time within 5 minutes from the current time is set as the final operation notification time, the job processing unit 50 is recognized as operating. On the other hand, if the last operation notification time is set to a time before 5 minutes from the current time, the job processing unit 50 is recognized as not operating at the present time.

  Next, an operation until a job is registered in the job queue 16 will be described.

  For example, the user operates the operation panel of the image forming apparatus 1 to display a list of flows, selects a flow to be used for job execution from the list, and then scans a document. In response to this user operation, the image forming apparatus 1 transmits a job including a user ID that identifies a user who has requested execution of the job, scan image data, and a flow ID that identifies a flow specified by the user, to the cloud. 2

  When a job is sent from the image forming apparatus 1, the job reception unit 11 in the cloud 2 receives the job and passes it to the job management unit 12. The job management unit 12 newly generates and assigns a job ID for identifying the job to the job, and further includes the user ID included in the sent job, the date and time when the job was received, and the document (scanned image data). In addition, it is registered in the job queue 16. The document may not be the document itself but may be information for specifying the storage location of the document.

  Next, job execution processing performed by the job processing unit 50 will be described with reference to the flowchart shown in FIG. In the distribution process performed in the job execution process, each flow is distributed to each job processing unit 50 using consistent hashing (consistent hash method). This distribution process is independently executed in each job processing unit 50. However, since all input values and processing contents used for the distribution process are the same, each job processing unit 50 performs this distribution process individually. The same processing result can be obtained even if it is executed.

  The job processing unit 50 in a state where the job can be newly executed acquires the job from the job queue 16 (step 101). Subsequently, the job processing unit 50 recognizes the job processing unit 50 that is currently operating by referring to the job processing unit table (step 102). Specifically, when the job processing unit monitoring unit 13 updates the final operation notification time every 5 minutes, the distribution processing unit 51 calculates the time difference between the current time and each final operation notification time, and the calculation result Is determined to be operating. Here, it is assumed that all job processing units JP1 to JP3 are operating. The distribution processing unit 51 calculates a hash based on the IP address of each job processing unit 50, and each job processing unit 50 is placed on a virtual circumference based on the consistent hash method as shown in FIG. Mapping is performed (step 103). In the present embodiment, a hash of 100 to 200 points is calculated in order to improve the balance of the positions of the job processing units 50 on the circumference. In FIG. 8A, the circled numbers represent the job processing unit 50, and the numbers 1 to 3 correspond to the job processing units JP1 to JP3.

  In this embodiment, the IP address is used as unique information in each job processing unit 50 so that different hash values are calculated. In the present embodiment, the IP address of each job processing unit 50 is registered in the job processing unit table and obtained from this table. However, other information may be used as information unique to each job processing unit 50. Further, the acquisition destination of unique information may be held inside, or may be acquired from other than the job processing unit table.

  Subsequently, the distribution processing unit 51 refers to the flow multiplicity table (step 103), calculates a hash using the flow ID of each flow as a key, and each job processing unit 50 as shown in FIG. 8B. Each flow is mapped on the circumference to which is mapped (step 104). In FIG. 8B, a circled letter represents a flow, and each letter A to E corresponds to each flow A to E.

  Through the above processing, as shown in FIG. 8B, the job processing units JP1 to JP3 and the flows A to E are mapped on the same virtual circumference. According to the consistent hash method, which flow is assigned to which job processing unit 50 is determined by moving each flow on the circumference in the clockwise direction. That is, each flow has a hash value larger than that of itself and is distributed to the job processing unit 50 having the closest value. Therefore, according to the example shown in FIG. Flows C and D are assigned to job processing unit JP2 and flow E is assigned to job processing unit JP3.

  As described above, when the flow in charge is determined, the control unit 54 refers to the flow specified for the job acquired in step 101 and determines whether the job is an execution target. For example, in the job processing unit JP1, if the acquired job flow is neither flow A nor flow B (N in step 106), it is determined that a job that is not the subject of execution is acquired, and the job is not executed. Next, the next job registered in the job queue 16 is acquired (step 101).

  On the other hand, if the flow of the acquired job is flow A or flow B (Y in step 106), the control unit 54 determines that the job to be executed has been acquired, and executes the job to the job execution unit 53. The job execution unit 53 starts executing the job in response to this instruction (step 107). Then, the control unit 54 deletes the job registered in the job queue 16 by sending a deletion instruction to the job management unit 12 (step 108). Note that the job management itself performed by the job management unit 12 such as acquisition and deletion of jobs from the job queue 16 may be the same method as before.

  In the present embodiment, each job processing unit 50 and each flow is automatically mapped on the same circumference every time a job is acquired as described above. The reason why each job processing unit 50 is mapped each time the job processing unit 50 acquires a job is that the job processing unit 50 is first acquired from the time the job is acquired until the next job is acquired. This is to cope with the case where the increase or decrease occurs. If the job processing unit JP3 is deleted without changing the flow type from the state shown in FIG. 8B, the flow E is executed by the job processing unit JP1 instead of the job processing unit JP3. . Second, when a new flow is added between the time when a job is acquired and the time when the next job is acquired, it is necessary to determine the job processing unit 50 responsible for that flow. As described above, in order to cope with increase / decrease in the job processing unit 50 and the flow, in the present embodiment, each mapping process is performed every time a job is acquired.

  Of course, it is not necessary to limit to this method. For example, the setting contents of the current job processing unit table and flow multiplicity table are temporarily stored, and the setting content temporarily stored every time a job is acquired, the newly acquired job processing unit table and flow multiplicity When there is a difference between the setting contents of the table, hash calculation may be performed to map each job processing unit 50 and each flow on the circumference.

  By the way, the number of jobs that are not executed in the job queue 16 and are in a waiting state (hereinafter simply referred to as “ If the rate of increase in which the “number of waiting jobs” increases is relatively high, the number of waiting jobs will increase in the job queue 16. Further, if the increasing rate of the job for which the flow A is designated is relatively higher than the job consumption rate in the job processing unit JP1 that executes a certain flow (for example, the flow A), the job queue 16 waits for the flow A. The number of jobs will increase. In the present embodiment, since one flow is executed by one job processing unit 50, the increase in the number of waiting jobs in this flow A only causes an increase in load in the job processing unit JP1. Become. In order to solve this problem, it can be considered that another job processing unit 50 is added as a sharing destination of the flow A in addition to the job processing unit JP1. In order to add a flow sharing destination, a copy of flow A is created on the circumference shown in FIG. 8, and job processing other than the job processing unit JP1 to which this flow A copy is currently assigned Mapping may be performed so as to be distributed to the unit 50. This “duplication of flow” is synonymous with adding a flow sharing destination. For example, the number of job processing units 50 that execute the flow A is increased by mapping the copy of the flow A to a position on the circumference that the job processing unit JP1 is not in charge of. As a result, the load on the job processing unit JP1 is reduced. In the present embodiment, when the increase in load cannot be eliminated even by duplication of this flow, the load distribution is performed by increasing the number of job processing units 50 by scale-out for the first time at this time.

  FIG. 9 is a flowchart showing processing on the job management server 10 side in this embodiment. Hereinafter, distributed processing of job load such as duplication of flow and scale-out performed when the number of jobs waiting to be executed increases. 7 and the flowchart shown in FIG. In the present embodiment, in FIG. 9, it is illustrated that the process is terminated every time assuming that the process is periodically repeated. However, in the state where the process is always started without being terminated. The process may be repeatedly executed (that is, return to step 201). Also, here, it is assumed that three job processing units JP1 to JP3 are operating and five flows A to E are used as shown in FIG. FIG. 10A is the same as FIG. 8B.

  The job queue monitoring unit 14 acquires the job accumulation status in order to monitor the job accumulation status in the job queue 16 (step 201). Specifically, the number of waiting jobs is obtained for each flow. And, the number of waiting jobs for a certain flow satisfies a predetermined flow duplication condition (increase condition), for example, the flow duplication condition that the number of waiting jobs for a certain period (for example, 10 minutes) continues to exceed the threshold of 10. If this is the case (Y in step 202), 1 is added to the multiplicity of the flow in the flow multiplicity table (step 206). Here, it is assumed that the flow A matches the flow duplication condition for the first time. That is, it is determined that N is determined in step 203 and the flow multiplicity of the flow A is updated to 2.

  In FIG. 7, the distribution processing unit 51 in the job processing unit 50 refers to the job processing unit table (step 102) after the job is acquired by the job acquisition unit 52 (step 101), and each job processing unit 50. Are mapped on the circumference (step 103). Here, since the job processing units JP1 to JP3 are also operating, as shown in FIG. 8A, the job processing units JP1 to JP3 are mapped on the circumference.

  Subsequently, when the distribution processing unit 51 refers to the flow multiplicity table (step 104), the multiplicity of the flow A is 2. Here, mapping of each flow when the flow multiplicity is 1 calculates a hash using the flow ID of each flow including the flow A as a key as described above, and maps each flow on the circumference (step 105). ). This state is shown in FIG. If the flow multiplicity is 2 because the flow is duplicated, the distribution processing unit 51 further performs a predetermined calculation for the flow multiplicity 2 with the flow ID of the flow as an input, so that the flow A key having a value different from the ID is generated. Then, hash calculation is performed based on the newly generated key, and mapping is performed on the circumference (step 105). FIG. 10B shows a state when the duplicated flow A71 is mapped on the circumference. According to the example of FIG. 10B, the duplication flow A (hereinafter, “duplication flow A”) 71 is allocated to the closest job processing unit JP3 which is larger than the duplication flow A71. In other words, the predetermined calculation for the flow multiplicity 2 that generates a key with the flow ID as an input needs to generate a key so that the flow A is shared with other than the job processing unit JP1 by hash calculation. Since the subsequent processing (steps 106 to 108) has already been described, description thereof is omitted here.

  In the job management server 10, a process (steps 207 to 211) subsequent to the process of adding 1 to the multiplicity of the flow in the flow multiplicity table (step 206) will be described later.

  FIG. 11 is a diagram illustrating a relationship between a job and the job processing unit 50 in the present embodiment. FIG. 11 shows jobs that have not yet been executed in the job queue and are accumulated and jobs that are being distributed to the job processing units 50 and are being executed. In FIG. 11, each job is identified by adding a serial number to the flow ID of the flow designated for the job in the order of registration. FIG. 11A shows the job status corresponding to FIG. According to FIG. 11A, A1, A2, B1, and A3 are assigned to the job processing unit JP1 and are being executed. D3, D4, and C1 are allocated to the job processing unit JP2 and are being executed. E1 is assigned to the job processing unit JP3 and is being executed. In the job queue, A4, A5,. . . , A15 are accumulated. Prior to the duplication of the flow, as shown in FIG. 10A, the flow A and the flow B are distributed to the job processing unit JP1, and the flow C and the flow D are distributed to the job processing unit JP2. The flow E is distributed to the job processing unit JP3.

  Then, when a copy of the flow A is created and mapped on the circumference as shown in FIG. 10B, the job (A4) in which the flow A is designated as shown in FIG. 11B. No. 76 is also executed in the job processing unit JP3.

  The job queue monitoring unit 14 in the job management server 10 acquires the job accumulation status in order to monitor the job accumulation status in the job queue 16, and obtains the number of waiting jobs for each flow (step 201). If the flow duplication condition that the number of waiting jobs in flow A has exceeded 10 for a certain period of time is satisfied (Y in step 202), all job processing units 50 (specifically, Since the flow A is not shared by the job processing unit JP2) (N in step 203), 1 is added to the multiplicity of the flow in the flow multiplicity table (step 206). As a result, the flow multiplicity of the flow A is updated to 3.

  In FIG. 7, the distribution processing unit 51 in the job processing unit 50 refers to the job processing unit table (step 102) after the job is acquired by the job acquisition unit 52 (step 101), and each job processing unit 50. Are mapped on the circumference (step 103). Here, since the job processing units JP1 to JP3 are also operating, as shown in FIG. 8A, the job processing units JP1 to JP3 are mapped on the circumference.

  Subsequently, when the distribution processing unit 51 refers to the flow multiplicity table (step 104), the multiplicity of the flow A is 3. Therefore, the distribution processing unit 51 first maps each flow when the flow multiplicity is 1. That is, a hash is calculated using the flow IDs of the flows A to E as keys, and each flow is mapped on the circumference (step 105). This state is shown in FIG. Subsequently, the distribution processing unit 51 generates a key having a value different from the flow ID by performing the predetermined calculation for the flow multiplicity 2 described above, and performs a hash calculation based on the newly generated key. To map on the circumference (step 105). FIG. 10B shows a state when the duplicated flow A71 is mapped on the circumference. The distribution processing unit 51 further performs a predetermined calculation for the flow multiplicity 3 with the flow ID of the flow as an input, thereby obtaining a value different from the key obtained for the flow ID and the flow multiplicity 2. Generate a key. Then, hash calculation is performed based on the newly generated key, and mapping is performed on the circumference (step 105). FIG. 10C shows a state when the duplicated flow A72 is mapped on the circumference. According to the example of FIG. 10C, the duplication flow A72 is allocated to the closest job processing unit JP2 which is larger than the duplication flow A72. In other words, the predetermined calculation for flow multiplicity 3, which generates a key with the flow ID as an input, is shared by other than the job processing unit JP1 and job processing unit JP3 to which flow A has already been shared by hash calculation. You need to generate a key. Since the subsequent processing (steps 106 to 108) has already been described, description thereof is omitted here.

  As a result, as shown in FIG. 11C, the job (A13) 77 in which the flow A is designated is also executed in the job processing unit JP2.

  By repeating the above processing, the flow A is executed by a plurality of job processing units 50. If the state where the number of waiting jobs of the flow A satisfies the flow duplication condition is not resolved, as described above The number of job processing units 50 to which the flow A is distributed is increased, and is eventually shared by all the job processing units JP1 to JP3.

  The job queue monitoring unit 14 in the job management server 10 acquires the job accumulation status in order to monitor the job accumulation status in the job queue 16, and obtains the number of waiting jobs for each flow (step 201). Assume that A satisfies the flow replication condition. In this case (Y in step 202), the job queue monitoring unit 14 subsequently checks whether or not the flow A is copied to all the job processing units 50. This may be confirmed by comparing the number of active job processing units 50 with the value of the flow multiplicity. When the flow A is duplicated in all the job processing units 50 (Y in Step 203), the scale-out is performed only at this stage. That is, the job queue monitoring unit 14 performs the scale-out by satisfying the predetermined first job processing unit increase condition that the flow A satisfies the flow duplication condition and has been copied to all the job processing units 50. When the autoscaler 15 is instructed, the autoscaler 15 scales out in accordance with this instruction (step 204). Further, the job processing unit monitoring unit 13 performs an update for adding a record relating to a newly generated job processing unit to the job processing unit table (step 205). Here, the job processing unit JP4 is added, and the current time is set as the final operation notification time. Subsequently, the job queue monitoring unit 14 adds 1 to the multiplicity of the flow in the flow multiplicity table (step 206). As a result, the flow multiplicity of the flow A is updated to 4.

  In FIG. 7, the distribution processing unit 51 in the job processing unit 50 refers to the job processing unit table (step 102) after the job is acquired by the job acquisition unit 52 (step 101), and each job processing unit 50. Are mapped on the circumference (step 103). At this time, the distribution processing unit 51 recognizes that the job processing units JP1 to JP4 are operating, and as shown in FIG. 10D, the distribution processing unit 51 is represented by reference numeral 74 in addition to the job processing units JP1 to JP3. The job processing unit JP4 is mapped on the circumference.

  Subsequently, when the distribution processing unit 51 refers to the flow multiplicity table (step 104), the multiplicity of the flow A is 4. The distribution processing unit 51 maps each flow on the circumference with reference to the setting contents of the flow multiplicity table. However, the processing of the multiplicity of flows B to E and flow A from 1 to 3 is the same as described above. Description is omitted.

  Further, as described above, the distribution processing unit 51 performs a predetermined calculation for the flow multiplicity 4 using the flow ID of the flow as an input, thereby generating a key having a value different from the flow ID. The distribution processing unit 51 performs a predetermined calculation for the flow multiplicity 4, thereby generating a key having a value different from the key obtained for the flow ID, the flow multiplicity 2 and the flow multiplicity 3. . Then, hash calculation is performed based on the newly generated key, and mapping is performed on the circumference (step 105). FIG. 10D shows a state when the duplicated flow A73 is mapped on the circumference. According to the example of FIG. 10D, the duplication flow A73 is assigned to the closest job processing unit JP4 which is larger than the duplication flow A73. Focusing on the mapping position, the duplication flow A71 is assigned to the job processing unit JP3 when the multiplicity is 3, but is assigned to the job processing unit JP4 by mapping the job processing unit JP4. . That is, the flow A is distributed to the job processing units JP1, JP2, and JP4. Therefore, here, it is necessary to map the duplication flow A73 so as to be shared by the job processing unit JP3. Since the subsequent processing (steps 106 to 108) has already been described, description thereof is omitted here.

  As a result of this scale-out, as shown in FIG. 11D, the job (A21) 78 in which the flow A is designated is also executed in the job processing unit JP4.

  As described above, even after the flow A is executed by all the job processing units 50, if the state where the number of waiting jobs of the flow A still satisfies the flow duplication condition is not solved, it is dealt with by scale-out. . After this, if the state where the number of waiting jobs of the flow A satisfies the flow duplication condition is not solved, the scale-out is repeatedly executed. In this way, load distribution is achieved.

  As described above, when the number of waiting jobs for a certain flow (flow A in the above example) increases, the number of job processing units 50 that execute the flow is increased. Then, if the state of a load with a high number of waiting jobs is maintained even if the job is handled by all the job processing units 50, that is, if the state satisfying the flow duplication condition continues, it is handled by scale-out. I did it.

  However, even if the number of waiting jobs in a certain flow does not satisfy the flow duplication condition (N in step 202 in FIG. 9), the total number of waiting jobs in the job queue 16 is determined according to the scale-out execution condition (predetermined second job). When the processing unit increase condition) is met, for example, when the scale-out execution condition that the number of all waiting jobs exceeds the threshold of 30 for a certain period (for example, 10 minutes) continues (Y in step 212), Also in this case, it may be determined that the high load state continues in the entire job processing unit 50 that processes the flow A, and the scale-out may be performed (step 204).

  Note that the flow duplication condition for increasing the number of job processing units 50 in charge of one flow is not limited to the above example, and may be set as appropriate in consideration of the type of job, the performance of the job processing unit 50, and the like. The same applies to the scale-out execution conditions. For example, when the value obtained by dividing the number of waiting jobs by the number of job processing units 50 exceeds a predetermined threshold for a certain period, or when the number of jobs being executed and waiting for restart exceeds a predetermined threshold for a certain period You may set as implementation conditions.

  The above describes the response when the number of waiting jobs increases. Here, the processing when the load is reduced after performing the scale-out according to the above-described processing flow will be described using steps 207 to 211 in FIG. 9.

  At present, it is assumed that job processing units JP1 to JP4 and flows A to E are mapped on the circumference as shown in FIG. 12A is the same as FIG. 10D.

  With reference to the status of the job queue 16 acquired in step 201, a duplication release condition (decrease condition) in which the number of waiting jobs in the duplication flow is determined in advance, for example, a threshold that the number of waiting jobs is 5 for a certain period (for example, 10 minutes) When the copy cancellation condition that the state has been lowered is satisfied (Y in Step 207), the job queue monitoring unit 14 subtracts 1 from the multiplicity of the copy flow in the flow multiplicity table (Step 208).

  In FIG. 7, the distribution processing unit 51 in the job processing unit 50 refers to the job processing unit table (step 102) after the job is acquired by the job acquisition unit 52 (step 101), and each job processing unit 50. Are mapped on the circumference (step 103). Here, since the job processing units JP1 to JP4 are also operating, the job processing units JP1 to JP4 are mapped on the circumference as shown in FIG.

  Subsequently, when the distribution processing unit 51 refers to the flow multiplicity table (step 104), the multiplicity of the flow A is 3. Therefore, here, the process when the multiplicity of the flow A is 3 is executed (step 105). That is, since the mapping state is returned to the multiplicity of 3, the mapping state is such that the replication flow 73 is deleted from the state shown in FIG. 12A as shown in FIG.

  As described above, the order in which the duplicate flow is deleted when the duplicate release condition is satisfied is the reverse of the duplicate order of the duplicate flow 73, duplicate flow 72, and duplicate flow 71.

  Even if the duplication flow does not satisfy the duplication cancellation condition, if the total number of waiting jobs in the job queue 16 matches the job processing unit reduction condition, for example, a threshold of 10 for the total number of waiting jobs for a certain period (for example, 10 minutes). If the job processing unit reduction condition that the state below the value has continued is satisfied (Y in step 209), the job processing unit 50 is reduced (step 210). Specifically, the job processing unit monitoring unit 13 performs an update to delete the job processing unit 50 set in the job processing unit table (step 211). Any job processing unit JP1 to JP4 may be targeted for deletion, but in the present embodiment, the added job processing unit JP4 is deleted.

  In FIG. 7, the distribution processing unit 51 in the job processing unit 50 refers to the job processing unit table (step 102) after the job is acquired by the job acquisition unit 52 (step 101), and each job processing unit 50. Are mapped on the circumference (step 103). At this time, the distribution processing unit 51 recognizes that the job processing units JP1 to JP3 are operating, and maps them on the circumference. In this way, the job processing unit JP4 is deleted. FIG. 12C shows a state when the job processing unit JP4 is deleted from the state of FIG. In FIG. 12B, the job processing unit JP3 was in charge of the flow E and the job processing unit JP4 was in charge of the flow A. However, when the job processing unit JP4 is deleted, the flow A71 is again assigned to the job processing unit JP3. Will be. As a result, the job processing unit JP3 takes charge of the flows A and E.

  Note that the replication flow to be canceled and the job processing unit 50 to be reduced do not necessarily have to be in the reverse order of increase. For example, considering the current load status applied to each job processing unit 50, You may make it delete from the small job process part 50. FIG.

  In the present embodiment, as described above, each job processing unit 50 autonomously increases or decreases the number of job processing units 50 in charge of the flow according to the number of waiting jobs accumulated in the job queue 16 to distribute the load. I was able to do it. The scale-out is performed only when all the job processing units 50 are assigned to a flow having a large number of waiting jobs.

  In the present embodiment, the flow is distributed using the consistent hash method, but an alternative method may be used.

  DESCRIPTION OF SYMBOLS 1 Image forming apparatus, 2 Cloud, 5 Job execution server, 10 Job management server, 11 Job reception part, 12 Job management part, 13 Job processing part monitoring part, 14 Job queue monitoring part, 15 Auto scaler, 16 Job queue, 17 Job processing unit table storage unit, 18 flow multiplicity table storage unit, 50 job processing unit, 51 distribution processing unit, 52 job acquisition unit, 53 job execution unit, 54 control unit, 61 CPU, 62 ROM, 63 RAM, 64 Hard disk drive (HDD), 65 HDD controller, 66 network controller, 67 internal bus.

Claims (7)

A registration means for registering a job in which processing definition information for defining processing to be performed on data to be processed is specified in a job queue;
One or a plurality of job processing means for processing only jobs for which processing definition information assigned to the job is specified among jobs stored in the job queue;
Have
Each job processing means includes:
A distribution processing unit that distributes the process definition information to each job processing unit by executing a distribution process common to all the job processing units;
A job execution unit that executes a job in which the process definition information distributed to itself by the distribution processing unit is designated;
Have
The distribution processing unit determines the number of job processing means to which the process definition information specified for the job is allocated according to the number of jobs accumulated in the job queue that is aggregated for each process definition information. A distributed processing system characterized by increasing or decreasing the number. Adjusting means for adjusting the number of the job processing means;
The adjusting unit is configured to execute the processing when the number of jobs in which the process definition information assigned to all the job processing units is assigned by the distribution processing unit satisfies a predetermined first job processing unit increase condition. 2. The distributed processing system according to claim 1, wherein the number of job processing means is increased.   When the number of jobs accumulated in the job queue aggregated for each process definition information satisfies a predetermined increase condition, the distribution processing unit distributes the process definition information specified for the job. 2. The distributed processing system according to claim 1, wherein the number of said job processing means is increased.   The distribution processing unit satisfies a reduction condition in which the number of jobs stored in the job queue and specified in the process definition information with a plurality of job processing means as distribution destinations is determined in advance. 2. The distributed processing system according to claim 1, wherein the number of job processing means to which the processing definition information designated for the job is assigned is reduced.   The adjustment unit increases the number of job processing units even when the number of jobs stored in the job queue satisfies a predetermined second job processing unit increase condition. Item 3. The distributed processing system according to Item 2.   3. The adjustment unit according to claim 2, wherein the adjustment unit reduces the number of job processing units when the number of jobs accumulated in the job queue satisfies a predetermined job processing unit reduction condition. Distributed processing system. Computer
Registration means for registering a job in which job definition information that defines processing to be performed on data to be processed is specified in a job queue;
One or a plurality of job processing means for processing only jobs for which processing definition information assigned to the job is specified among jobs stored in the job queue;
Function as
Each job processing means includes:
A distribution processing unit that distributes the process definition information to each job processing unit by executing a distribution process common to all the job processing units;
A job execution unit that executes a job in which the process definition information distributed to itself by the distribution processing unit is designated;
Have
The distribution processing unit determines the number of job processing means to which the process definition information specified for the job is allocated according to the number of jobs accumulated in the job queue that is aggregated for each process definition information. A program characterized by increasing or decreasing.

Images