JP5997659B2 - Distributed processing system and distributed processing method - Google Patents

Description

  The present invention belongs to a technical field in which an application load is distributedly processed in a plurality of processing units of a physical server group including physical servers.

  In recent years, with the expansion of the system scale and the spread of server virtualization technology, it has become an issue to reduce the procurement cost of the entire system. In particular, in system operation using server virtualization technology, the number of processing units (virtual machines) for processing processing requests from clients can be flexibly changed. By adjusting the number of processing copies to be executed according to the amount, the number of processing copies to be used can be reduced and the number of physical servers can be reduced.

As a conventional technique for adjusting the number of processing units to be executed according to a change in application load, a technique described in Patent Document 1 can be cited.
The technology described in Patent Document 1 determines whether to add a processing unit by monitoring the load of a processing request input from a client and the load of each processing unit of an application that processes the processing request. A scale-out system configuration is used to add more units. However, the technique described in Patent Document 1 only increases the number of processing units when the load increases, and the number of processing units cannot be increased or decreased according to load fluctuations.
In order to solve this problem, a scale-out system configuration is conceivable in which the load of a processing request input from a client is monitored, and both processing units can be added / removed in accordance with load fluctuations. The system configuration in this case is to monitor the load amount due to the processing request input from the client at the stage before allocating to each processing unit, and use the upper and lower thresholds for the load amount obtained as a result of the monitoring. A simple processing unit increase / decrease determination criterion is provided.

JP 2006-343899 A

  However, in the system configuration as described above, in the case of a system specification in which it is impossible to monitor the total amount of load in the distribution unit that actually distributes processing requests to each processing unit (for example, a load balancer that distributes by simple round robin, etc. Is difficult to apply to the distribution unit. Furthermore, since the increase / decrease setting is simply determined based on the upper and lower thresholds, the processing unit increase / decrease processing is executed each time the thresholds are exceeded, increasing the overhead required for changing the number of processing units and changing the arrangement of the processing units. However, system stability may be impaired.

  The present invention has been made in view of such a background, and the present invention can set an appropriate number of processing units according to the load amount even when the load amount cannot be monitored in the distribution unit, It is an object of the present invention to provide a distributed processing system and a distributed processing method capable of suppressing an increase in overhead associated with a change in the number of processing units and ensuring system stability.

In order to solve the above-described problem, the invention according to claim 1 is directed to a distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, and a process that processes the distributed processing request A plurality of physical servers in which one or more processing units are arranged, the distribution device, each of the processing units, and each of the plurality of physical servers, and the processing units arranged in the physical server Each of the processing units, a processing unit load amount information collecting unit that collects the used resource amount or the number of processing cases of the processing unit, and the collected A processing unit load amount information notifying unit for notifying the resource adjusting device of the amount of used resources or the number of processing cases as processing unit load amount information, (1) An upper limit threshold for the processing unit load amount information for determining whether to add the processing unit, a first duration threshold indicating a period during which a value equal to or greater than the upper limit threshold continues, and (2 ) A processing unit increase / decrease determination threshold for storing a lower limit threshold for the processing unit load information and a second duration threshold indicating a period during which a value lower than the lower threshold continues to determine deletion of the processing unit A storage unit for storing information, a processing unit load amount information receiving unit for receiving the processing unit load amount information from each of the processing units , and all processing units of the distributed processing system based on the processing unit load amount information first the use resource amount or the processed number of processing units account for more than a predetermined percentage, additional terms of the processing unit when the period in which the at least the upper threshold exceeds said first duration threshold of Thereby judged satisfied, based on the processor load information, the used resource amount or the processed number of processor occupying the second predetermined ratio or more of the total processor of the distributed processing system, the lower threshold A processing unit increase / decrease determination unit that determines that the deletion condition of the processing unit is satisfied when the period less than the second duration threshold is exceeded, and the physical unit when it is determined that the additional condition of the processing unit is satisfied A processing unit placement execution unit that controls the server to delete a processing unit when it is determined that the processing unit is added to the server, and when it is determined that a deletion condition of the processing unit is satisfied. When adding the processing unit, the processing unit additional information including information of the processing unit to be added is generated and transmitted to the distribution device, and when the processing unit is deleted, the processing unit to be deleted A processing unit addition / deletion information transmission unit that generates processing unit deletion information including the information of the processing unit and transmits the processing unit deletion information to the distribution device, and the distribution device includes the processing unit as a distribution destination of the processing request The storage unit information to be stored is stored, and the information of the processing unit to be added indicated in the processing unit additional information is registered in the distribution destination information, and the deletion indicated in the processing unit deletion information is deleted. A processing unit addition / deletion execution unit that deletes information of the processing unit from the distribution destination information; a distribution processing unit that distributes the received processing request to the processing unit with reference to the distribution destination information; A distributed processing system characterized by comprising:

According to a fourth aspect of the present invention, there is provided a distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, a processing unit that processes the distributed processing request, and one or more processing units. A plurality of physical servers in which the processing units are arranged, and resource adjustment that adjusts the number of the processing units that are connected to the distribution device, the processing units, and the plurality of physical servers and are arranged in the physical servers A distributed processing method of a distributed processing system comprising: a step in which each of the processing units collects the amount of used resources or the number of processed cases of the processing unit itself; and the collected amount of used resources or the number of processed cases A step of notifying the resource adjustment device as processing unit load amount information, and the resource adjustment device adds (1) the processing unit. An upper limit threshold for the processing unit load amount information and a first duration threshold indicating a period during which a value equal to or higher than the upper limit threshold continues, and (2) for determining deletion of the processing unit, A storage unit for storing processing unit increase / decrease determination threshold information in which a lower limit threshold for the processing unit load amount information and a second duration threshold indicating a period during which a value lower than the lower limit threshold continues are stored; Receiving the processing unit load information from each of the processing units, and the processing units occupying a first predetermined ratio or more of all processing units of the distributed processing system based on the processing unit load information. Determining that an additional condition of the processing unit is satisfied when a period in which the amount of resources used or the number of processes exceeds the first threshold exceeds a first duration threshold; Based on the load information, the dispersion the use resource amount or the processed number of processor occupying the second predetermined ratio or more of the total processing of the processing system, the duration is the second below the lower threshold A step of determining that the deletion condition of the processing unit is satisfied when a period threshold is exceeded, and a step of controlling to add a processing unit to the physical server when it is determined that the additional condition of the processing unit is satisfied And a step of controlling the physical server to delete the processing unit when it is determined that the processing unit deletion condition is satisfied, and information on the processing unit to be added when adding the processing unit Generating the processing unit additional information including the processing unit and transmitting the processing unit to the allocating device; and, when deleting the processing unit, generating processing unit deletion information including information on the processing unit to be deleted Transmitting to an apparatus, and the distribution apparatus includes a storage unit that stores distribution destination information indicating the processing unit that is a distribution destination of the processing request, and adds the processing unit Registering information of the processing unit to be added indicated in the information in the distribution destination information; deleting information of the processing unit to be deleted indicated in the processing unit deletion information from the distribution destination information; And a step of distributing the received processing request to the processing unit with reference to the distribution destination information.

By doing so, the distributed processing system (distributed processing method) determines whether or not the resource adjustment device satisfies the additional condition or the deleted condition of the processing unit based on the processing unit load amount information received from each processing unit. Can be determined. When the additional condition of the processing unit is satisfied, the resource adjustment device adds the processing unit of the physical server, and then transmits information on the processing unit to be added to the distribution device, and the distribution device adds the information. Information of the processing unit can be registered in the distribution destination information. Further, when the deletion condition of the processing unit is satisfied, the resource adjustment device transmits information on the processing unit to be deleted to the distribution device, deletes it from the distribution destination information, and deletes the processing unit to be deleted from the physical server. Can be deleted.
Therefore, according to the present invention, an appropriate number of processing units can be set according to the load amount of the processing unit even when the distribution unit cannot monitor the load amount.
Further, the resource adjustment apparatus is configured to execute a predetermined period (first continuation period ) in the processing units that occupy a predetermined ratio (first predetermined ratio, second predetermined ratio) or more of all the processing sections of the distributed processing system. Since the addition / deletion of the processing unit is determined based on the processing unit load amount information of the threshold value and the second duration threshold value), it is not necessary to change the number of processing units every time each threshold value is exceeded as in the prior art. Therefore, an increase in overhead due to the change in the number of processing copies can be suppressed.

  The invention according to claim 2 is a round robin in which the distribution processing unit of the distribution device equally sets the distribution ratio of the processing request to each of the processing units registered in the distribution destination information. Alternatively, the processing unit that is a distribution destination is determined by a weighted round robin in which the distribution ratio of the processing request to each processing unit is set according to the processing capacity of the physical server in which the processing unit is arranged. The distributed processing system according to claim 1 is provided.

  By doing so, the distribution device of the distributed processing system can determine a processing unit as a distribution destination by using round robin or weighted round robin, and distribute processing requests.

The invention according to claim 3 is a distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, a processing unit that processes the distributed processing request, and one or more of the one or more A plurality of physical servers in which processing units are arranged, and a resource adjustment device that is connected to the distribution device, each of the processing units, and the plurality of physical servers, and adjusts the number of the processing units arranged in the physical servers; Each of the processing units includes a processing unit load amount information collecting unit that collects the amount of used resources or the number of processing cases of the processing unit, and the collected amount of used resources or the number of processing cases. A processing unit load amount information notifying unit that notifies the resource adjustment device as processing unit load amount information, and the resource adjustment device is configured to: (1) An upper limit threshold for the processing unit load amount information and a first duration threshold indicating a period during which a value equal to or higher than the upper limit threshold continues, and (2) determining deletion of the processing unit A storage unit storing processing unit increase / decrease determination threshold information in which a lower limit threshold for the processing unit load amount information and a second duration threshold indicating a period during which a value lower than the lower limit threshold continues are stored; A processing unit load amount information receiving unit that receives the processing unit load amount information from each processing unit, and a period in which the amount of used resources or the number of processes is equal to or greater than the upper limit threshold based on the processing unit load amount information. When the first duration threshold is exceeded, it is determined that the additional condition of the processing unit is satisfied, and based on the processing unit load amount information, the used resource amount or the processing number is When it is determined that the processing unit increase / decrease determination unit determines that the deletion condition of the processing unit is satisfied when the period below the limit threshold exceeds the second duration threshold value, and the additional condition of the processing unit is satisfied, A processing unit arrangement execution unit that controls to add a processing unit to the physical server and controls the physical server to delete the processing unit when it is determined that the processing unit deletion condition is satisfied. When the processing unit is added, processing unit additional information including information on the processing unit to be added is generated and transmitted to the distribution device, and when the processing unit is deleted, the processing unit to be deleted A processing unit addition / deletion information transmission unit that generates processing unit deletion information including the information of the processing unit and transmits the processing unit deletion information to the distribution device, and the distribution device includes the processing unit as a distribution destination of the processing request The distribution destination information is stored And information on the processing unit to be added indicated in the processing unit addition information is registered in the distribution destination information, and information on the processing unit to be deleted indicated in the processing unit deletion information is registered in the distribution. A processing unit addition / deletion execution unit to be deleted from the destination information, and a distribution processing unit that distributes the received processing request to the processing unit with reference to the distribution destination information, and the processing unit includes : Consists of virtual machines placed on the physical server, and the physical server is in a standby state in which the physical server is in operation to place a processing unit that executes processing for the processing request, and the processing request is not accepted Physical server resource information collection in which each physical server collects the amount of resources used by the physical server. And a physical server resource information notification unit that notifies the resource adjustment device of the collected used resource amount as physical server resource information, and the resource adjustment device further processes the processing request in the storage unit Processing unit arrangement, which is processing schedule information indicating a processing schedule indicating a period for executing an application for each processing unit, and information relating the processing unit and the physical server on which the processing unit is arranged Information and physical server selection threshold information in which an upper limit threshold for the amount of resource used by the physical server and an upper limit threshold for the amount of resource scheduled to be used of the physical server calculated using the processing schedule information are stored. And receiving the physical server resource information from each of the physical servers. When the physical server resource information receiving unit and the processing unit increase / decrease determining unit determine that the additional condition of the processing unit is satisfied, for each of the physical servers, all the processing units arranged in the physical server are Extracting based on the processing unit arrangement information, calculating the use scheduled resource amount of each of the extracted processing units using the processing schedule information, summing the calculated planned use resource amount for each processing unit, The physical server resource information received from the physical server, the calculated scheduled resource amount for each physical server being calculated, the calculated scheduled resource amount not exceeding the upper limit threshold for the physical server scheduled resource amount The physical server in which the used resource amount shown in FIG. 1 does not exceed the upper threshold for the used resource amount of the physical server. When selecting and determining a physical server to which the processing unit is to be added from the selected physical servers, and when the processing unit increase / decrease determination unit determines that the deletion condition of the processing unit is satisfied, A placement destination physical server determination unit that determines a processing unit to be deleted from among the processing units that satisfy the deletion condition, wherein the processing unit placement execution unit, the processing unit increase / decrease determination unit, and the processing unit When it is determined that the additional condition is satisfied, control is performed so that the processing unit arranged in the physical server for the resource pool is moved to the physical server at the placement destination determined by the placement destination physical server determination unit. When the processing unit increase / decrease determination unit determines that the deletion condition of the processing unit is satisfied, control is performed to move the processing unit to be deleted to the physical server for the resource pool. When, was distributed processing system that it said.

As described above, the distributed processing system includes a processing unit configured with a virtual machine, a physical server, an active physical server in which a processing unit that executes processing for a processing request is arranged, and a resource in which a processing unit in a standby state is arranged. It consists of a physical server for the pool. Accordingly, when the resource adjustment device determines that the additional condition of the processing unit is satisfied, the processing unit arranged in the physical server for the resource pool can be moved to the physical server of the arrangement destination. Further, when it is determined that the deletion condition of the processing unit is satisfied, the resource adjustment device can move the processing unit to be deleted from the operating physical server to the physical server for the resource pool. Therefore, increase / decrease of a process part can be performed flexibly.
In addition, the resource adjustment device uses the physical server resource information received from each physical server and the scheduled use resource amount for each physical server to determine a physical server that is a more appropriate placement destination of the processing unit to be added. Can do.

  According to the present invention, even when the distribution unit cannot monitor the load amount, an appropriate number of processing units can be set according to the load amount, and an increase in overhead due to a change in the number of processing units can be suppressed, thereby stabilizing the system. It is possible to provide a distributed processing system and a distributed processing method that can ensure the performance.

It is a block diagram of the distributed processing system which concerns on this embodiment. It is a functional block diagram which shows the structural example of the physical server which concerns on this embodiment, and a process part. It is a functional block diagram which shows the structural example of the resource adjustment apparatus which concerns on this embodiment. It is a figure which shows an example of a data structure of the process schedule information which concerns on this embodiment. It is a figure which shows an example of a data structure of the process part use plan resource information which concerns on this embodiment. It is a figure which shows an example of a data structure of the process part arrangement | positioning information which concerns on this embodiment. It is a functional block diagram which shows the structural example of the distribution apparatus which concerns on this embodiment. It is a flowchart which shows the flow of a process of the distributed processing system which concerns on this embodiment.

  Next, the distributed processing system 1 and the like in a mode for carrying out the present invention (hereinafter referred to as “the present embodiment”) will be described.

<System configuration and processing overview>
Initially, with reference to FIG. 1, the structure and process outline | summary of the distributed processing system 1 which concern on this embodiment are demonstrated.
The distributed processing system 1 according to the present embodiment monitors the load amount of each processing unit 20 that performs distributed processing for a predetermined period (every predetermined time interval), and determines the number of processing units 20 according to the load amount variation. A scale-out distributed processing system that can be changed flexibly.

  As shown in FIG. 1, the distributed processing system 1 according to the present embodiment distributes a processing request from the client 5 to the processing unit 20, and a plurality of physical servers 30 (30A, 30B, 30C,...). A plurality of processing units 20 (20a, 20b, 20c,...) That are provided above and execute processing of processing requests distributed by the distribution device 10 and return response information to the client 5; A resource adjustment device 40 that determines the increase / decrease in the number of processing units 20 and the location of each processing unit 20 on the physical server 30 according to the amount of load associated with the execution of processing.

Each processing unit 20 (20a, 20b, 20c,...) Is configured by a virtual machine and is arranged on an arbitrary physical server 30 (30A, 30B, 30C,...). A plurality of processing units 20 (virtual machines) may be provided on one physical server 30. Further, the processing unit 20 (virtual machine) arranged on a certain physical server 30 can be moved to another physical server 30 by using a virtualization technique.
The resource adjustment device 40 indicates the processing amount of each processing unit 20 (20a, 20b, 20c,...) Based on load amount information (“processing unit load amount information” to be described later, for example, the amount of resources used and the number of processing cases). The number of 20 is adjusted (addition / deletion of the processing unit 20), and each processing unit is based on load information (“physical server resource information” described later) of each physical server 30 (30A, 30B, 30C,...). The physical server 30 to be the 20 placement destination is determined. The physical server 30 includes an active physical server 30 (30A, 30B, 30C,...) In which the processing unit 20 is arranged and executes processing for a processing request, and a standby processing unit 20 that does not accept a processing request (not shown). ) Are arranged in two types, the physical server 30 (30P) for the resource pool.

When it is determined that the processing unit 20 needs to be added based on the load amount information (processing unit load amount information) of each processing unit 20, the resource adjustment device 40 is the physical that is the placement destination of the newly added processing unit 20. The server 30 is determined, and the processing unit 20 to be added is moved from the physical server 30 (30P) for the resource pool to the physical server 30 that is the arrangement destination. Thereafter, the resource adjustment device 40 transmits information on the processing unit 20 newly added to the distribution device 10 (“processing unit additional information” described later), and the distribution device 10 distributes the processing request from the client 5. The added processing unit 20 is registered in the distribution destination information 131 in which each processing unit 20 is registered.
Further, when it is determined that the processing unit 20 needs to be deleted based on the load amount information (processing unit load amount information) of each processing unit 20, the resource adjustment device 40 provides information on the processing unit 20 to be deleted (described later). To the distribution device 10, and the distribution device 10 deletes the processing unit 20 to be deleted from the distribution destination information 131. Thereafter, the resource adjustment device 40 moves the processing unit 20 to be deleted to the physical server 30 (30P) for resource pool.

  By doing in this way, the distributed processing system 1 which concerns on this embodiment can adjust the number of the suitable process parts 20 according to the load amount of each process part 20. FIG. Further, the distributed processing system 1 determines whether or not the processing unit 20 needs to be added or deleted every predetermined period. Therefore, an increase in overhead due to the change in the number of processing copies can be suppressed, and system stability can be ensured.

<Configuration of each device>
Next, the distribution device 10, the processing unit 20, the physical server 30, and the resource adjustment device 40 that configure the distributed processing system 1 according to the present embodiment will be specifically described.

<< Physical Server 30 and Processing Unit 20 >>
First, configuration examples of the physical server 30 and the processing unit 20 according to the present embodiment will be described.
FIG. 2 is a block diagram illustrating a configuration example of the physical server 30 and the processing unit 20 according to the present embodiment. The physical server 30 places a plurality of processing units 20 (virtual machines) by applying a virtualization technique, and the processing unit 20 executes processing for a processing request from the client 5. In the present embodiment, a description will be given assuming that one processing unit 20 (virtual machine) executes one application.

  The physical server 30 is a device that includes a control unit, an input / output unit, and a storage unit that are not shown. As shown in FIG. 2, the control unit includes a virtualization control unit 31, a physical server resource information collection unit 32, and a physical server resource information notification unit 33.

  The virtualization control unit 31 performs control for constructing a virtualization platform on the physical server 30 and arranging a plurality of processing units 20 (virtual machines) based on the virtualization technology.

  The physical server resource information collection unit 32 displays various resource usage (CPU usage rate, memory usage rate, disk I / O (Input / Output) amount, network I / O amount, etc.) of the physical server 30 itself as physical server resources. Collect as information. The physical server resource information collection unit 32 uses at least one of a CPU (Central Processing Unit) usage rate, a memory usage rate, a disk I / O amount, and a network I / O amount as physical server resource information for a predetermined time. Collect at intervals.

  The physical server resource information notification unit 33 notifies the resource adjustment device 40 of physical server resource information collected by the physical server resource information collection unit 32 at predetermined time intervals via an input / output unit (not shown).

  The processing unit 20 is a virtual machine placed on the physical server 30 by the virtualization control unit 31 of the physical server 30, and as illustrated in FIG. 2, the application processing unit 21, the processing unit load information collection unit 22, and the processing A load amount information notification unit 23 is provided.

  The application processing unit 21 receives a processing request acquired from the client 5 via the distribution device 10, executes processing according to the application, and transmits the result to the client 5 as response information.

  The processing unit load amount information collecting unit 22 collects its own load amount information (processing unit load amount information) for each processing unit 20 (virtual server). Here, the processing unit load amount information is the amount of resources used and the number of processing cases. The processing unit load amount information collection unit 22 collects at least one of the CPU usage rate, the memory usage rate, the disk I / O amount, and the network I / O amount as the used resource amount. Alternatively, the processing unit load amount information collection unit 22 collects the number of processing cases (the number of processing cases / unit time) that is the number of processing requests received from the client 5. The processing unit load amount information collection unit 22 collects any of the various used resource amounts or the number of processings as the processing unit load amount information set in advance as its own processing unit load amount information at predetermined time intervals. To do.

  The processing unit load amount information notification unit 23 notifies the resource adjustment device 40 of the processing unit load amount information collected by the processing unit load amount information collection unit 22 at a predetermined time interval.

<< Resource adjustment device 40 >>
Next, a configuration example of the resource adjustment device 40 according to the present embodiment will be described.
FIG. 3 is a functional block diagram illustrating a configuration example of the resource adjustment device 40 according to the present embodiment.
The resource adjustment device 40 determines whether to increase or decrease the number of the processing units 20 based on the processing unit load amount information (used resource amount or number of processing cases) acquired from each processing unit 20 (20a, 20b, 20c,...). judge. When the resource adjustment device 40 determines that the processing unit 20 needs to be added, the resource adjustment device 40 moves the new processing unit 20 from the physical server 30 (30P) for the resource pool to the physical server 30 that is the placement destination. Information on the processing unit 20 to be added is transmitted to the sorting device 10. When the resource adjustment device 40 determines that the processing unit 20 needs to be deleted, the distribution device 10 transmits information on the processing unit 20 to be deleted, and the processing unit 20 to be deleted is used for the resource pool. Move to the physical server 30 (30P).
As shown in FIG. 3, the resource adjustment device 40 includes a control unit 41, an input / output unit 42, and a storage unit 43.

  The input / output unit 42 inputs / outputs information to / from the distribution device 10, the processing units 20 (20a, 20b, 20c,...), The physical servers 30 (30A, 30B, 30C,. I do. The input / output unit 42 also performs input / output between a communication interface that transmits / receives information via a communication line and an input unit such as a keyboard (not shown) or an output unit such as a monitor. It consists of.

The control unit 41 controls the entire resource adjustment device 40, and includes a processing unit load information receiving unit 411, a physical server resource information receiving unit 412, a processing unit increase / decrease determination unit 413, a placement destination physical server determination unit 414, and a processing unit arrangement. An execution unit 415 and a processing unit addition / deletion information transmission unit 416 are included.
In addition, this control part 41 is implement | achieved when CPU which abbreviate | omitted illustration expand | deployed and executed the program stored in the memory | storage part 43 on RAM (Random Access Memory), for example.

  The processing unit load amount information receiving unit 411 receives the processing unit load amount information (the amount of used resources or the number of processing cases) transmitted by each processing unit 20 (20a, 20b, 20c,...) And stores it in the storage unit 43. The processing unit load amount information 431 stored in the storage unit 43 is referred to in the processing unit increase / decrease determination process described later by the processing unit increase / decrease determination unit 413.

  The physical server resource information receiving unit 412 receives the physical server resource information (used resource amount, etc.) transmitted by each physical server 30 (30A, 30B, 30C,...) And stores it in the storage unit 43. The physical server resource information 432 stored in the storage unit 43 is referred to in the placement destination physical server determination process described later by the placement destination physical server determination unit 414.

The processing unit increase / decrease determination unit 413 determines increase / decrease (addition or deletion) of the processing unit 20 based on the processing unit load amount information 431 (used resource amount or number of processing cases) received by the processing unit load amount information receiving unit 411. Note that the processing unit increase / decrease determination unit 413 performs processing unit increase / decrease determination processing at intervals exceeding a predetermined period (continuation period threshold value used for determination) to be described later after the processing unit load information 431 received at predetermined time intervals is accumulated. Run.
Specifically, the processing unit increase / decrease determination unit 413 uses (1) used resource amount (any of CPU usage rate, memory usage rate, disk I / O amount, network I / O amount) as processing unit load amount information. The processing unit increase / decrease determination process executed in the case of using the case, (2) the case of using the number of processing cases, and the case will be described.

(Processing unit increase / decrease determination processing based on the amount of resources used by each processing unit 20)
[When processing part is added]
When the total number of the processing units 20 is N, the processing unit increase / decrease determination unit 413 continues the upper limit threshold b _{1 of the} amount of resource used by any number of the processing units 20 having a predetermined ratio a ₁ % or more. When the predetermined period c ₁ (first continuation period threshold) is exceeded (this is referred to as “first additional condition”), the number of processing units 20 is added by a predetermined ratio d ₁ %.
[When processing parts are reduced]
When the total number of processing units 20 is N, the processing unit increase / decrease determination unit 413 uses the resource usage for all types of resource usage preset by the number of processing units 20 equal to or greater than a predetermined ratio e ₁ %. When the quantity lower limit threshold f ₁ is continuously kept below a predetermined period g ₁ (second duration threshold) (this is referred to as “first deletion condition”), the number of processing units 20 is set to a predetermined ratio. h Reduce by ₁ %.

(Processing unit increase / decrease determination processing based on the number of processing cases of each processing unit 20)
[When processing part is added]
When the total number of processing units 20 is N, the processing unit increase / decrease determination unit 413 sets the upper limit threshold b ₂ for the number of processing units (the number of processing cases / unit time) to be greater than or equal to a predetermined ratio a ₂ %. When the predetermined period c ₂ (first continuation period threshold) is continuously exceeded (this is referred to as “second additional condition”), the number of processing units 20 is added by a predetermined ratio d ₂ %.
[When processing parts are reduced]
Processor decrease determination unit 413, when the total number of processing unit 20 is N stage, predetermined ratio e _2% or more in the number of processing unit 20, the lower limit threshold value f ₂ of Handles (Handles / unit time) Is continued and falls below the predetermined period g ₂ (second continuation period threshold) (this is referred to as “second deletion condition”), and the number of processing units 20 is reduced by a predetermined ratio h ₂ %.

The processing unit increase / decrease determination unit 413 includes the processing unit addition condition ("first addition condition" or "second addition condition") and the deletion condition ("first deletion condition" or "second deletion" described above). Based on “condition”), increase / decrease (addition / deletion) of the processing unit 20 is determined. The processing unit increase / decrease determination unit 413 determines that the number of the processing units 20 is not changed when neither the addition condition nor the deletion condition is satisfied.
Information about each threshold and a predetermined period used by the processing unit increase / decrease determination unit 413 for the processing unit increase / decrease determination is stored in advance in the processing unit increase / decrease determination threshold information 436 in the storage unit 43.

Further, when the processing unit increase / decrease determination unit 413 determines that the “first addition condition” is satisfied, the processing unit increase / decrease determination unit 413 executes a process of “adding the number of processing units by a predetermined ratio d ₁ %”. When the total number of 20 is N = 50 and the predetermined ratio d ₁ % = 3%, the number of processing units 20 to be added is 50 × 0.03 = 1.5. In such a case, the logic is set so that two decimal places are rounded up and two units are added. The same applies to the “second additional condition”.
On the other hand, when the processing unit increase / decrease determination unit 413 determines that the “first deletion condition” is satisfied, the processing unit increase / decrease determination unit 413 executes a process of “reducing the number of processing units by a predetermined ratio h ₁ %”. When the total number of 20 is N = 50 and the predetermined ratio h ₁ % = 3%, the number of processing units 20 to be deleted is 50 × 0.03 = 1.5. In such a case, the logic is set so that one decimal place is deleted and one unit is deleted. The same applies to the “second deletion condition”.

  The placement destination physical server determination unit 414 determines which physical server 30 the processing unit 20 is added to when the processing unit increase / decrease determination unit 413 determines that the additional condition of the processing unit 20 is satisfied. Also, the placement destination physical server determination unit 414 determines which processing unit 20 is the processing unit 20 to be deleted when the processing unit increase / decrease determination unit 413 determines that the deletion condition of the processing unit 20 is satisfied.

(Destination physical server determination process when processing unit 20 is added)
First, the placement destination physical server determination process when the processing unit 20 is added by the placement destination physical server determination unit 414 will be described.
When the processing unit increase / decrease determination unit 413 determines that the additional condition of the processing unit 20 is satisfied, the placement destination physical server determination unit 414 determines the processing unit increase / decrease determination unit 413 from the physical server 30 (30P) for the resource pool. The physical server 30 that is the destination of the number of processing units 20 to be added is determined from the physical servers 30 that satisfy the following physical server selection criteria.

[Physical server selection criteria]
For the physical server 30 (30A, 30B, 30C,...) That is currently in operation, the physical server resource information receiving unit 412 refers to the physical server resource information 432, and various types of used resources (CPU usage rate, memory usage rate). , All of the disk I / O amount and network I / O amount set in advance) do not exceed the upper limit threshold value x set for each of the various used resource amounts, and continue to be physical for a predetermined period in the future. A physical server that does not exceed the upper threshold y is selected for the amount of resources scheduled to be used by the server 30.
Note that the upper limit threshold value x and the upper limit threshold value y of the scheduled resource amount used by the physical server 30 are stored in advance in the storage unit 43 of the resource adjustment device 40 as physical server selection threshold information 437.

  Of the above physical server selection criteria, the scheduled use resource amount of the physical server 30 is calculated as the sum of the scheduled use resource amounts of the processing units 20 (virtual machines) already arranged in each physical server 30. In the storage unit 43 of the resource adjustment device 40, processing schedule information 433, processing unit usage scheduled resource information 434, and processing unit arrangement information 435 are stored as information for calculating the scheduled resource usage of the physical server 30. .

FIG. 4 is a diagram illustrating an example of a data configuration of the processing schedule information 433 according to the present embodiment.
As illustrated in FIG. 4, the process schedule information 433 stores a period (process occurrence period) in which a process by the application is scheduled to occur in association with the processing unit 20 that executes each application as a process schedule.

FIG. 5 is a diagram illustrating an example of a data configuration of the processing unit use scheduled resource information 434 according to the present embodiment.
As shown in FIG. 5, the processing unit scheduled use resource information 434 stores the amount of resources scheduled to be used (scheduled use resource amount) according to the type of application executed by each processing unit 20.
If the processing schedule of a certain processing unit 20 is not stored in the processing schedule information 433 of FIG. 4, the placement destination physical server determination unit 414, for example, the processing unit scheduled use resource information 434 for all within a predetermined period. Assuming that a predetermined resource amount based on is used, the scheduled resource amount of the processing unit 20 is calculated.

FIG. 6 is a diagram illustrating an example of a data configuration of the processing unit arrangement information 435 according to the present embodiment.
The processing unit arrangement information 435 is information that associates the processing unit 20 with the physical server 30 on which the processing unit 20 is arranged.

  The placement destination physical server determination unit 414 (see FIG. 3) extracts all the processing units 20 arranged in the physical server 30 for each physical server 30 based on the processing unit arrangement information 435, and extracts each processing unit. The sum of the 20 scheduled resource amounts is calculated based on the processing schedule information 433 (see FIG. 4) and the processing unit usage scheduled resource information 434 (see FIG. 5), and the scheduled resource amount for each physical server 30 is calculated. Then, the placement destination physical server determination unit 414 compares the calculated scheduled resource usage amount of the physical server 30 with the upper limit threshold value y.

  The placement destination physical server determination unit 414, when more physical servers 30 satisfying the physical server selection criteria are selected than the number of physical servers 30 that have additionally determined, the physical servers 30 satisfying the physical server selection criteria. An arbitrary physical server 30 is selected from the list.

(Determination processing of the processing unit 20 to be deleted when the processing unit 20 is deleted)
Next, the determination process of the processing unit 20 to be deleted when the processing unit 20 is deleted by the placement destination physical server determination unit 414 will be described.
When the processing unit increase / decrease determination unit 413 determines that the deletion condition of the processing unit 20 is satisfied, the placement-destination physical server determination unit 414 determines the number of processing units 20 to be deleted from among the processing units 20 that satisfy the deletion condition. An arbitrary processing unit 20 is selected and determined as the processing unit 20 to be deleted. The placement destination physical server determination unit 414 determines the number of processing units to be deleted from the processing units 20 satisfying the deletion condition of the processing unit 20 in ascending order of resource usage or in descending order of the number of processing cases. Only the processing unit 20 may be determined.

When the processing unit 20 is added, the processing unit placement execution unit 415 newly adds the processing unit 20 included in the resource server 30 (30P) for the resource pool to the processing unit 20 that is determined by the placement destination physical server determination unit 414. The physical server 30 (30P) for the resource pool and the physical server 30 of the arrangement destination are controlled so as to move to the physical server 30 of the arrangement destination. At this time, the processing unit arrangement execution unit 415 updates the processing unit arrangement information 435 when the arrangement of the newly added processing unit 20 is completed.
In addition, when the processing unit 20 is deleted, the processing unit arrangement execution unit 415 performs control to move the processing unit 20 to be deleted from the physical server 30 to the physical server 30 (30P) for the resource pool. At this time, when the migration of the processing unit 20 to be deleted to the physical server 30 (30P) for the resource pool is completed, the processing unit arrangement execution unit 415 updates the processing unit arrangement information 435.

  The processing unit addition / deletion information transmission unit 416 uses the added information of the processing unit 20 (identification information, address, etc. of the processing unit 20) as processing unit additional information to the distribution device 10 via the input / output unit 42. Send. In addition, the processing unit addition / deletion information transmission unit 416 uses the deleted processing unit 20 information (such as the identification information of the processing unit 20) as processing unit deletion information to the distribution device 10 via the input / output unit 42. Send.

  The storage unit 43 includes storage means such as a hard disk, a flash memory, and a RAM. As described above, the processing unit load amount information 431, physical server resource information 432, processing schedule information 433, processing unit use scheduled resource information 434, processing unit. Arrangement information 435, processing unit increase / decrease determination threshold information 436, physical server selection threshold information 437, and the like are stored.

≪Distributing device 10≫
Next, a configuration example of the sorting apparatus 10 according to the present embodiment will be described.
FIG. 7 is a functional block diagram illustrating a configuration example of the sorting apparatus 10 according to the present embodiment.
The distribution apparatus 10 distributes the processing request received from the client 5 to each processing unit 20 according to a predetermined method. The distribution device 10 is a device that returns the response information received from the processing unit 20 to the client 5 that has transmitted the processing request.
As shown in FIG. 7, the distribution device 10 includes a control unit 11, an input / output unit 12, and a storage unit 13.

  The input / output unit 12 inputs / outputs information to / from each processing unit 20, the resource adjustment device 40, and the like. Further, the input / output unit 12 is an input / output interface that performs input / output between a communication interface that transmits / receives information via a communication line and an input unit such as a keyboard (not shown) or an output unit such as a monitor. It consists of.

The control unit 11 controls the entire distribution apparatus 10, and includes a distribution method setting unit 111, a processing unit addition / deletion information reception unit 112, a processing unit addition / deletion execution unit 113, and a distribution processing unit 114. Is done.
In addition, this control part 11 is implement | achieved when CPU which abbreviate | omitted illustration expand | deployed and executed the program stored in the memory | storage part 13, for example.

  The distribution method setting unit 111 sets a method for determining the processing unit 20 as a distribution destination of the processing request from the client 5 with reference to the distribution destination information 131 stored in the storage unit 13. The distribution method setting unit 111 can set a method such as round robin or weighted round robin.

In the round robin method, the distribution ratio of processing requests to the processing units 20 is set equally.
On the other hand, in the weighted round robin method, the distribution ratio of processing requests to each processing unit 20 is set to a specified value. As the value of the distribution ratio, for example, a value corresponding to the specification value of the physical server 30 in which the processing unit 20 is arranged is set in a hetero environment where the server specification value (processing capability) of the physical server 30 is different. The spec value corresponds to, for example, the number of CPU cores of the physical server 30, the memory capacity, and the like. For example, the distribution ratio of processing requests is set to be high for the physical server 30 having a larger number of CPU cores than the other physical servers 30.

  The distribution method setting unit 111 obtains distribution method setting information from a system management apparatus (not shown) via the input / output unit 12 before starting the processing by the distribution processing unit 114, and uses the round robin method. Or a weighted round robin method is preset.

  The processing unit addition / deletion information reception unit 112 receives the processing unit addition information or the processing unit deletion information from the resource adjustment device 40 and passes it to the processing unit addition / deletion execution unit 113.

  When the processing unit addition / deletion execution unit 113 acquires the processing unit additional information, the processing unit addition / deletion execution unit 113 uses the information on the newly added processing unit 20 included in the processing unit additional information as the distribution destination information 131 in the storage unit 13. Register with. The distribution destination information 131 is information in which the identification information and address of the processing unit 20 that is the distribution destination of the processing request from the client 5 is stored as a list.

  Further, when the processing unit addition / deletion execution unit 113 acquires the processing unit deletion information, the processing unit addition / deletion execution unit 113 deletes the information of the processing unit 20 to be deleted stored in the distribution destination information 131 in the storage unit 13.

  Upon receiving a processing request from the client 5, the distribution processing unit 114 refers to the distribution destination information 131 in the storage unit 13 and determines the distribution destination according to the distribution method set by the distribution method setting unit 111. The processing request is transmitted to the processing unit 20 of the determined distribution destination.

  The storage unit 13 includes storage means such as a hard disk, a flash memory, and a RAM, and stores the distribution destination information 131 and the like described above.

<Process flow>
Next, a processing flow of the distributed processing system 1 according to the present embodiment will be described.
FIG. 8 is a flowchart showing a processing flow of the distributed processing system 1 according to the present embodiment.
In the distributed processing system 1 according to the present embodiment, each processing unit 20 (20a, 20b, 20c,...) Processes the processing unit load amount information at a predetermined time interval as a premise process for explaining the following flowchart. It is assumed that each physical server 30 (30A, 30B, 30C,...) Transmits physical server resource information to the resource adjustment device 40 at predetermined time intervals.
The number of physical servers 30 in the initial state and the arrangement of the processing unit 20 (virtual server) on the physical server 30 are described in, for example, Non-Patent Document 1 (Hitotoshi Nakazato et al., “Optimum arrangement of virtual machines constituting OSS” It is assumed that the virtual machine initial placement method described in “Proposal of Method”, IEICE, IEICE Technical Report, 112 (120), 2012/07/12, 31-36) has been set. Based on the arrangement of the processing units 20 (20a, 20b, 20c,...) On the physical server 30 (30A, 30B, 30C,...) In the initial state, the processing schedule is stored in the storage unit 43 of the resource adjustment device 40. It is assumed that the information 433 and the processing unit arrangement information 435 have been stored.
Furthermore, the processing unit increase / decrease determination threshold information 436, which is various threshold values used for the processing unit increase / decrease determination processing, obtained from the system management apparatus (not shown) of the distributed processing system 1, and the various types used for the placement destination physical server determination processing. It is assumed that the physical server selection threshold information 437 that is the threshold value of the resource is stored in the storage unit 43 of the resource adjustment device 40.

  First, the processing unit load amount information receiving unit 411 of the resource adjustment device 40 receives processing unit load amount information from each processing unit 20 (step S10).

Further, the physical server resource information receiving unit 412 of the resource adjustment device 40 receives physical server resource information from each physical server 30 (step S11).
Note that either step S10 or step S11 may be executed first.

  Next, the processing unit increase / decrease determination unit 413 of the resource adjustment device 40 performs a processing unit increase / decrease determination process (step S12).

  Here, the processing unit increase / decrease determination unit 413, in the processing unit increase / decrease determination process, when neither the addition condition nor the deletion condition of the processing unit 20 described above is satisfied (step S12 → “addition / deletion condition is not satisfied”), The process ends without increasing or decreasing the number of processing units 20. On the other hand, when the additional condition is satisfied in the processing unit increase / decrease determination process (step S12 → “additional condition”), the process proceeds to the next step S13. In the processing unit increase / decrease determination process, when the deletion condition is satisfied (step S12 → “deletion condition”), the process proceeds to the next step S18.

(Additional processing of processing unit 20)
Next, an additional process of the processing unit 20 in steps S13 to S17 will be described.
In step S13, the processing unit increase / decrease determination unit 413 of the resource adjustment device 40 determines the number of processing units 20 to be added.

Subsequently, the placement destination physical server determination unit 414 of the resource adjustment device 40 performs placement destination physical server determination processing for determining the physical server 30 to be the placement destination based on the physical server selection criteria (step S14).
Specifically, the placement destination physical server determination unit 414 refers to the processing unit placement information 435 (FIG. 6), extracts the processing unit 20 placed in each physical server 30, and stores the processing unit 20 in the storage unit 43. With reference to the processing schedule information 433 (FIG. 4) and the processing unit use scheduled resource information 434 (FIG. 5), the extracted usage planned resource amount of the processing unit 20 is calculated. Then, the placement destination physical server determination unit 414 calculates the scheduled resource amount for each physical server 30 by calculating the sum of the extracted scheduled resource amounts of the processing unit 20. Next, the placement destination physical server determination unit 414 determines that the actual used resource amount indicated in the physical server resource information acquired in step S11 does not exceed the upper limit threshold value x, and the planned use resource amount of the physical server 30 is the upper limit threshold value. A physical server 30 that does not exceed y is selected. If no physical server 30 satisfying the physical server selection criteria is selected, one physical server 30 to be operated is added and the placement destination physical server determination process is executed again.
Subsequently, the placement destination physical server determination unit 414 determines the physical server 30 that is the placement destination of the processing unit 20 from the physical servers 30 selected as satisfying the physical server selection criteria. Note that the determination of the physical server 30 as the placement destination is arbitrarily determined.

  Next, the processing unit arrangement execution unit 415 of the resource adjustment device 40 moves the processing unit 20 to be newly added from the physical server 30 for the resource pool to the physical server 30 at the arrangement destination determined in Step S14 (Step S14). S15). At this time, the processing unit placement execution unit 415 updates the processing unit placement information 435 in the storage unit 43 so that the newly added processing unit 20 is placed in the physical server 30 that is the placement destination.

  Subsequently, the processing unit addition / deletion information transmission unit 416 of the resource adjustment device 40 generates processing unit addition information including information such as identification information and address of the processing unit 20 to be newly added, and transmits the processing unit addition information to the distribution device 10. (Step S16).

  Then, the processing unit addition / deletion information reception unit 112 of the distribution device 10 receives the processing unit addition information from the resource adjustment device 40, and the processing unit addition / deletion execution unit 113 stores the distribution stored in the storage unit 13. Information related to the processing unit 20 to be added to the destination information 131 is registered (step S17), and the process ends.

(Deleting process of the processing unit 20)
Next, the deletion process of the processing unit 20 in steps S18 to S21 will be described.
In step S18, the processing unit increase / decrease determination unit 413 of the resource adjustment device 40 determines the number of processing units 20 to be deleted. In addition, the processing unit increase / decrease determination unit 413 determines the processing unit 20 to be actually deleted from among the processing units 20 that satisfy the deletion condition. The determination of the processing unit 20 to be deleted may be made by arbitrarily selecting the processing unit 20 satisfying the deletion condition and determining the processing unit 20 to be deleted, or from the processing unit 20 with the least amount of resources used. The number of processing units 20 to be deleted may be selected in order or in order from the processing unit 20 having the smallest number of processing cases, and the processing unit 20 to be deleted may be determined.
Note that the placement destination physical server determination unit 414 reassigns the processing unit 20 to another physical server 30 assuming that the processing unit 20 to be deleted has been deleted by the virtual machine relocation method described in Non-Patent Document 1. By arranging, the number of operating physical servers 30 can be reduced.

  Subsequently, the processing unit addition / deletion information transmission unit 416 of the resource adjustment device 40 generates processing unit deletion information including the identification information of the processing unit 20 to be deleted, and transmits the processing unit deletion information to the distribution device 10 (step S19).

  Then, the processing unit addition / deletion information reception unit 112 of the distribution device 10 receives the processing unit deletion information from the resource adjustment device 40, and the processing unit addition / deletion execution unit 113 stores the distribution stored in the storage unit 13. The information of the processing unit 20 to be deleted stored in the destination information 131 is deleted (step S20).

  Next, the processing unit arrangement execution unit 415 of the resource adjustment device 40 moves the processing unit 20 to be deleted from the physical server 30 that is the arrangement destination so far to the physical server 30 (30P) for resource pool (step S30). S21). Note that the processing unit allocation execution unit 415 deletes the processing unit 20 to be deleted from the physical server 30 that is the deletion target, and deletes the processing unit allocation information 435 in the storage unit 43 from the physical server 30 (30P) for resource pool. Update as you move. Then, the processing of the entire distributed processing system 1 is finished.

  As described above, according to the distributed processing system 1 and the distributed processing method according to the present embodiment, even if it is impossible to monitor the load amount in the allocating unit, it depends on the load amount of each processing unit 20. Therefore, the appropriate number of processing units 20 can be adjusted. Further, since it is determined whether or not the processing unit 20 needs to be added or deleted every predetermined period, an increase in overhead due to a change in the number of processing units can be suppressed, and system stability can be ensured.

Although the present embodiment has been described above, the present invention is not limited to the embodiment described here.
For example, in the present invention, the processing unit 20 has been described on the assumption that it is a virtual machine. For example, a plurality of CPUs (cores) provided in the physical server 30 are used as the processing unit 20, and resources used by the processing unit 20 are as follows. The amount or the number of processing cases may be transmitted to the resource adjustment device 40 as processing unit load amount information, and the processing unit increase / decrease determination may be performed. In this case, the processing unit 20 itself cannot be moved to another physical server 30, but when adding the processing unit 20, when the standby CPU (core) is put into operation and the processing unit 20 is deleted. Can adjust the number of the processing units 20 by putting a CPU (core) in an operating state into a standby state.

DESCRIPTION OF SYMBOLS 1 Distributed processing system 5 Client 10 Distribution apparatus 11,41 Control part 12,42 Input / output part 13,43 Storage part 20 Processing part 21 Application processing part 22 Processing part Load amount information collection part 23 Processing part Load amount information notification part 30 Physical server 31 Virtualization control unit 32 Physical server resource information collection unit 33 Physical server resource information notification unit 40 Resource adjustment device 111 Distribution method setting unit 112 Processing unit addition / deletion information reception unit 113 Processing unit addition / deletion execution unit 114 Distribution processing unit 131 Distribution destination information 411 Processing unit load amount information reception unit 412 Physical server resource information reception unit 413 Processing unit increase / decrease determination unit 414 Placement destination physical server determination unit 415 Processing unit placement execution unit 416 Processing unit addition / deletion information transmission Section 431 Processing section load information 432 Physical server resource information 43 Processing schedule information 434 processor resources to be used information 435 processor arrangement information 436 processor decrease determination threshold information 437 physical server selection threshold information

Claims (4)

A distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, a processing unit that processes the distributed processing request, and a plurality of the physical units that arrange one or more of the processing units A distributed processing system comprising: a server; and a resource adjustment device that is connected to each of the distribution device, the processing unit, and the plurality of physical servers, and adjusts the number of the processing units arranged in the physical server. ,
Each of the processing units
A processing unit load amount information collecting unit that collects the amount of resources used or the number of processing cases of the processing unit itself,
A processing unit load amount information notifying unit that notifies the resource adjustment device of the collected resource amount used or the number of processing cases as processing unit load amount information,
The resource adjustment device includes:
(1) a first continuation period threshold value indicating a period during which an upper limit threshold value for the processing unit load amount information and a value equal to or greater than the upper limit threshold value for determining the addition of the processing unit; and (2) the process Processing unit increase / decrease determination threshold information for storing a lower limit threshold value for the processing unit load amount information and a second duration threshold value indicating a period during which a value lower than the lower limit threshold value continues is determined. A storage unit,
A processing unit load information receiving unit for receiving the processing unit load information from each of the processing units;
Based on the processing unit load amount information, a period in which the amount of resources used or the number of processing cases of processing units occupying a first predetermined ratio or more of all processing units of the distributed processing system is equal to or greater than the upper limit threshold Determining that the additional condition of the processing unit is satisfied if the first duration threshold is exceeded;
Based on the processing unit load amount information, a period in which the amount of used resources or the number of processing cases of processing units occupying a second predetermined ratio or more of all processing units of the distributed processing system is lower than the lower limit threshold is the first time. A processing unit increase / decrease determination unit that determines that the deletion condition of the processing unit is satisfied when the duration threshold value of 2 is exceeded,
When it is determined that the additional condition of the processing unit is satisfied, the physical server is controlled to add a processing unit, and when it is determined that the deletion condition of the processing unit is satisfied, the physical server A processing unit placement execution unit that controls to delete the processing unit;
When the processing unit is added, processing unit additional information including information on the processing unit to be added is generated and transmitted to the distribution device. When the processing unit is deleted, information on the processing unit to be deleted A processing unit addition / deletion information transmitting unit that generates and transmits processing unit deletion information including the processing unit deletion information,
The sorting device is
A storage unit that stores distribution destination information indicating the processing unit that is a distribution destination of the processing request;
The information of the processing unit to be added indicated in the processing unit additional information is registered in the distribution destination information, and the information of the processing unit to be deleted indicated in the processing unit deletion information is deleted from the distribution destination information. Processing part addition / deletion execution part,
A distributed processing system, comprising: a distribution processing unit that distributes the received processing request to the processing unit with reference to the distribution destination information. The distribution processing unit of the distribution device is
The round robin in which the distribution ratios of the processing requests to the respective processing units registered in the distribution destination information are set evenly, or the distribution ratios of the processing requests to the respective processing units, Determining the processing unit to be a distribution destination by weighted round robin set according to the processing capacity of the physical server where
The distributed processing system according to claim 1. A distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, a processing unit that processes the distributed processing request, and a plurality of the physical units that arrange one or more of the processing units A distributed processing system comprising: a server; and a resource adjustment device that is connected to each of the distribution device, the processing unit, and the plurality of physical servers, and adjusts the number of the processing units arranged in the physical server. ,
Each of the processing units
A processing unit load amount information collecting unit that collects the amount of resources used or the number of processing cases of the processing unit itself,
A processing unit load amount information notifying unit that notifies the resource adjustment device of the collected resource amount used or the number of processing cases as processing unit load amount information,
The resource adjustment device includes:
(1) a first continuation period threshold value indicating a period during which an upper limit threshold value for the processing unit load amount information and a value equal to or greater than the upper limit threshold value for determining the addition of the processing unit; and (2) the process Processing unit increase / decrease determination threshold information for storing a lower limit threshold value for the processing unit load amount information and a second duration threshold value indicating a period during which a value lower than the lower limit threshold value continues is determined. A storage unit,
A processing unit load information receiving unit for receiving the processing unit load information from each of the processing units;
Based on the processing unit load amount information, it is determined that an additional condition of the processing unit is satisfied when a period in which the amount of used resources or the number of processing cases is greater than or equal to the upper limit threshold exceeds the first duration threshold ,
A processing unit that, based on the processing unit load amount information, determines that the usage resource amount or the number of processing cases satisfies a deletion condition of the processing unit when a period in which the amount of used resources is less than the lower limit threshold exceeds the second duration threshold An increase / decrease determination unit;
When it is determined that the additional condition of the processing unit is satisfied, the physical server is controlled to add a processing unit, and when it is determined that the deletion condition of the processing unit is satisfied, the physical server A processing unit placement execution unit that controls to delete the processing unit;
When the processing unit is added, processing unit additional information including information on the processing unit to be added is generated and transmitted to the distribution device. When the processing unit is deleted, information on the processing unit to be deleted A processing unit addition / deletion information transmitting unit that generates and transmits processing unit deletion information including the processing unit deletion information,
The sorting device is
A storage unit that stores distribution destination information indicating the processing unit that is a distribution destination of the processing request;
The information of the processing unit to be added indicated in the processing unit additional information is registered in the distribution destination information, and the information of the processing unit to be deleted indicated in the processing unit deletion information is deleted from the distribution destination information. Processing part addition / deletion execution part,
A distribution processing unit that distributes the received processing request to the processing unit with reference to the distribution destination information;
The processing unit is configured by a virtual machine arranged on the physical server,
The physical server includes an active physical server that arranges a processing unit that executes processing for the processing request, and a physical server for a resource pool that arranges the processing unit in a standby state that does not accept the processing request. Configured,
Each of the physical servers
A physical server resource information collection unit that collects the used resource amount of the physical server itself;
A physical server resource information notification unit for notifying the resource adjustment device of the collected used resource amount as physical server resource information,
The resource adjustment device includes:
The storage unit further includes processing schedule information indicating a processing schedule indicating a period for which an application for processing the processing request is to be executed for each processing unit, and the processing unit and the processing unit. The processing unit arrangement information, which is information associating the physical server, the upper limit threshold for the used resource amount of the physical server, and the upper limit threshold for the scheduled use amount of the physical server calculated using the processing schedule information are stored. And physical server selection threshold information to be stored,
A physical server resource information receiving unit for receiving the physical server resource information from each of the physical servers;
When the processing unit increase / decrease determination unit determines that the additional condition of the processing unit is satisfied, for each of the physical servers, all the processing units arranged in the physical server are extracted based on the processing unit arrangement information. And calculating the scheduled use resource amount of each of the extracted processing units using the processing schedule information, and summing the calculated scheduled use resource amount for each of the processing units to obtain the scheduled use resource for each physical server. The amount of used resources indicated by the physical server resource information received from the physical server does not exceed the upper limit threshold for the amount of scheduled resource used of the physical server. Selecting a physical server that does not exceed the upper threshold for the amount of resource used by the physical server, and selecting the selected physical server Among the processing units that satisfy the deletion condition when the processing unit increase / decrease determination unit determines that the deletion condition of the processing unit is satisfied, while determining the physical server to which the processing unit is to be added. A placement destination physical server determination unit that determines a processing unit to be deleted from
When the processing unit increase / decrease determining unit determines that the additional condition of the processing unit is satisfied, the processing unit placement execution unit determines that the processing unit placed on the resource pool physical server is the placement destination physical server. When the determination unit determines that the processing unit increase / decrease determination unit satisfies the deletion condition of the processing unit, the processing unit to be deleted is controlled. Controlling to move to the physical server for the resource pool,
It said distributed processing system. A distribution device that distributes a processing request received from a client to a processing unit arranged in a physical server, a processing unit that processes the distributed processing request, and a plurality of the physical units that arrange one or more of the processing units Distributed processing of a distributed processing system comprising: a server; and a resource adjustment device that is connected to each of the distribution device, the processing unit, and the plurality of physical servers, and adjusts the number of the processing units arranged in the physical server A method,
Each of the processing units
A step of collecting the amount of resources used or the number of processing cases of the processing unit itself;
Performing the step of notifying the resource adjustment device of the collected use resource amount or the number of processing cases as processing unit load amount information;
The resource adjustment device includes:
(1) a first continuation period threshold value indicating a period during which an upper limit threshold value for the processing unit load amount information and a value equal to or greater than the upper limit threshold value for determining the addition of the processing unit; and (2) the process Processing unit increase / decrease determination threshold information for storing a lower limit threshold value for the processing unit load amount information and a second duration threshold value indicating a period during which a value lower than the lower limit threshold value continues is determined. A storage unit,
Receiving the processing unit load information from each of the processing units;
Based on the processing unit load amount information, a period in which the amount of resources used or the number of processing cases of processing units occupying a first predetermined ratio or more of all processing units of the distributed processing system is equal to or greater than the upper limit threshold Determining that the additional condition of the processing unit is satisfied when the first duration threshold is exceeded;
Based on the processing unit load amount information, a period in which the amount of used resources or the number of processing cases of processing units occupying a second predetermined ratio or more of all processing units of the distributed processing system is lower than the lower limit threshold is the first time. Determining that the deletion condition of the processing unit is satisfied if the duration threshold of 2 is exceeded;
Controlling to add a processing unit to the physical server when it is determined that the additional condition of the processing unit is satisfied;
A step of controlling the physical server to delete the processing unit when it is determined that the deletion condition of the processing unit is satisfied;
When adding the processing unit, generating processing unit additional information including information of the processing unit to be added, and transmitting the processing unit additional information to the distribution device;
When deleting the processing unit, generating processing unit deletion information including information of the processing unit to be deleted and transmitting to the sorting device,
The sorting device is
A storage unit for storing distribution destination information indicating the processing unit to which the processing request is allocated;
Registering information of the processing unit to be added indicated in the processing unit additional information in the distribution destination information;
Deleting the information of the processing unit to be deleted indicated in the processing unit deletion information from the distribution destination information;
A step of distributing the received processing request to the processing unit with reference to the distribution destination information.

Images