JP2015162061A - distributed processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a time to be spent on multistage processing.SOLUTION: A distributed processing system includes a plurality of cluster systems as a clustered server group, each custer system which being configured in a multistage, and executes the processing of request information received from the outside. In the distributed processing system, a server 1 includes: a storage part 12 for storing distribution information 121 in which information associating key information included in the request information with a server of the distribution destination of the request information including the key information is stored for the whole cluster system; and a distribution processing part 111 for, when receiving the request information, determining the server of the cluster system in the next stage associated with the key information included in the request information by referring to the distribution information 121.

Description

  The present invention relates to a distributed processing system that performs processing by combining cluster systems that are clustered server groups in multiple stages.

  In recent years, in order to easily improve the resilience and scalability of server failures, a distributed processing system has been studied in which a plurality of servers are logically connected to appear as if they were one server. Yes.

  In the distributed processing system, as described in Non-Patent Document 1, one process is divided into a plurality of processes, and one process is assigned to one cluster system for each process. Has been disclosed (see FIG. 1).

Satoru Kondo, 1 other person, “A study on exclusive control of cache data between clusters”, IEICE technical report, IEICE, September 2012, vol.112, No.208, NS2012-53 , P.13-18

  However, a distributed processing system that executes multi-stage processing advances the processing of each stage in order, and returns the final result processed in the final stage as response information, so that the response time becomes longer as the number of stages becomes deeper There's a problem.

  For example, in a distributed processing system, in order to achieve load distribution without affecting the client side, a load balancer using a simple distribution algorithm such as round robin is placed in the front stage of the server group to form a cluster system This method is generally used.

  Therefore, when a cluster system is configured by placing a load balancer in front of the server group, packets (request information) transmitted from user terminals operated by the user are used for the load balancer of the first (first) cluster system. Via, it is once transferred to one of the servers in the first-stage cluster system. Then, the server to which the packet is transferred determines again the processing destination server of the packet in the first-stage cluster system by a distributed algorithm such as consistent hashing used in the distributed database or the like (distribution) Process). Then, a processing destination server in the first-stage cluster system executes processing corresponding to the request. When the processing destination server ends, the processing destination server transfers the processing result to the load balancer provided in the next second-stage cluster system.

In this way, the second stage load balancer, the distribution process in the second stage cluster system and the request process for processing the request information, ..., the distribution in the nth stage load balancer, the nth stage cluster system Processing and request processing are executed. Then, the result of the request processing in the n-th stage is the response information from the n-th stage cluster system, the (n−1) -th stage cluster system,..., The second-stage cluster system, the first-stage cluster. It is sent back to the client terminal via the system.
Therefore, the response time becomes longer as the number of steps becomes deeper.

  Therefore, an object of the present invention is to provide a technique for reducing the time required for multistage processing.

  The distributed processing system of the present invention is a distributed processing system that includes a plurality of cluster systems that are clustered server groups, configures the cluster system in multiple stages, and executes processing of request information received from the outside, A storage unit storing distribution information in which the server associates key information included in the request information with a server to which the request information including the key information is allocated for all the cluster systems; and A distribution processing unit that, when receiving the request information, refers to the distribution information and determines a server of the next-stage cluster system associated with the key information included in the request information. .

  According to such a configuration, the distributed processing system can directly transmit / receive information between cluster systems between servers. That is, the distributed processing system can reduce the number of hops that pass through the server as compared to the case where the load balancer always passes through. Therefore, the distributed processing system can reduce the time required for multistage processing.

  The cluster system in the distributed processing system includes a load balancer that distributes the request information to a server in the cluster system, and the server includes a distribution information changing unit that changes the distribution information. In the case of renewing the first cluster system, the distribution information changing unit includes the server in the cluster system other than the first cluster system in the distribution information before renewing the first cluster system. Is changed to information with the load balancer of the first cluster system as the distribution destination.

  According to such a configuration, the distributed processing system can change the distribution destination to the load balancer when renewing the cluster system. As a result, when the entire cluster system is renewed, the load balancer can be instantly switched as a single point. The reason why switching can be performed instantaneously is that each server can specify the distribution destination of all cluster systems as a load balancer. As described above, since the distributed processing system can change the distribution destination of all the cluster systems, it is possible to reduce the time required for the above-described multistage processing and to easily update the cluster system.

  According to the present invention, the time required for multistage processing can be reduced.

It is a figure which shows the structural example of a distributed processing system. It is a figure which shows the function example of a server. It is a figure which shows an example of distribution information. It is a figure which shows the process example of a distributed processing system. It is a figure which shows the example of a process from the cluster system renewal to the swap of a cluster system. It is a figure which shows the example of a process after the swap of a cluster system. It is a figure which shows an example of the distribution information at the time of cluster system renewal. It is a figure which shows the process example which always passes through a load balancer as a comparative example.

  A mode for carrying out the present invention (hereinafter referred to as “the present embodiment”) will be described in detail with reference to the drawings as appropriate.

(Distributed processing system)
First, a configuration example of the distributed processing system 6 will be described with reference to FIG.
In FIG. 1, the distributed processing system 6 includes seven servers 1a, 1b, 1c, 1d, 1e, 1f, and 1g, and three load balancers 2a, 2b, and 2c. The number of servers and the number of load balancers need not be limited to the numbers shown in FIG. However, the distributed processing system 6 needs to include at least one load balancer in order to receive request information from the outside. In the following description, reference numeral 1 is used when the servers are not particularly distinguished. In addition, when the load balancer is not particularly distinguished, reference numeral 2 is used.

A solid line connecting the servers 1 and a solid line connecting the server 1 and the load balancer 2 indicate that communication is possible.
In FIG. 1, the servers 1a, 1b, and 1c are clustered (clustered) to form a cluster system 3a. Note that the cluster system is constructed by linking processes on a server, and allows a plurality of processes to operate on a single server 1. For example, the first processes of the servers 1d and 1e are clustered to form the cluster system 3b. Further, the second processes 1f and 1g of the server 1e are clustered to form a cluster system 3c. In the following description, reference numeral 3 is used when the cluster systems are not particularly distinguished.
The first process and the second process of the server 1e can be realized by, for example, a virtual machine (VM).

  A load balancer 2a is arranged in the cluster system 3a. The cluster system 3b can include a load balancer 2b. The cluster system 3c can include a load balancer 2c. The load balancer 2 has a function of distributing received packets to the servers 1 based on a distribution algorithm such as round robin, and realizes load distribution of the servers 1 in the cluster system 3.

  In the present embodiment, when the distributed processing system 6 receives request information (packets) such as data processing (for example, storage, reference, update) from the user terminal 4 operated by the user, the first-stage cluster system 3a. The request process is executed on the server 1. Next, the distributed processing system 6 issues request information to the second-stage cluster system 3b during the first-stage processing and waits for a response to the processing result. Further, it is assumed that the distributed processing system 6 executes the request processing by taking over the processing result of the second stage to the cluster system 3c of the third stage. The flow of this process is an example, and the request information may be issued to the cluster system 3c during the first stage process.

  The server 1 has a function of executing processing for request information, determining a server 1 as a distribution destination of the next-stage cluster system 3 in the process, transmitting new request information to the determined server 1, and waiting for a response. . In addition, when the server 1 is deployed in the last-stage cluster system, the server 1 has a function of executing a request process and returning the process result as response information. Details of the functions of the server 1 will be described later.

  When receiving the request information, the load balancer 2 executes a process of distributing the request information to the server 1 in the cluster system 3 in which the load balancer 2 is based on a simple distribution algorithm such as round robin. However, the request information does not always have to pass through the load balancer 2 and may be received directly by the server 1.

  The user terminal 4 is a personal computer, for example, and is operated by the user. The user terminal 4 has a function of transmitting request information to the distributed processing system 6 and receiving response information corresponding to the request information based on a user operation.

  The administrator terminal 5 is a computer, for example, and is used when maintaining the distributed processing system 6. For example, the administrator terminal 5 is used for setting or changing distribution information 121 (described later) that is referred to when determining a distribution destination server 1 that is in charge of processing request information.

(server)
Next, a function example of the server 1 will be described with reference to FIG.
The server 1 includes a processing unit 11, a storage unit 12, and a communication unit 13.
The storage unit 12 is a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc). The storage unit 12 stores distribution information 121.

  The distribution information 121 is information that associates the key information included in the request information with the server 1 that is the distribution destination of the request information including the key information, and stores the information for all the cluster systems 3. The key information is, for example, a URL (Uniform Resource Locator) or an IP address. When a plurality of processes are operating on one server 1, the distribution information 121 stores the relationship between the processes activated on the servers 1 of all the cluster systems 3 and the key information.

An example of the distribution information 121 will be described with reference to FIG.
In FIG. 3, the distribution information 121 is expressed by a hash space of consistent hashing. As shown in FIG. 3, in the cluster system 3a, the assigned areas of the servers 1a to 1c are represented corresponding to the hash value of the key information. In addition, regarding the cluster system 3b, the areas in charge of the first processes of the server 1d and the server 1e are shown corresponding to the hash value of the key information. For the cluster system 3c, the second process of the server 1e and the areas in charge of the servers 1f and 1g are shown corresponding to the hash value of the key information.

  If the distribution information 121 is a server 1 for which the assigned area of the hash space is not indicated, the server 1 does not execute the request processing, so the number of servers 1 operating in the distributed processing system 6 Can be limited. Specifically, since the server 1a has a responsible area only in the cluster system 3a, it can be seen that only the request processing of the cluster system 3a is in charge and the request processing of the cluster systems 3b and 3c is not executed. As in the example shown in FIG. 3, when the server 1 executes only one specific type for each cluster system 3 or a small number of types of processes such as the server 1e, the specific type This process can monopolize the memory area. Therefore, the number of servers 1 can be adjusted to the number that can obtain the most parallel effect.

  The distribution information 121 is not limited to being expressed in a consistent hashing hash space as shown in FIG. 3, but may be expressed by a distribution method using a relational database, a DHT (Distributed Hash Table), or the like. Absent. In addition, the distribution information 121 stores a predetermined calculation formula for determining the distribution destination server 1 corresponding to the key information, and each time the key information is acquired, the distribution processing unit 111 (described later) performs the predetermined calculation. You may make it determine a distribution destination by calculating a type | formula.

  Returning to FIG. 2, the processing unit 11 includes a CPU (Central Processing Unit) and a main memory (not shown), expands an application program stored in the storage unit 12 to the main memory, and distributes the distribution processing unit 111 and the request. The processing unit 112 and the distribution information changing unit 113 are embodied.

  When receiving the request information, the distribution processing unit 111 has a function of referring to the distribution information 121 and transmitting the request information to the distribution destination server 1 associated with the key information included in the request information. Further, the distribution processing unit 111 can be called during the request processing by the request processing unit 112 (described later). The distribution processing unit 111 refers to the distribution information 121 and is assigned to the next distribution destination associated with the new key information. The server 1 is determined, and the processing result is transmitted to the determined next-stage server 1. The distribution processing unit 111 is not necessarily the server 1 in the next stage, but the software 1 that can process the new request information operates according to the processing content, and the server 1 in the further stage (cluster system 3). You may make it decide.

  The request processing unit 112 has a function of executing a predetermined process on the received request information and returning a response.

  The distribution information changing unit 113 has a function of receiving the instruction information transmitted from the administrator terminal 5 and changing the assigned area of the server 1 stored in the distribution information 121.

  The communication unit 13 is an interface for performing communication with an external device or the like. The communication unit 13 transmits / receives information to / from the user terminal 4 and the administrator terminal 5, transmits the processing result of the processing unit 11 to the other server 1 and the load balancer 2, and transmits to the other server 1 and the load balancer. 2 has a function of receiving information from.

(Processing example)
Next, a processing example of the distributed processing system 6 will be described with reference to FIG.
In step S1, the user terminal 4 transmits request information (packets) such as data processing (for example, storage, reference, update).

  In step S2, the load balancer 2a of the first-stage cluster system 3a receives the request information and distributes the received request information to the servers 1 in the cluster system 3a by round robin or the like. FIG. 4 illustrates the case of distribution to the server 1c.

  In step S3, the server 1c refers to the distribution information 121 and determines the server 1 in the distribution destination cluster system 3a that is in charge of the request processing. FIG. 4 illustrates the case where the server 1a is determined.

In step S4, the server 1a executes request processing.
In step S5, the server 1a refers to the distribution information 121 and determines the server 1 in the cluster system 3b at the next stage of the distribution destination in charge of request processing. FIG. 4 illustrates the case where the server 1d is determined.
Further, the server 1a determines the server 1 of the cluster system 3c at the further stage where software capable of processing new request information operates according to the processing content. FIG. 4 illustrates the case where the server 1e is determined.

In step S6, the server 1d executes request processing.
In step S7, the server 1d refers to the distribution information 121 and determines the server 1 in the cluster system 3c at the next stage of the distribution destination in charge of request processing. FIG. 4 illustrates the case where the server 1f is determined.

In step S8, the server 1e and the server 1f execute request processing.
Although not shown in FIG. 4, when the cluster system 3c is in the final stage, the server 1e and the server 1f return processing results, and the server 1 that issued the request information is also processed in the backtrack. Reply the result.

(Processing example in comparative example)
Here, as a comparative example, a processing example in the case of always passing through the load balancer 2 arranged in each cluster system will be described with reference to FIG.

In step S71, the user terminal 4 transmits request information (packet).
In step S72, the load balancer 2a of the first-stage cluster system 3a receives the request information, and distributes the received request information to the servers 1 in the cluster system 3a by round robin or the like. FIG. 7 illustrates the case of distribution to the server 1c.

In step S73, the server 1c determines the server 1 in the distribution destination cluster system 3a that is in charge of request processing. FIG. 7 illustrates a case where the server 1a is determined.
In step S74, the server 1a executes request processing. Then, the server 1a transmits the processing result to the load balancer 2b of the cluster system 3b at the next stage.

  In step S75, the load balancer 2b of the second-stage cluster system 3b receives the request information, and distributes the received request information to the servers 1 in the cluster system 3b by round robin or the like. FIG. 7 illustrates the case of distribution to the server 1e.

In step S76, the server 1e determines the server 1 in the distribution destination cluster system 3b that is in charge of request processing. FIG. 7 illustrates a case where the server 1d is determined.
In step S77, the server 1d executes request processing. Then, the server 1d transmits the processing result to the load balancer 2c of the next-stage cluster system 3c.

  In step S78, the load balancer 2c of the third-stage cluster system 3c receives the request information, and distributes the received request information to the servers 1 in the cluster system 3c by round robin or the like. FIG. 7 illustrates the case of distribution to the server 1g.

In step S79, the server 1g determines the server 1 in the distribution destination cluster system 3c that is in charge of request processing. FIG. 7 illustrates the case where the server 1f is determined.
In step S80, the server 1f executes request processing.

  As described above, the distributed processing system 6 of the present invention reduces the number of hops passing through the load balancer 2 and the server 1 as compared to the comparative example shown in FIG. 7 as shown in FIG. Specifically, in FIG. 4, the maximum number of hops from the user terminal is “5”, whereas in the comparative example illustrated in FIG. 7, the number of hops from the user terminal 4 is “9”. That is, the distributed processing system 6 determines the server 1 of the next-stage cluster system 3 as a distribution destination by storing each server 1 in the distribution information 121 storing information about the areas in charge of all the servers 1, Processing results can be directly transmitted and received between the servers 1. As a result, the distributed processing system 6 can reduce the time required for multistage processing.

(Renewal of cluster system)
Next, when the cluster system 3 is updated when the software of the server 1 is updated, it is necessary to ensure the consistency of the data of the server 1 in the cluster system before and after the update. A method for facilitating ensuring the consistency will be described below.

  First, a processing example when the cluster system 3 is renewed will be described with reference to FIGS. 5A and 5B. In the present embodiment, a case will be described in which the cluster system 3b is renewed. FIG. 5A shows an example of processing from before the cluster system renewal to the swap of the cluster system. FIG. 5B shows an example of processing after swapping of the cluster system.

As shown in FIG. 5A, when the (old) cluster system 3b is to be swapped with the (new) cluster system 3d, the (old) cluster system 3b is changed to the (new) cluster system by using a single point load balancer 2. If it is changed to 3d at a stretch, it can be switched instantaneously. In other words, if there is a single point, it becomes easier to make the data consistent before and after the renewal.
Therefore, the following method is used.

  First, in step S51 of FIG. 5A, the distribution information changing unit 113 of the server 1 of the cluster system 3a and the cluster system 3c rewrites its distribution information 121 and changes the distribution destination in the cluster system 3b to the load balancer 2b. To do. Specifically, the distribution information 121 is changed as shown in “after S51” shown in the upper part of FIG. This change is executed by the distribution information changing unit 113 based on the instruction information received from the administrator terminal 5 (see FIG. 1).

  In step S52 of FIG. 5A, after the servers 1h and 1i into which new software is introduced and the load balancer 2d are activated, the load balancer 2b of the (old) cluster system 3b is changed to the load balancer 2d of the (new) cluster system 3d. Swap. The swap method is to change the distribution information of each server 1 of the other cluster system 3 by exchanging the IP address outside the cluster system 3 of the load balancer 2 between the load balancer 2b and the load balancer 2d. There is a way to realize without. The distribution information 121 after the swap is as shown in “after S52” shown in the middle of FIG. For example, a new assigned area is set for the servers 1h and 1i of the swapped (new) cluster system 3d. Further, in the second processes of the servers 1a to 1c and the server 1e and the servers 1f to 1g, the distribution destination is changed to the load balancer 2d of the (new) cluster system 3d. Note that the second process of the servers 1a to 1c and the server 1e and the change of the servers 1f to 1g are executed by the distribution information changing unit 113 based on the instruction information received from the administrator terminal 5 (see FIG. 1).

  Next, in step S53 of FIG. 5B, the distribution information changing unit 113 of the server 1 of the cluster system 3a and the cluster system 3c rewrites its distribution information 121, and the distribution destination is changed to the server 1 of the (new) cluster system 3d. change. Specifically, the distribution information 121 is changed as shown in “after S53” shown in the lower part of FIG. This change is executed by the distribution information changing unit 113 based on the instruction information received from the administrator terminal 5 (see FIG. 1).

In steps S51 to S53 described above, the administrator terminal 5 has been described as transmitting instruction information. However, the administrator terminal 5 may receive an instruction from the activated (new) cluster system 3d. .
In this way, the distributed processing system 6 can easily execute the data consistency before and after the renewal of the cluster system 3 by changing the distribution information 121 stored in each server 1.

  As described above, the distributed processing system 6 according to the present embodiment can directly transmit and receive information between the cluster systems 3 between the servers 1. That is, the distributed processing system 6 can reduce the number of hops that pass through the server 1 as compared to the case where the load balancer 2 always passes through. Therefore, the distributed processing system 6 can reduce the time required for multistage processing. Further, the distributed processing system 6 can change the distribution destination to the load balancer 2 when the cluster system 3 is renewed. As a result, the distributed processing system 6 can easily update the cluster system 3.

1, 1a to 1i server 2, 2a to 2c, 2d load balancer 3, 3a to 3c, 3d cluster system 4 user terminal 5 administrator terminal 6 distributed processing system 11 processing unit 12 storage unit 13 communication unit 111 distribution processing unit 112 request Processing unit 113 Distribution information changing unit 121 Distribution information

Claims (2)

A distributed processing system comprising a plurality of cluster systems that are clustered server groups, configured in multiple stages, and processing request information received from outside,
The server
A storage unit storing distribution information in which information relating the key information included in the request information and a server to which the request information including the key information is allocated is stored for all the cluster systems;
When receiving the request information, with reference to the distribution information, a distribution processing unit that determines a server of the next-stage cluster system associated with the key information included in the request information;
A distributed processing system comprising: The cluster system includes a load balancer that distributes the request information to the servers in the cluster system,
The server includes a distribution information changing unit that changes the distribution information,
When renewing the first cluster system among the cluster systems,
Before the renewal of the first cluster system, the distribution information change unit uses the distribution information to specify the server in the cluster system other than the first cluster system as a distribution destination. The distributed processing system according to claim 1, wherein the information is changed to information with a load balancer of the system as a distribution destination.

Images