JP5723309B2 - Server and program - Google Patents

Description

  The present invention relates to a technology for data replication in a plurality of servers constituting a cluster that performs data processing in a coordinated manner.

  In recent years, with the rise of cloud computing, it has been required to efficiently process and retain a large amount of data. Thus, distributed processing technology has been developed that realizes efficient processing by operating a plurality of servers in a coordinated manner.

  When performing distributed processing, it is necessary to distribute processing target (management target) data to each server constituting the cluster (hereinafter also referred to as “cluster member” or “member”). At this time, in order to increase the processing capacity of the entire cluster, it is desirable that the number of data (data amount) handled by each cluster member is averaged.

  As a typical data distribution method, a remainder obtained by dividing a value obtained by multiplying the key of each data by a hash function (hereinafter referred to as “hash (key)”) by the number N of cluster members, that is, “hash (key) mod N There is a method of distributing data to cluster members having "" as a number. In this case, it is assumed that numbers “0” to “N−1” are assigned to each cluster member in advance. When such a distribution method is used, if a cluster member is added, the value of N changes, and the cluster member in charge of a lot of data is changed, so that the data in charge must be rearranged.

  Therefore, as a method for suppressing the number of data that the cluster member in charge changes with the addition of the cluster member to about 1 / N, there is a distribution method using a consistent hashing method (see Non-Patent Document 1). is there. This consistent hash method is used in Amazon Dynamo (see Non-Patent Document 2) and the like.

  In this data distribution method using the consistent hash method, an ID (IDentifier) is assigned to both the cluster member and the data, and when the ID space is traced clockwise from the data ID, the first cluster member encountered is Take charge of data.

  In addition, when managing a large amount of data in a cluster-structured distributed processing system, data redundancy is maintained by maintaining a copy of the data so that even if a failure occurs in one cluster member, processing can be continued on other cluster members. Needs to be realized. The same applies to a distributed processing system that uses a data management technique based on the consistent hash method.

  As shown in FIG. 5, in the consistent hash method, IDs are assigned to both cluster members (members 1 to 4) and data (data A to D, indicated by black circles (●)), and the ID space is determined from the data ID. The cluster member meeting first is determined as the charge of the data. Then, the cluster member that is further to the right of the cluster member in charge (next clockwise) is assigned the duplicate data.

  For example, in FIG. 5, data A follows the ID space in the clockwise direction, and the member 1 who meets first is in charge, and the duplicate data is assigned to the member 2 on the right side of the member 1 in the ID space. By determining the cluster member responsible for the original data / replicated data in this way, even if the cluster member leaves, the cluster member that owns the replicated data becomes a new cluster member responsible for the data. There is an advantage that you can. When a plurality of pieces of duplicate data are taken, the second duplicate data can be assigned to the cluster member on the right side.

David Karger et al., “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, [online], 1997, ACM, [searched February 20, 2012], Internet <URL: http://www.akamai.com/dl/technical_publications/ConsistenHashingandRandomTreesDistributedCachingprotocolsforrelievingHotSpotsontheworldwideweb.pdf> Giuseppe DeCandia et al., “Dynamo: Amazon's Highly Available Key-value Store”, [online], 2007, ACM, [searched February 20, 2012], Internet <URL: http://www.allthingsdistributed.com /files/amazon-dynamo-sosp2007.pdf>

Here, when a specific server allocation policy for cluster members in the ID space is determined, there are cases where it is desired to operate with two different policies instead of one.
As described above, the data management method based on the consistent hash method employs a method in which replicated data is arranged on the right side of the cluster member in charge of data. Here, as shown in FIG. 6, the members constituting the cluster are selected from K (in this case, five) data center areas (areas under the control of the data center that manages the server) that are geographically separated. Suppose that

  Consider, for example, a case where a server cannot be used in units of data center areas due to a catastrophic disaster, a large-scale failure, or the like (hereinafter referred to as a “severe catastrophe”). At this time, a large number of servers in the cluster are removed simultaneously. Usually, in the ID space of the consistent hash method, the servers are randomly arranged without considering the physical position. Therefore, if the server to be removed is a cluster member adjacent in the ID space, the original Both data and duplicated data are lost (when there is only one duplicated data).

On the other hand, considering the communication time and the like when creating duplicate data for the cluster member on the right side in the ID space, it is better that the cluster members adjacent in the ID space are physically close to each other.
That is, if an ID space is created while considering the physical location of the server, an ID space that takes into account high-speed data replication, an ID space that takes into account a catastrophic disaster, and the like can be created. However, the method using a single ID space cannot achieve both high-speed data replication and severe disaster countermeasures, for example.

  Therefore, the present invention has been made in view of the circumstances described above, and a server and a program for enabling a plurality of different allocation policies for servers as cluster members to be operated in a distributed processing system. The issue is to provide.

  In order to solve the above-described problem, the present invention allocates a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster in a circular ID space, and each of the servers includes: Managing the data located between itself and the next server around the predetermined direction in the ID space, and located between the next server and the next server around the predetermined direction. In a distributed processing system for storing a copy of data, the server processes requests from client machines distributed by a load balancer, and each of the plurality of servers includes a plurality of predetermined physical regions of three or more. The first ID space is divided into the number of the regions, and each divided portion A storage unit that stores first ID space management information in which the servers in the same area are arranged, and second ID space management information arranged so that the servers in the same area are not adjacent to each other in the second ID space; The request from the client machine distributed by the load balancer is processed, and the request is processed from the self in the first ID space to the next server around the predetermined direction with reference to the first ID space management information. Requesting storage of a copy of the data used in the process, and referring to the second ID space management information from the self in the second ID space to the next server around the predetermined direction, the data used for processing the request And a processing unit that requests storage of a copy.

  According to this, by using the first ID space management information having the first ID space for high-speed data replication and the second ID space management information having the second ID space for severe disaster countermeasures, high-speed data replication and severe disaster countermeasures can be performed. A server in a compatible distributed processing system can be realized. That is, by using a plurality of different ID spaces, it is possible to provide a server that can operate a plurality of different server allocation policies at the same time.

  Further, the present invention provides the first ID space management information in which the first ID space is divided into the number of the regions, and the server in the same region is arranged in each divided part. Referring to requesting the next server to store a copy of the data used for processing the request around the predetermined direction from itself in the first ID space, the second ID is performed synchronously with the processing of the request. For processing the request to the next server around the predetermined direction from itself in the second ID space with reference to the second ID space management information arranged so that the servers in the same area are not adjacent to each other in the space When requesting storage of a copy of the received data, it is performed asynchronously with the processing of the request.

  According to this, data replication using the first ID space management information with a short processing time is performed synchronously with the request processing, and data replication using the second ID space management information with a long processing time is the same as the request processing. By doing so, the possibility of processing delay in the server can be reduced.

  Further, the present invention is a program for causing a computer to function as the server.

  According to this, a computer in which such a program is installed can be caused to function as a server.

  According to the present invention, by using a plurality of different ID spaces in a distributed processing system, it is possible to provide a server and a program for enabling a plurality of different server allocation policies to be operated simultaneously.

It is explanatory drawing of the outline | summary of this invention. It is a block diagram of the distributed processing system etc. of this embodiment. (A) is a figure which shows the example of 1st ID space management information, (b) is a figure which shows the example of 2nd ID space management information. It is a flowchart which shows the flow of the process by the server of this embodiment. It is explanatory drawing of the conventional consistent hash method. It is explanatory drawing of the physical position of the cluster member in the conventional consistent hash method.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings (refer to drawings other than the referenced drawings as appropriate). In order to facilitate understanding, the outline of the present embodiment will be described first with reference to FIG. 1 and then described in detail.

(Outline of this embodiment)
In this embodiment, as an example of a policy with different allocation of servers that are cluster members, a policy of high-speed data replication and catastrophic disaster countermeasures will be described.
First, as shown in FIG. 1A, the first ID space for high-speed data replication is divided in advance according to the number K of data center areas (a natural number of 3 or more, here 5). When adding a server to a cluster, the insertion destination of a new cluster member in the ID space is determined. For example, the location where the distance between the cluster members (distance in the first ID space) is the largest in the first ID space is searched (specified), and the center position between the cluster members is inserted into the new cluster member ID insertion destination. (1 in FIG. 1A). The reason for selecting such a position is that, in a distributed processing system with a cluster configuration, it is desirable that the load among the cluster members be averaged as much as possible. Statistically, the number of data is much larger than the number of cluster members. This is because the number of data handled by each cluster member is almost proportional to the distance between the cluster members in the ID space when there are many.

  After determining the insertion destination of a new cluster member in the first ID space ((1) in FIG. 1A), the data center area corresponding to the insertion destination is specified and physically exists in the data center area. A server is selected ((2) in FIG. 1A) and incorporated into a cluster member. With such an algorithm, server selection can be performed in consideration of the physical location of the server, high-speed data replication can be realized, and the processing load on each server can be further averaged.

  In addition, when adding a server to a cluster, the center position of the place where the distance between cluster members is the widest in the first ID space is selected, but this is only an example. That is, when adding a server to a cluster, the position selected in the first ID space may be arbitrary, and the essential item is a data center area corresponding to the position selected in the ID space (hereinafter also referred to as “region”). Is to select a server that physically exists. Thereby, high-speed data replication can be realized.

  Next, as shown in FIG. 1B, in the second ID space for catastrophic disaster countermeasures, first, the servers in the same region are distributed as shown so that the servers in the same region are not adjacent (in order for each region). To line up.

  Then, when adding a server, the location where the distance between the cluster members (distance on the second ID space) is the largest in the second ID space is searched (specified), and the center position between the cluster members is determined as a new cluster. It is determined as a member ID insertion destination ((1) in FIG. 1B).

  After determining the insertion destination of a new cluster member in the second ID space ((1) in FIG. 1 (b)), the area of K-1 servers around the insertion destination (only at least both neighbors are acceptable). Is investigated ((2) in FIG. 1B). Then, a server in a region to which none of the K-1 (at least, both adjacent) servers belongs is selected ((3) in FIG. 1B), and the server is added.

  The “periphery K−1 servers” specifically refers to {(K−1) / 2} servers for both sides of the insertion destination when K is an odd number. When K is an even number, it means (K / 2) servers on one side of the insertion destination and {(K-2) / 2} servers on the other side.

  With such an algorithm, servers in the same region do not adjoin in the second ID space, so that data is lost even if all servers existing in one of the three or more regions go down. Can be avoided. In addition, the processing load on each server can be further averaged.

  To explain again, due to the characteristics of the consistent hash method, the “probability of losing any one of the data (also with duplicates)” is “continuous M (redundant number (total number of original data and duplicated data))” It is equal to the probability that servers in the same area will be placed. According to the algorithm described in FIG. 1B, the probability can always be 0% when M ≧ 2. That is, even if one area is completely destroyed (all servers existing in that area are down), if M is 2 or more, no data is lost (duplicated data always remains).

  In addition, in the case of a method of examining the area of “K-1 surroundings” servers as well as “both neighbors” of the insertion destination, when K is an odd number, any continuous {(K + 1) / 2 in the second ID space } All servers belong to different regions. When K is an even number, any continuous (K / 2) servers in the second ID space all belong to different areas. As a result, if the number of redundancy is 3 or more, even if all the servers in one region are down, the expected value of the number of servers holding duplicate data of the original data held by the down server is more Can be bigger.

  For example, if K = 5 and M = 3, even if one area is completely destroyed, any server that holds duplicate data of the original data held by the server in that area is not in that area. They belong to different regions and keep replica data. That is, in this case, even if two areas are completely destroyed, one server always holds duplicate data.

  If the values of K and M are further increased, it is possible to increase the possibility that duplicate data has not been lost even if more areas are completely destroyed. For example, when K = 7 and M = 4, even if three areas are completely destroyed, one server always holds duplicate data.

  In addition, when adding a server to a cluster, the center position of the place where the distance between cluster members is the widest in the second ID space is selected, but this is only an example. That is, when adding a server to a cluster, the position to be selected in the second ID space may be arbitrary, and an essential matter is to select a server in a region to which neither of the servers adjacent to the selected position in the second ID space belongs. That is. Thereby, even if all the servers existing in one of the three or more regions go down, a situation in which data is lost can be avoided. In other words, it has the effect of catastrophic disaster countermeasures.

(Embodiment)
Next, this embodiment will be described. As shown in FIG. 2, the distributed processing system 1000 according to the present embodiment includes a load distribution device 3 and a plurality of servers 4 constituting a cluster 100. The load balancer 3 is connected to a plurality of client machines 1 via a network 2 such as the Internet.

  The main operation will be described. The load distribution device 3 receives a request from the client machine 1 via the network 2. The load balancer 3 distributes the request to one of the plurality of servers 4. The server 4 to which the request has been distributed processes the request and requests the other two servers 4 to store a copy of the data.

Next, the configuration of the load balancer 3 and the server 4 will be described.
The load distribution device 3 includes a storage unit 31, a processing unit 32, an input unit 33, a display unit 34, and a communication unit 35.
The storage unit 31 is a means for storing information, and includes a memory such as a random access memory (RAM) or a read only memory (ROM), a hard disk drive (HDD), or the like. The storage unit 31 stores first ID space management information 311 and second ID space management information 312. The first ID space management information 311 is the same as the first ID space management information 411, the second ID space management information 312 is the same as the second ID space management information 412, and the first ID space management information 411 and the second ID space management information 412 are the same. Will be described later. The storage unit 31 also stores an operation program for the processing unit 32 (not shown).

  The processing unit 32 is means for performing arithmetic processing based on information stored in the storage unit 31, and is configured by, for example, a CPU (Central Processing Unit). When the processing unit 32 updates the first ID space management information 311 by adding the server 4 to the cluster 100 or the like, the processing unit 32 transmits the latest first ID space management information 311 to all the servers 4. Further, when the second ID space management information 312 is updated by adding the server 4 to the cluster 100 or the like, the processing unit 32 transmits the latest second ID space management information 312 to all the servers 4.

  The processing unit 32 also determines to which server 4 the request from the client machine 1 received by the communication unit 35 is distributed. The determination of the distribution is based on the first ID space management information 311 or the second ID space management information 312. Which is used is determined in advance uniquely for the system.

The input unit 33 is a means for the user of the server 4 to input information, and is realized by, for example, a keyboard or a mouse.
The display unit 34 is a means for displaying information, and is realized by, for example, an LCD (Liquid Crystal Display).
The communication unit 35 is a communication interface used for communication with an external device.

  The server 4 is a computer device that processes requests from the client machines 1 distributed by the load balancer 3. As described above, in the consistent hash method, a plurality of data to be managed and a plurality of servers 4 that manage the data and constitute the cluster 100 are allocated to the circular ID space, and each server 4 manages data located between itself and the next server 4 in the clockwise direction (predetermined direction) in the ID space, and the next server is further rotated clockwise (predetermined direction) from the next server 4 It is assumed that a copy of data located between 4 and 4 is stored. Each of the plurality of servers 4 belongs to one of a plurality of predetermined three or more physical regions.

The server 4 includes a storage unit 41, a processing unit 42, and a communication unit 43.
The storage unit 41 is a means for storing information, and includes a memory such as a RAM or a ROM, an HDD, or the like. The storage unit 41 stores first ID space management information 311 and second ID space management information 312 received from the load balancer 3 as first ID space management information 411 and second ID space management information 412, respectively. The storage unit 41 also stores an operation program for the processing unit 42 (not shown).

  The processing unit 42 is a unit that performs arithmetic processing based on information stored in the storage unit 41, and is configured by a CPU, for example. The processing unit 42 processes a request from the client machine 1 distributed by the load balancer 3. The processing unit 42 further refers to the first ID space management information 411 and requests the next server 4 to store a copy of the data used for the processing of the request around the predetermined direction from itself in the first ID space. The processing unit 42 further refers to the second ID space management information 412 and requests the next server 4 to store a copy of the data used for the processing of the request around the predetermined direction from itself in the second ID space.

  In addition, when the processing unit 42 refers to the first ID space management information 411 and requests the next server 4 to store a copy of the data used for processing the request around the predetermined direction from itself in the first ID space, It is preferable to synchronize with the processing of the request. In addition, when the processing unit 42 refers to the second ID space management information 412 and requests the next server 4 to store a copy of the data used for processing the request around the predetermined direction from itself in the second ID space, It is preferable to execute the request asynchronously (at a convenient timing).

  Thus, data replication using the first ID space management information 411 with a short processing time is performed in synchronization with the request processing, and data replication using the second ID space management information 412 with a long processing time is the same as the request processing. If this is done, the possibility of processing delay in the server 4 can be reduced.

The communication unit 43 is a communication interface used for communication with an external device.
The server 4 may include an input unit and a display unit.

Next, the first ID space management information 411 will be described. In the first ID space management information 411, the first ID space is divided into the number of regions, and the servers 4 in the same region are arranged in each divided part (see FIG. 1A).
FIG. 3A shows the ID in the ID space and the IP address information as the identifier of the server 4 for the first ID space management information 411.

  The first ID space management information 411 is information for managing the server 4 in charge of data to be managed using an ID calculated by a predetermined hash value conversion. The ID is an ID in the ID space, and is stored in order to specify an area of data for which the server 4 is in charge of management, and is sorted by the size of the ID value.

  For example, in FIG. 3A, when the ID value in the first row is “446D0BE6”, the data belonging to the area with the identifiers “00000000” to “446D0BE6” and the IP address “192.168.8.0”. .10 "indicates that the server 4 is in charge. In the case of the second row, the data belonging to the identifier of the value from the identifier obtained by adding 1 to the ID value of the previous row to the ID of the row, the IP address “192.168.8.0. 25 "indicates that the server 4 is in charge.

  Next, the second ID space management information 412 will be described. In the second ID space management information 412, the servers 4 in the same area are arranged so as not to be adjacent to each other in the second ID space (see FIG. 1B). FIG. 3B shows information on the ID in the ID space and the IP address as the identifier of the server 4 for the second ID space management information 412.

Next, processing by the server 4 will be described.
As shown in FIG. 4, in step S <b> 1, the processing unit 42 of the server 4 determines whether or not the request from the client machine 1 has been received from the load balancer 3. If yes, the process proceeds to step S <b> 2. If no, the process returns to step S1.
In step S2, the processing unit 42 performs data processing for the request.

  In step S <b> 3, the processing unit 42 refers to the first ID space management information 411 in the storage unit 41 and requests the next server 4 to store a copy of the data in the clockwise direction in the first ID space.

  In step S <b> 4, the processing unit 42 refers to the second ID space management information 412 in the storage unit 41, and requests the next server 4 to store the data copy in the clockwise direction in the second ID space.

  As described above, according to the present embodiment, the first ID space management information 411 having the first ID space for high-speed data replication and the second ID space management information 412 having the second ID space for catastrophic disaster countermeasures are used. Thus, the server 4 in the distributed processing system 1000 capable of achieving both high-speed data replication and severe disaster countermeasures can be realized. In other words, by using a plurality of different ID spaces, it is possible to provide the server 4 that can operate a plurality of different allocation policies of the server 4 in a compatible manner, and the distributed processing system 1000 can be operated flexibly. Become.

  In the server 4, data replication using the first ID space management information 411 with a short processing time is performed synchronously with the request processing, and data replication using the second ID space management information 412 with a long processing time is the same as the request processing. If it is performed on an aperiodic basis, the possibility of processing delay can be reduced.

Although description of this embodiment is finished above, the aspect of the present invention is not limited to these.
For example, in this embodiment, the case where two different ID spaces reflecting two policies of high-speed data replication and catastrophic disaster countermeasures are used as an example. However, the present invention is not limited to this, and a plurality of ID spaces of different policies are used. Since the effect of reducing the possibility of data loss or the like can be expected simply by using it, a plurality of different ID spaces based on various policies can be used.

  Moreover, although the case where the data center area is used as a unit as an area has been described as an example, another unit such as a further divided data center area or a prefecture may be adopted.

In the present embodiment, the consistent hash method is assumed, but another method may be assumed.
The present invention can also be embodied as a program for causing a computer to function as the server 4.
In addition, specific configurations and processes can be appropriately changed without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 Client machine 2 Network 3 Load distribution apparatus 4 Server 31 Storage part 32 Processing part 33 Input part 34 Display part 35 Communication part 41 Storage part 42 Processing part 43 Communication part 100 Cluster 311 1st ID space management information 312 2nd ID space management information 411 First ID space management information 412 Second ID space management information 1000 Distributed processing system

Claims (3)

In a circular ID (IDentifier) space, a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster are allocated, and each of the servers rotates in a predetermined direction from itself in the ID space. A distributed processing system for managing the data located between the next server and storing a copy of the data located between the next server and the next server around the predetermined direction. In the server for processing a request from a client machine distributed by the load balancer,
Each of the plurality of servers belongs to one of a plurality of predetermined physical areas of three or more,
1st ID space management information by which the 1st ID space is divided | segmented into the number of the said area | region, and the said server of the same area is arrange | positioned in each of the said division | segmentation, and
A storage unit for storing second ID space management information arranged so that the servers in the same region are not adjacent to each other in the second ID space;
The request from the client machine distributed by the load balancer is processed, and the request is processed from the self in the first ID space to the next server around the predetermined direction with reference to the first ID space management information. Requesting storage of a copy of the data used in the process, and referring to the second ID space management information from the self in the second ID space to the next server around the predetermined direction, the data used for processing the request A processing unit requesting storage of a copy;
A server comprising: The processor is
When referring to the first ID space management information and requesting the next server to store a copy of the data used for processing the request around the predetermined direction from itself in the first ID space, When synchronously referring to the second ID space management information and requesting the storage of a copy of the data used for processing the request to the next server around the predetermined direction from itself in the second ID space, The server according to claim 1, wherein the server performs the request asynchronously. Program for functioning as the server according to the computer to claim 1 or claim 2.

Images