JP2013182553A - Management device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a processing load generated during data duplication even when adding a server to a cluster for performing distributed processing.SOLUTION: A management device 4 in a distributed processing system 1000 can reduce a processing load generated during data duplication and achieve high-speed processing by selecting a server 5 physically existing in an area corresponding to a position in an ID space determined for a server 5 to be added when adding the server 5 constituting a cluster 100.

Description

  The present invention relates to a management apparatus that manages a plurality of servers constituting a cluster that performs data processing in a coordinated manner.

  In recent years, with the rise of cloud computing, it has been required to efficiently process and retain a large amount of data. Thus, distributed processing technology has been developed that realizes efficient processing by operating a plurality of servers in a coordinated manner.

  When performing distributed processing, it is necessary to distribute processing target (management target) data to each server constituting the cluster (hereinafter also referred to as “cluster member” or “member”). At this time, in order to increase the processing capacity of the entire cluster, it is desirable that the number of data (data amount) handled by each cluster member is averaged.

  As a typical data distribution method, a remainder obtained by dividing a value obtained by multiplying the key of each data by a hash function (hereinafter referred to as “hash (key)”) by the number N of cluster members, that is, “hash (key) mod N There is a method of distributing data to cluster members having "" as a number. In this case, it is assumed that numbers “0” to “N−1” are assigned to each cluster member in advance. When such a distribution method is used, if a cluster member is added, the value of N changes, and the cluster member in charge of a lot of data is changed, so that the data in charge must be rearranged.

  Therefore, as a method for suppressing the number of data that the cluster member in charge changes with the addition of the cluster member to about 1 / N, there is a distribution method using a consistent hashing method (see Non-Patent Document 1). is there. This consistent hash method is used in Amazon Dynamo (see Non-Patent Document 2) and the like.

  In this data distribution method using the consistent hash method, an ID (IDentifier) is assigned to both the cluster member and the data, and when the ID space is traced clockwise from the data ID, the first cluster member encountered is Take charge of data.

  In addition, when managing a large amount of data in a cluster-structured distributed processing system, data redundancy is maintained by maintaining a copy of the data so that even if a failure occurs in one cluster member, processing can be continued on other cluster members. Needs to be realized. The same applies to a distributed processing system that uses a data management technique based on the consistent hash method.

  As shown in FIG. 7, in the consistent hash method, IDs are assigned to both cluster members (members 1 to 4) and data (data A to D, indicated by black circles (●)), and the ID space is determined from the data ID. The cluster member meeting first is determined as the charge of the data. Then, the cluster member that is further to the right of the cluster member in charge (next clockwise) is assigned the duplicate data.

  For example, in FIG. 7, data A is traced clockwise in the ID space and is assigned to the member 1 that first encounters, and the duplicated data is assigned to the member 2 on the right side of the member 1 in the ID space. By determining the cluster member responsible for the original data / replicated data in this way, even if the cluster member leaves, the cluster member that owns the replicated data becomes a new cluster member responsible for the data. There is an advantage that you can. When a plurality of pieces of duplicate data are taken, the second duplicate data can be assigned to the cluster member on the right side.

David Karger et al., “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, [online], 1997, ACM, [searched February 20, 2012], Internet <URL: http://www.akamai.com/dl/technical_publications/ConsistenHashingandRandomTreesDistributedCachingprotocolsforrelievingHotSpotsontheworldwideweb.pdf> Giuseppe DeCandia et al., “Dynamo: Amazon's Highly Available Key-value Store”, [online], 2007, ACM, [Search February 20, 2012], Internet <URL: http://www.allthingsdistributed.com /files/amazon-dynamo-sosp2007.pdf>

  As described above, the data management method based on the consistent hash method employs a method in which replicated data is arranged on the right side of the cluster member in charge of data. Here, as shown in FIG. 8, the members constituting the cluster are selected from K (in this case, five) data center areas (areas managed by the data center that manages the server) that are geographically separated. Suppose that

  At this time, usually, in the ID space of the consistent hash method, the servers are randomly arranged without considering the physical position (physical position), so that adjacent cluster members in the ID space are physically connected to each other. May be far away. In that case, there is a concern that the network cost when creating duplicate data for the cluster member on the right side in the ID space becomes enormous, causing processing delay. The physical distance between cluster members represents the cost of the NW (network) required for server-to-server communication. As a specific example, not only the physical distance but also the topology, NW band, etc. are considered. It is possible to determine.

  Therefore, the present invention has been made in view of the above-described circumstances, and when the management apparatus in the distributed processing system determines which of a plurality of servers a request from a client machine is distributed to, the physical location of the server It is an object of the present invention to reduce the processing load that occurs during data replication by performing server selection in consideration of the above.

  In order to solve the above-described problem, the present invention allocates a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster in a circular ID space, and each of the servers includes: Managing the data located between itself and the next server around the predetermined direction in the ID space, and located between the next server and the next server around the predetermined direction. In a distributed processing system that stores a copy of data, a management device that determines to which of the plurality of servers a request from a client machine is distributed, and manages a server in charge of managing the data in the ID space ID space management information, the ID space is divided into a predetermined number of physical regions, and the regions and the ID / region allocation information for managing the association with the designated ID space, and server management for managing the association between the plurality of servers physically present in the region and the standby server for each region When adding the storage server that stores information and the standby server to the cluster, based on the insertion position in the ID space for the server to be added, refer to the ID / region allocation information. The area corresponding to the insertion position is specified, the server management information is referred to, a server physically existing in the specified area is selected, and the ID space is determined using the selected server and the insertion position information. And a processing unit for updating the management information.

  According to this, when adding a standby server to a cluster, a server that physically exists in an area corresponding to the insertion position in the ID space is selected, thereby reducing the processing load generated during data replication. be able to.

  Further, according to the present invention, a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster are allocated to the annular ID space, and each of the servers is assigned to itself in the ID space. Manages the data located between the next server around the predetermined direction and stores a copy of the data located between the next server and the next server around the predetermined direction. In a distributed processing system, a management device that determines to which of the plurality of servers a request from a client machine is assigned, ID space management information for managing a server in charge of managing the data in the ID space, The ID space is divided into a predetermined number of physical regions, and the region and the divided ID space Storage for storing ID / region allocation information for managing association, and server management information for managing association between the plurality of servers physically present in the region and a standby server for each region When the standby server is added to the cluster and the cluster, the ID space management information is referred to specify the widest portion between the servers in the ID space, and the ID / region allocation information The region corresponding to the center position of the location in the ID space is identified with reference to the server, the server management information is referenced to select a server physically existing in the identified region, and the selected server And a processing unit that updates the ID space management information using the information on the center position.

  According to this, when adding a standby server to a plurality of servers, selecting a server physically present in an area corresponding to the central position of the widest portion between the servers in the ID space. Thus, it is possible to reduce the processing load generated at the time of data replication and to further average the processing load on each server. In the present specification and claims, the “center position” includes not only the exact center position but also the position near the center.

  Further, the present invention is characterized in that the area is a data center area managed by a data center that manages the server.

  According to this, by adopting the data center area as a region, the operation of the cluster and the management apparatus becomes easier.

  Further, the present invention is a program for causing a computer to function as the management device.

  According to this, a computer in which such a program is installed can be caused to function as a management device.

  According to the present invention, when a management apparatus in a distributed processing system determines which of a plurality of servers a request from a client machine is distributed to, data replication is performed by performing server selection in consideration of the physical location of the server. The processing load sometimes generated can be reduced.

It is explanatory drawing of the outline | summary of this invention. It is a block diagram of the management apparatus etc. of this embodiment. It is a figure which shows ID space management information of this embodiment. It is a figure which shows ID and area allocation information of this embodiment. It is a figure which shows the server management information of this embodiment. It is a flowchart which shows the flow of the process by the management apparatus of this embodiment. It is explanatory drawing of the conventional consistent hash method. It is explanatory drawing of the physical position of the cluster member in the conventional consistent hash method.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings (refer to drawings other than the referenced drawings as appropriate). In order to facilitate understanding, the outline of the present embodiment will be described first with reference to FIG. 1, and then the embodiment will be described.

(Outline of this embodiment)
As shown in FIG. 1, the ID space is divided in advance according to the number of data center areas (here, 5). When a server is added to the cluster, a new cluster member insertion destination (synonymous with “insertion position”, ID) in the ID space is determined ((1) in FIG. 1).

  After determining the insertion destination of the new cluster member in the ID space ((1) in FIG. 1), the data center area corresponding to the insertion destination is specified, and the server physically existing in the data center area is selected. ((2) in FIG. 1), it is incorporated in the cluster member. With such an algorithm, server selection can be performed in consideration of the physical location of the server, the processing load generated during data replication can be reduced, and the processing load on each server can be further averaged.

(Embodiment)
Next, this embodiment will be described. As illustrated in FIG. 2, the distributed processing system 1000 according to the present embodiment includes a load distribution device 3, a management device 4, and a plurality of servers 5 configuring the cluster 100. The load balancer 3 is connected to a plurality of client machines 1 via a network 2 such as the Internet.

  The main operation will be described. The data distribution request from the client machine 1 is received by the load balancer 3 via the network 2. The load balancer 3 distributes the request to one of a plurality of servers 5 that perform data processing based on the server allocation table (ID space management information) on the data ID space. The distributed server 5 processes the request. The management device 4 manages ID space management information.

Next, the configuration of the load distribution device 3 and the management device 4 will be described.
The load distribution device 3 includes a storage unit 31, a processing unit 32, and a communication unit 33.
The storage unit 31 is a means for storing information, and includes a memory such as a random access memory (RAM) or a read only memory (ROM), a hard disk drive (HDD), or the like. The storage unit 31 stores ID space management information 411 received from the management device 4 as ID space management information 311. The storage unit 31 also stores an operation program for the processing unit 32 (not shown).

  The processing unit 32 is means for performing arithmetic processing based on information stored in the storage unit 31, and is configured by, for example, a CPU (Central Processing Unit).

The communication unit 33 is a communication interface used for communication with an external device.
In addition, the load balancer 3 may further include an input unit for the user of the load balancer 3 to input information, a display unit for displaying information, and the like.

  The management device 4 is a computer device that determines to which of the plurality of servers 5 a request from the client machine 1 is distributed based on the consistent hash method. As described above, in the consistent hash method, a plurality of data to be managed and a plurality of servers 5 (cluster members) that manage the data and configure the cluster 100 are allocated to the circular ID space. Each server 5 manages (responsible for) data located between itself and the next server 5 in the clockwise direction (predetermined direction) in the ID space, and further clockwise from the next server 5 It is assumed that a copy of the data located until the next server 5 is stored.

The management device 4 includes a storage unit 41, a processing unit 42, an input unit 43, a display unit 44, and a communication unit 45.
The storage unit 41 is a means for storing information, and includes a memory such as a RAM or a ROM, an HDD, or the like. The storage unit 41 stores ID space management information 411, ID / area allocation information 412, and server management information 413. The storage unit 41 also stores an operation program for the processing unit 42 (not shown).

  The ID space management information 411 is information for managing the server 5 in charge of data using an ID calculated by performing predetermined hash value conversion on the management target data (described later in FIG. 3).

  The ID / region allocation information 412 is information for managing the association between a region and the divided ID space under the assumption that the ID space is divided into a predetermined number of physical regions ( (See below in FIG. 4).

  The server management information 413 is information for managing the association with a plurality of servers 5 physically present in the area for each area (described later in FIG. 5).

  The processing unit 42 is a unit that performs arithmetic processing based on information stored in the storage unit 41, and is configured by a CPU, for example. When adding a new (standby) server 5 to the cluster 100, the processing unit 42 first determines an insertion position in the ID space for the new server 5. Next, the processing unit 42 refers to the ID / region allocation information 412 to identify the region corresponding to the insertion position. Next, the processing unit 42 refers to the server management information 413 and selects the server 5 that physically exists in the specified area. Next, the processing unit 42 updates the ID space management information 411 using the selected server and the information on the insertion position. Details will be described later with reference to FIG.

  Further, when adding a new server 5 to the cluster 100, the processing unit 42 refers to the ID space management information 411 as an example, and identifies the place where the space between the servers 5 is the widest in the ID space. In addition, the processing unit 42 refers to the ID / region allocation information 412 to identify the region corresponding to the center position of the location in the ID space. Next, the processing unit 42 refers to the server management information 413 and selects the server 5 that physically exists in the specified area. Next, the processing unit 42 updates the ID space management information 411 using the information on the selected server and the center position. Details will be described later with reference to FIG.

The input unit 43 is a means for the user of the management device 4 to input information, and is realized by, for example, a keyboard or a mouse.
The display unit 44 is a means for displaying information, and is realized by an LCD (Liquid Crystal Display), for example.
The communication unit 45 is a communication interface used for communication with an external device.

Next, the ID space management information 411 will be described. As shown in FIG. 3, the ID space management information 411 includes ID and server columns, and is sorted according to the size of the ID value.
The ID is an ID in the ID space, and is stored in order to specify an area of data for which the server 5 is in charge of management.
The server represents an identifier of the server 5 constituting the cluster 100.

  For example, when the ID value in the first row shown in FIG. 3 is “0056”, this indicates that “server A” is in charge of data belonging to the area having identifiers “0000” to “0056”. When the ID value of the second row is “0172”, “Server B” stores the data belonging to the identifiers “0057” to “0172” obtained by adding 1 to the ID value of the previous row. Indicates that you are in charge.

Next, the ID / area allocation information 412 will be described. As shown in FIG. 4, the ID / region assignment information 412 includes regions ID and ID columns.
The region ID represents an identifier of a predetermined plurality of physical regions.
ID represents a range of IDs in the ID space corresponding to the area of the area ID in the row.

Next, the server management information 413 will be described. As shown in FIG. 5, the server management information 413 is composed of each column of area ID and server.
The region ID represents an identifier of a predetermined plurality of physical regions.
The server represents the identifier of the server 5 that physically exists in the area of the corresponding area ID. The server 5 includes servers that have not been used as the cluster 100 yet.

Next, processing by the management device 4 will be described.
As shown in FIG. 6, in step S1, the processing unit 42 of the management device 4 determines whether or not there is a cluster configuration change request. If Yes, the process proceeds to step S2, and if No, the process returns to step S1. . The cluster configuration change request here may be, for example, an input to that effect by the user of the management apparatus 4 or a request to that effect from another apparatus.

  In step S2, the processing unit 42 determines whether or not the cluster configuration change request is to add a cluster member. If Yes, the process proceeds to step S3, and if No, the process proceeds to step S7.

  In step S3, the processing unit 42 determines the insertion destination (ID) of the cluster member to be added in the ID space.

  Where to insert may be designated by the maintenance person through the input unit 43, or the processing unit 42 can calculate by any method. Note that the position to be selected in the ID space may be arbitrary, and the essential matter is to select the server 5 that physically exists in the data center area (region) corresponding to the position selected in the ID space. As a result, server selection considering the physical location of the server 5 can be performed, and the processing load generated during data replication can be reduced.

  For example, the processing unit 42 refers to the ID space management information 411 stored in the storage unit 41 to search (specify) a location where the distance between cluster members (distance on the ID space) is the largest in the ID space, and The center position between the cluster members is determined as the insertion destination of the new cluster member ID. The reason for selecting such a position is that, in a distributed processing system with a cluster configuration, it is desirable that the load among the cluster members be averaged as much as possible. Statistically, the number of data is much larger than the number of cluster members. This is because the number of data handled by each cluster member is almost proportional to the distance between the cluster members in the ID space when there are many.

  Subsequently, in step S <b> 4, the processing unit 42 refers to the ID / region allocation information 412 in the storage unit 41 and identifies the data center area (region ID) that is responsible for the determined ID.

  Subsequently, in step S5, the processing unit 42 refers to the server management information 413 in the storage unit 41, and physically exists in the data center area (that is, corresponds to the area ID specified in step S4). 5 is selected.

  In step S6, the processing unit 42 updates the ID space management information 411 and ends the process. Specifically, when going through steps S3 to S5, in the ID space management information 411, a new line is inserted, and the corresponding ID and the identifier of the server 5 to be added are written. For example, when a part of the cluster member is deleted due to a failure or the like without going through steps S3 to S5, the corresponding line is deleted in the ID space management information 411.

Thereafter, the load distribution device 3 receives the latest ID space management information 411 from the management device 4 and stores it as the ID space management information 311 in the storage unit 31. Then, the load balancer 3 distributes the request received from the client machine 1 to one of the plurality of servers 5 by the consistent hash method based on the ID space management information 311.
Further, the plurality of servers 5 receive the latest ID space management information 411 from the management apparatus 4 and store it in a storage unit (not shown), and replicate the data described above based on the consistent hash method.

  As described above, according to the management device 4 of the present embodiment, when adding the server 5 constituting the cluster 100, the server physically existing in the area corresponding to the position in the ID space determined for the server 5 to be added. By selecting 5, it is possible to reduce the processing load (network cost) generated at the time of data replication and to realize high-speed processing. This method is particularly effective when the frequency of data update or replication is high or when the data size to be replicated is large.

Moreover, when determining the position in the ID space for the server 5 to be added, by selecting the central position of the widest portion between the servers 5 in the ID space, the processing load generated at the time of data replication is reduced, The load on each server 5 can be further averaged.
Further, by adopting the data center area as a region, the operation of the cluster 100 and the management apparatus 4 becomes easier.

Although description of this embodiment is finished above, the aspect of the present invention is not limited to these.
For example, the management device 4 and the load distribution device 3 may be configured to coexist on the same hardware.
In the present embodiment, the consistent hash method is assumed, but another method may be assumed.

  Further, without using the load balancer 3, each client machine 1 holds the ID space management information 411 received from the management device 4 and directly accesses any of the plurality of servers 5 via the network 2. Also good.

Moreover, although the case where the data center area is used as a unit as an area has been described as an example, another unit such as a further divided data center area or a prefecture may be adopted.
The present invention can also be embodied as a program for causing a computer to function as the management device 4.
In addition, about a concrete structure, it can change suitably in the range which does not deviate from the main point of this invention.

DESCRIPTION OF SYMBOLS 1 Client machine 2 Network 3 Load distribution apparatus 4 Management apparatus 5 Server 31 Storage part 32 Processing part 33 Communication part 41 Storage part 42 Processing part 43 Input part 44 Display part 45 Communication part 100 Cluster 311 ID space management information 411 ID space management information 412 ID / area allocation information 413 Server management information 1000 Distributed processing system

Claims (4)

In a circular ID (IDentifier) space, a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster are allocated, and each of the servers rotates in a predetermined direction from itself in the ID space. A distributed processing system for managing the data located between the next server and storing a copy of the data located between the next server and the next server around the predetermined direction. In the management device for determining which of the plurality of servers the request from the client machine is distributed,
ID space management information for managing a server in charge of managing the data in the ID space;
ID / region allocation information for managing the association between the region and the divided ID space, wherein the ID space is divided into a predetermined number of physical regions; and
A storage unit that stores, for each region, server management information that manages the association between the plurality of servers physically present in the region and the standby server;
When adding the standby server to the cluster, the region corresponding to the insertion position is identified by referring to the ID / region allocation information based on the insertion position in the ID space for the added server. A processing unit that refers to the server management information, selects a server that physically exists in the specified area, and updates the ID space management information using the selected server and the information on the insertion position;
A management apparatus comprising: In a circular ID (IDentifier) space, a plurality of data to be managed and a plurality of servers that manage the data and constitute a cluster are allocated, and each of the servers rotates in a predetermined direction from itself in the ID space. A distributed processing system for managing the data located between the next server and storing a copy of the data located between the next server and the next server around the predetermined direction. In the management device for determining which of the plurality of servers the request from the client machine is distributed,
ID space management information for managing a server in charge of managing the data in the ID space;
ID / region allocation information for managing the association between the region and the divided ID space, wherein the ID space is divided into a predetermined number of physical regions; and
A storage unit that stores, for each region, server management information that manages the association between the plurality of servers physically present in the region and the standby server;
When adding the standby server to the cluster, refer to the ID space management information to identify the widest part between the servers in the ID space, and refer to the ID / region allocation information. The region corresponding to the center position of the location in the ID space is identified, the server management information is referenced to select a server physically existing in the identified region, and the selected server and the center A processing unit for updating the ID space management information using the position information of
A management apparatus comprising:   3. The management apparatus according to claim 1, wherein the area is a data center area managed by a data center that manages the server.   The program for functioning a computer as a management apparatus as described in any one of Claim 1- Claim 3.

Images