JP2015162216A - Decentralized processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a decentralized processing system capable of executing data duplication processing again even when data duplication processing which is asynchronous to request processing of data to be managed, is failed.SOLUTION: A processing part 42 of a server 4 being an original copy management server of a decentralized processing system 1000 updates original copy data which is stored in a storage part 41 by processing a request from a client machine 1, then transmits the updated original copy data to a server 4 which is a duplication management server in asynchronous to the request processing for making the server 4 create duplication data, monitors a creation state of the duplication data by the server 4 being the duplication management server. When creation of the duplication data is failed, the duplication management server which failed to create the duplication data is withdrawn from the decentralized processing system 1000, and another server 4 among plural servers serves as the new duplication management server, and the new duplication management server is made create the duplication data.

Description

The present invention relates to a data replication technique handled by a distributed processing system including a plurality of servers.

  In recent years, with the rise of cloud computing, it has been required to efficiently process and retain a large amount of data. Thus, distributed processing technology has been developed that realizes efficient processing by operating a plurality of servers in a coordinated manner.

  When performing distributed processing, it is necessary to distribute processing target (management target) data to each server constituting the cluster (hereinafter also referred to as “cluster member” or “member”). At this time, in order to increase the processing capacity of the entire cluster, it is desirable that the number of data (data amount) handled by each cluster member is averaged.

  As a typical data distribution method, the remainder obtained by dividing the value obtained by multiplying the key of each data by a hash function (hereinafter referred to as “hash (key)”) by the number N of cluster members, that is, “hash (key) mod N There is a method of distributing data to cluster members having "" as a number. In this case, it is assumed that numbers “0” to “N−1” are assigned in advance to each cluster member. When such a distribution method is used, if a cluster member is added, the value of N changes, and the cluster member in charge of a lot of data is changed, so that the data in charge must be rearranged.

  Therefore, as a method for suppressing the number of data that the cluster member in charge changes with the addition of the cluster member to about 1 / N, a distribution method using a consistent hashing method (see Non-Patent Document 1) is used. is there.

  In this data distribution method using the consistent hash method, an ID (IDentifier) is assigned to both the cluster member and data, and the cluster member first encountered when the circular ID space is traced clockwise from the data ID. Is in charge of the data.

  Also, when managing a large amount of data in a clustered distributed processing system, keep a copy of the data so that even if a failure occurs in one cluster member, processing can be continued on other cluster members. It is necessary to realize data redundancy. The same applies to a distributed processing system that uses a data management technique based on the consistent hash method.

As shown in FIG. 4, in the consistent hash method, IDs are assigned to both cluster members (members 1 to 4) and data (data A to D, indicated by black circles), and the ID space is clockwise from the data ID. , And the first cluster member encountered is determined to be responsible for the data.
Then, the cluster member that is further to the right of the cluster member in charge (following clockwise rotation) is assigned the replication data.

  For example, in FIG. 4, for data (original data) A, member 1 who first encounters by tracing clockwise in the ID space is in charge, and for duplicate data of data A, member 1 in ID space Member 2 on the right is in charge. In this way, by determining the cluster member in charge of the original data and the duplicated data, even if the cluster member has left, the cluster that owns the duplicated data of the data A handled by the detached cluster member There is an advantage that the member can cope with it by becoming a new cluster member in charge of the data A. When creating a plurality of duplicate data, the cluster member on the right can be assigned to the second duplicate data.

  Here, as a database server that can be used for such a technique, for example, a distributed database server described in Cassandra (Non-Patent Document 2) or the like, in which a plurality of servers can dynamically join and leave a cluster. Is mentioned. In the technique described in Non-Patent Document 2, uniform distribution and high-speed access of data using the consistent hash method is realized. Also, by providing duplicate data to a plurality of servers, fault tolerance can be improved. Having duplicate data for multiple servers can increase fault tolerance, but the processing load for replication, consistency of replication information among multiple servers, and the amount of data stored to hold replication information In general, a plurality of servers that hold the same replicated data are not provided.

  Also, for example, when processing a request with a strict response time such as a session control server is realized, it takes a long time to send a request response after synchronizing the response contents of the request and the replicated data. The response is returned first, and the copy data to be managed is created asynchronously with the response of the request, thereby securing a response time.

David Karger et al., “Consistent Hashing and Random Tres: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, [online], 1997, ACM, [February 28, 2014 search], Internet <URL : Http://dl.acm.org/citation.cfm?id=258660> Avinash Lakshman et al., “Cassandra-A Decentralized Structured Storage System”, [online], [searched February 28, 2014], Internet <URL: http://www.cs.cornell.edu/projects/ladis2009/ papers / lakshman-ladis2009.pdf>

  However, even if replication data creation fails due to processing delay in the cluster, etc., the server is normally replying to the request to the client machine, so the request cannot be retransmitted. Since the cluster is operated with the data and the replicated data unmatched, there is a risk that the data will be lost if the server that manages the original data goes down.

Therefore, the present invention has been made in view of the above circumstances, and a distributed processing system capable of re-execution of data replication processing even when data replication processing asynchronous with request processing of data to be managed fails. It is an issue to provide.

  In order to achieve the above object, the present invention provides a distributed processing system in which a plurality of servers manage data, the plurality of servers including an original management server that manages original data of the data, A copy management server for managing copy data, wherein the original management server stores the original data and also stores copy management information indicating which of the plurality of servers manages the data. The original data stored in the storage unit is updated by processing a request from a storage unit and a client machine, and the updated original data is transferred to the replication management server asynchronously with the processing of the request. A processing unit configured to transmit and create the replicated data, the processing unit configured to transmit the replicated data by the replication management server; The replication status is monitored, and when the replication data creation fails, the replication management server that failed to create the replication data is removed from the distributed processing system, and another server among the plurality of servers is newly replaced. The replication management server is configured such that a new replication management server creates the replication data.

  According to such a configuration, the replication data is created asynchronously with the request processing, and when the replication data creation fails, the replication management server that failed to create the replication data is separated from the distributed processing system, and a plurality of servers Since the other replication server is used as a new replication management server and the new replication management server creates replication data, it is possible to prevent the distributed processing system from operating with the original data and the replication data being unmatched. .

  The replication management server includes a processing unit that creates the replication data and stores the replication data in a storage unit of the replication management server, and the processing unit of the replication management server is configured to enable the original management server to create In the case of down, the copy data already created among the copy data to be created may be stored in the storage unit of the copy management server as the original data.

  According to this configuration, if the original management server goes down before the creation of all replicated data is completed, the created replicated data is stored in the storage unit of the relevant replication management server as the original data. Even when the original management server goes down, complete loss of data can be prevented.

  In the distributed processing system, IDs of a plurality of data to be processed and IDs of the plurality of servers that form a cluster and process requests related to the data are allocated to a circular ID (IDentifier) space, and the ID In the space, the data located between the ID of the data and the first server encountered in a predetermined direction is held as the original management server as the original data, The server other than the original management server may be the replication management server and hold the replication data.

  According to such a configuration, in a distributed processing system employing a so-called consistent hash method in which data and servers are arranged in a circular ID space, the distributed processing system is operated while the original data and the replicated data are unmatched. Can be prevented.

  According to the present invention, according to the present invention, data loss can be efficiently prevented even when replicated data is created asynchronously with request processing in a distributed processing system.

1 is a block diagram schematically showing a client machine and a distributed processing system according to an embodiment of the present invention. It is a figure which shows an example of ID space management information. It is a figure which shows an example of replication management information. It is a figure for demonstrating the server which manages original data and replication data in ID space of a consistent hash method. It is a flowchart for demonstrating the operation example of the original management server which concerns on embodiment of this invention. It is a flowchart for demonstrating the operation example of the replication management server which concerns on embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings as appropriate. Similar components are denoted by the same reference numerals, and redundant description is omitted.

<Distributed processing system>
As shown in FIG. 1, a distributed processing system 1000 according to an embodiment of the present invention includes a load distribution device 3 and a plurality of servers 4 that constitute a cluster 100. The load balancer 3 is connected to a plurality of client machines 1 via a network 2 such as the Internet.

  Here, main operations of the entire distributed processing system 1000 will be described. First, the load balancer 3 receives a request from the client machine 1 via the network 2. The load balancer 3 distributes the received request to one of the plurality of servers 4. The server 4 to which the request is distributed performs processing of the distributed request, and requests the other (four in the present embodiment) server 4 to store a copy of the data.

<Load distribution device>
Next, the configuration of the load distribution device 3 will be described. The load distribution device 3 includes a storage unit 31, a processing unit 32, an input unit 33, a display unit 34, and a communication unit 35.

  The storage unit 31 is a means for storing information, and includes a memory such as a random access memory (RAM) or a read only memory (ROM), a hard disk drive (HDD), or the like. The storage unit 31 stores ID space management information 311 and replication management information 312. The storage unit 31 also stores an operation program that embodies the processing unit 32. The ID space management information 311 is the same as the ID space management information 411 of the server 4 described later, and the replication management information 312 is the same as the replication management information 412 of the server 4 described later. Therefore, the description regarding the ID space management information 311 and the replication management information 312 is omitted.

  The processing unit 32 is means for performing arithmetic processing based on information stored in the storage unit 31, and is configured by, for example, a CPU (Central Processing Unit). The processing unit 32 updates the ID space management information 311 by, for example, leaving the server 4 to the cluster 100, and transmits the updated latest ID space management information 311 to all the servers 4.

  Further, the processing unit 32 determines which of the plurality of servers 4 the request from the client machine 1 received by the communication unit 35 is distributed based on the ID space management information 311.

  The input unit 33 is a means by which the administrator of the server 4 inputs information, and is realized by a keyboard, a mouse, or the like, for example.

  The display unit 34 is a means for displaying information, and is realized by, for example, an LCD (Liquid Crystal Display).

  The communication unit 35 is a communication interface used for communication with an external device.

<Server>
Next, the configuration of the server 4 will be described. The server 4 is a computer device that processes requests from the client machines 1 distributed by the load balancer 3. As described above, in the consistent hash method, a plurality of data to be managed and a plurality of servers 4 that manage the data and constitute the cluster 100 are allocated to the annular ID space. Functions as an original management server that stores and manages data (original data) positioned between itself and the next server 4 counterclockwise (predetermined direction) in the ID space, and the next server 4 It is assumed that the server functions as a replication management server that stores and manages data replication (replication data) located between the first server 4 and the next server 4 counterclockwise (in a predetermined direction).

  The server 4 includes a storage unit 41, a processing unit 42, and a communication unit 43.

  The storage unit 41 is a means for storing information, and includes a memory such as a RAM or a ROM, an HDD, or the like. The storage unit 41 stores original data and copy data corresponding to the server 4, and the ID space management information 311 and the copy management information 312 received from the load balancer 3 are the ID space management information 411 and Stored as copy management information 412. The storage unit 41 also stores an operation program that embodies the processing unit 42.

  The processing unit 42 is a unit that performs arithmetic processing based on information stored in the storage unit 41, and is configured by a CPU, for example. The processing unit 42 processes a request from the client machine 1 distributed by the load balancer 3. That is, the processing unit 42 updates the data (original data) corresponding to the request in the storage unit 41.

  The processing unit 42 further refers to the ID space management information 411 and requests the next server 4 to store a copy of the data used for processing the request from the ID space itself in a predetermined direction. Here, the processing unit 42 may be configured to request storage of a copy of only the original data updated by the request among the original data stored in the storage unit 41, and store the copy of all the original data. The structure which requests | requires may be sufficient. The processing unit 42 of the replication management server creates replication data (that is, receives the data transmitted from the processing unit 42 of the original management server as replication data), and converts the created replication data into the replication management server. The data is stored in the storage unit 41.

  In order to shorten the response time, the processing unit 42 returns a response to the request from the client machine 1 before the data storage in the next server 4 is completed, It shall be performed asynchronously with the processing. Further, the processing unit 42 updates the replication management information 412 according to the processing of the request and data replication, or transmits the update result of the replication management information 412 to the other server 4 and the load balancer 3. Can do.

  The communication unit 43 is a communication interface used for communication with an external device. The server 4 may include an input unit and a display unit.

  The ID space management information 411 is information for managing the server 4 in charge of the data to be managed using an ID calculated by a predetermined hash value conversion. The ID is a unique value indicating the position in the circular ID space, and is stored in order to specify the data area for which the server 4 is in charge of management, and is sorted according to the size of the ID value.

In the example shown in FIG. 2, the ID “9245D30E” is assigned to the server 4 whose IP address is “192.168.0.3” in the first line, and the server 4 has IDs “00000000” to “00000000” 9245D30E ”(ie, storing and managing original data of the area).
In the second line, the server 4 having the IP address “192.168.2.15” stores data belonging to the identifier of the value from the identifier obtained by adding 1 to the ID value of the previous line to the ID of the line. Indicates that you are in charge.

  The replication management information 412 is created for each data to be managed, and is information for managing the status of data replication. As shown in FIG. 3, the copy management information 412 includes a copy state and a copy version number associated with each of the IP address of the server 4 where the original data is stored and the IP address where the copy data is stored. Yes. For example, if the replication status is “0”, it indicates that the corresponding server 4 manages the original data of the data. If the replication status is “1”, the corresponding server 4 replicates the data. Indicates that data is being managed. In addition, since the data replication is performed asynchronously with the request from the client machine 1, the processing unit 42 of the server 4 that manages the original data relates to the original data at the time of generating the original data in order to copy additional differences as needed. The replication status can be updated by updating the replication version number and updating the replication version number regarding the replication data when the replication data creation is successful. In addition, the processing unit 42 of the server 4 that manages the original data can manage the presence or absence of replication failure by updating the replication status to “2” when replication fails. Further, the processing unit 42 of the server 4 updates the ID space management information 411 due to leaving the cluster 100 or the like, and when the IP address of the server 4 holding the original data is deleted, the ID space The IP address is deleted from the management information 411 and the replication management information 412, and the IP address of the server 4, which is a candidate for the next replication management server, is added to the replication management information 412. Further, the processing unit 42 of the server 4 indicates the replication status of the IP address of the server (replication management server) 4 that holds the replication data of the original data held by the server 4 that has left the cluster 100 in the replication management information 412. The server 4 is updated from “2” to “0”, and the server 4 is promoted from the replication management server to the original management server.

<Operation example of the original management server>
Next, taking the ID space shown in FIG. 4 as an example, processing by the server 4 that is an original management server (member 1) that manages certain data A as original data will be described.

  As shown in FIG. 5, in step S <b> 1, the processing unit 42 of the original management server (member 1) determines whether a request from the client machine 1 has been received from the load balancer 3. If Yes in step S1, the flow proceeds to step S2, and if No in step S1, the flow returns to step S1.

  In step S2, the processing unit 42 of the original management server (member 1) performs data processing on the received request. Specifically, the processing unit 42 updates the original data stored in the storage unit 41 based on the received request, and sends a reply to the request to the client machine 1 that transmitted the request.

  In step S 3, the processing unit 42 of the original management server (member 1) refers to the ID space management information 411 in the storage unit 41 asynchronously with the request from the client machine 1 and next to the ID space in the clockwise direction in the ID space. The server 4 is requested to store a copy of the data. Specifically, the processing unit 42 transmits the original data stored in the storage unit 41 to the adjacent server (member 2) 4 that is the duplicate data management server (member 2). When receiving the original data, the processing unit 42 of the server 4 that is the replication management server (member 2) stores the received original data in the storage unit 41 as the replicated data.

  In step S4, the processing unit 42 of the original management server (member 1) monitors a data replication completion notification (response) from the replication management server (member 2) asynchronously with the request from the client machine 1.

  When data replication is not normally completed in the replication management server (member 2) (No in step S4), the processing unit 42 of the original management server (member 1) stores the ID space management information 411 stored in the storage unit 41. , The next server (member 3) 4 is set as a new replication management server in the clockwise direction in the ID space, and the replication management server is requested to create replication data.

  That is, in FIG. 2, when the server 4 with the IP address “192.168.2.15” does not respond normally as a replication management server for the data A, the server 4 with the next IP address “192.168.8.5” This corresponds to a new replication management server for A.

  In step S4, when the creation of the duplicate data is not completed normally, the processing unit 42 of the replication management server transmits a replication failure notification and the processing unit 42 of the original management server receives the replication failure notification. The case where the processing unit 42 of the original management server cannot receive a normal completion notification within a certain time can be handled in the same manner.

  In step S4, when the processing unit 42 of the original management server receives a copy failure notification, the processing unit 42 may retransmit the entire original data as a retry of the difference update failure. In this case, the processing unit 42 resends the entire original data to the replication management server, and at the same time, executes processing in step S5 to request the creation of replication data for the next replication management server candidate. Can keep sex.

  In step S6, when the processing unit 42 of the original management server receives a copy data creation completion notification (Yes in step S6), in step S7, the processing unit 42 of the original management server removes the original replication management server from the cluster 100. Let go. Specifically, the processing unit 42 of the original management server refers to the ID space management information 411 and requests the management server to leave the original replication management server from the cluster 100. The management server deletes the information related to the original replication management server from its own ID space management information 411, and transmits the ID space management information 411 updated in this way to the other servers 4 and the load balancer 3. The processing unit 42 of the server 4 that has received the ID space management information 411 and the processing unit 32 of the load distribution apparatus 3 use the received ID space management information to store the ID space management information 411 and 311 in its own storage units 41 and 31. As a result, the departure of the original ID replication management server from the cluster 100 is completed. Here, the management server is designated in advance as the server 4 capable of updating the ID space management information 411 from among the servers 4 constituting the cluster 100. Each original management server may function as the management server.

  In step S6, when data replication is not normally completed in the new replication management server (member 3) (No in step S6), the processing unit 42 of the original management server (member 1) is stored in the storage unit 41. With reference to the stored ID space management information 411, the next server (member 4) 4 is set as a new replication management server in the clockwise direction in the ID space, and the replication management server is requested to create replication data. (Step S5).

  In step S <b> 6, in the example of FIG. 2, the management server deletes the line with the IP address “192.168.2.15” from the ID space management information 411. Note that the management server may start leaving the original replication management server from the cluster 100 without waiting for a copy data creation completion notification. In this case, the copy data stored in the original copy management server that has left is not converted into original data.

  In the case of Yes in step S4 and in the case of Yes in step S6, the processing unit 42 of the original management server updates the corresponding copy management information 412 and updates the update result with the update result of the copy management information 412. It transmits to the other server 4 and the load balancer 3.

<Operation example of replication management server>
Next, taking the ID space shown in FIG. 4 as an example, processing by the server 4 that is a replication management server (member 2) that manages replication data of a certain data A will be described.

  As shown in FIG. 6, in step S11, when the processing unit 41 of the replication management server (member 2) is requested to create replication data from the original management server (member 1), it creates replication data. When the original management server (member 1) transmits the original data, the processing unit 42 of the replication management server (member 2) receives the transmitted original data and stores the received original data in the storage unit 41 as the replicated data. .

  If the copy data creation is successful (Yes in step S12), in step S13, the processing unit 42 of the copy management server (member 2) sends a copy data creation completion notification to the original management server (member 1). Send.

  Further, if the creation of the duplicate data has failed (No in step S12), and the reason for the failure is not that the original management server (member 1) is down during the creation of the duplicate data (in step S14). In No), in step S15, the processing unit 41 of the replication management server (member 2) transmits a failure notification of replication data creation to the original management server (member 1).

  Further, if the creation of the duplicate data has failed (No in step S12), and the reason for the failure is that the original management server (member 1) is down during the creation of the duplicate data (in step S14). In step S16, the processing unit 41 of the replication management server (member 2) stores the created replication data among the replication data to be created in the storage unit 41 as original data (that is, the original data). Data that has been transmitted and received successfully before the management server goes down is stored in the storage unit 41, and the replication state of the replication management information 412 of the data is set to “0”), thereby eliminating the complete loss of data A prevent. In this case, the above-described separation of the replication management server in step S7 is not performed. Here, as a method for the replication management server to detect that the original management server is down, any method such as a method in which the replication management server and the original management server send and receive heartbeat signals between each other and monitor each other to detect the down is arbitrary. It is possible to adopt a known method. Further, the server 4 that has been promoted from the replication management server to the original management server can cause the original original management server to leave the cluster 100 in the same manner as the above-described separation of the original replication management server from the cluster 100.

  Further, after execution of step S13 and after execution of step S16, the processing unit 42 of the replication management server updates the corresponding replication management information 412 and updates the update result of the replication management information 412 to another server. 4 and the load balancer 3.

The distributed processing system 1000 according to the embodiment of the present invention creates replication data asynchronously with the request processing, and when the replication data creation fails, the replication management server that failed to create the replication data Since the other server 4 among the plurality of servers 4 constituting the cluster 100 is made a new replication management server and the new replication management server creates replication data, the original data and the replication data are It is possible to prevent the cluster 100 from being operated unmatched. Therefore, the distributed processing system 1000 can efficiently prevent data loss even when creating replicated data asynchronously with request processing.
Further, when the original management server goes down before the creation of all the replicated data is completed, the distributed processing system 1000 stores the created replicated data as the original data in the storage unit 41 of the relevant replication management server. Even if the original management server goes down during the creation of duplicate data, complete loss of data can be prevented.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it can change suitably. For example, the distributed processing system of the present invention is not limited to a system using the consistent hash method, and is applied to all systems in which original data held by the original management server is held as replication data by another server (replication management server). Is possible. In the above embodiment, the case where the redundancy is 2 (that is, one original management server and one replication management server for one data) has been described as an example. The present invention can also be applied to a distributed processing system having three or more (that is, one original management server and two or more replication management servers for one data).

100 clusters 1000 distributed processing systems 4 servers (original management server, replication management server)
41 Storage Unit 411 ID Space Management Information 412 Replication Management Information 42 Processing Unit

Claims (3)

A distributed processing system in which multiple servers manage data,
The plurality of servers are:
An original management server for managing original data of the data;
A replication management server for managing replication data of the data;
With
The original management server
A storage unit that stores the original data, and stores replication management information indicating which of the plurality of servers manages the data;
The original data stored in the storage unit is updated by processing a request from a client machine, and the updated original data is transmitted to the replication management server asynchronously with the processing of the request. It has a processing unit that creates data,
The processing unit monitors a state of creation of the replicated data by the replication management server, and when the replication data creation fails, leaves the replication management server that failed to create the replicated data from the distributed processing system. A distributed processing system, wherein another server of the plurality of servers is used as the new replication management server, and the new replication management server creates the replication data. The replication management server includes a processing unit that creates the replication data and stores the replication data in a storage unit of the replication management server,
When the original management server goes down during the creation of the duplicate data, the processing unit of the duplicate management server uses the duplicate data already created in the duplicate data to be created as the original data. The distributed processing system according to claim 1, wherein the storage unit of the replication management server stores the storage unit. An ID of the plurality of data to be processed and an ID of the plurality of servers that form a cluster and process requests related to the data are allocated to a circular ID (IDentifier) space, and the ID of the data in the ID space The data located between the server and the first server encountered in a predetermined direction is stored as the original data as the original management server, and other than the original management server in the cluster. The distributed processing system according to claim 1, wherein the server serves as the replication management server and holds the replicated data.

Images