JP2017220846A - Maintenance reduction system, node and maintenance reduction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten the processing time required for maintenance reduction of separating a node from a cluster.SOLUTION: Each of nodes 1 of a maintenance reduction system 1000 constituting a cluster comprises: a reduction node determination part 12 for determining a reduction node indicating a node 1 subjected to reduction from among multiple nodes 1; a distribution destination change processing part 151 for generating temporary distribution ID information 200 indicating each of nodes 1 excluding the reduction node in charge of data; an exclusive synchronizing processing part 152 for using the temporary distribution ID information 200 to extract original promotion target data and executing synchronizing processing between the original promotion target data and original data; an original promotion processing part 153 for promoting the original promotion target data into original data and updating the temporary distribution ID information; and a data rearrangement processing part 154 for executing duplication and rearrangement of data.SELECTED DRAWING: Figure 2

Description

  In the present invention, in a highly available system (distributed processing system) in which a plurality of nodes on a network are configured in a cluster configuration, some nodes of the cluster are removed from the cluster (reduced) in response to an instruction from the maintenance person. The present invention relates to a maintenance / reduction system, a node, and a maintenance / reduction method.

  In recent years, with the rise of cloud computing, it has been required to efficiently process and retain a large amount of data. Thus, distributed processing technology has been developed that realizes efficient processing by operating a plurality of servers in a coordinated manner.

  When performing distributed processing, it is necessary to determine data to be handled by each server (hereinafter referred to as “node”) constituting a distributed processing system having a cluster configuration. At this time, in order to increase the processing capability of the entire distributed processing system, it is desirable that the number of data handled by each node is averaged.

  As a representative data management method, a remainder obtained by dividing a value obtained by multiplying the key of each data by a hash function (hereinafter referred to as “hash (key)”) by the number of nodes N, that is, “hash (key) mod N”. There is a method in which a node having a number as a number manages data. In this case, it is assumed that numbers “0” to “N−1” are assigned to each node in advance. When such a management method is used, when a node is added or removed, the value of N changes, and the node in charge of storing the data changes for a lot of data. It will be necessary to rearrange.

  Therefore, as a method for suppressing the number of data that the node in charge changes with the addition / detachment of a node to about 1 / N, a data management method using a consistent hashing method (see Non-Patent Document 1). There is.

  In this data management method using the consistent hash method, IDs (IDentifiers) are assigned to both nodes and data. Then, when the closed ID space is traced clockwise from the ID of the data, the node that hits first is assumed to be in charge of the data. An example of how to give an ID to a node is a value (hash (IP address)) obtained by multiplying an IP address by a hash function. In the data management method using the consistent hash method, the area on the ID space where each node is responsible for processing the data is managed based on an ID table ("sorting ID information" described later).

  In a distributed processing system with a cluster configuration as described above, for example, in order to periodically check a server (node) or to disconnect a node suspected of malfunctioning, an instruction from a maintenance person (network administrator), etc. As a trigger, it has a function of safely leaving the corresponding node from the cluster before a specific node fails. This function is called “maintenance reduction”.

The maintenance reduction is conventionally performed by the following procedure.
(1) The start of reduction is determined based on an instruction from the maintenance person and predetermined logic for determining the reduction of the nodes constituting the cluster.
(2) Using the temporary ID table from which the node to be removed (hereinafter referred to as “removed node”) is deleted, signal distribution of the reduced node is stopped (hereinafter referred to as “distribution destination changing process”). .)
(3) The original data copy (replicated data) is rearranged so as to maintain redundancy (hereinafter referred to as “data rearrangement”).
(4) Promote replicated data to original data (hereinafter referred to as “original promotion”).
(5) The reduced node is updated to the deleted ID table (hereinafter referred to as “ID table update”).
(6) Disconnect the removed node from the cluster.

The above procedure will be described in detail below.
FIG. 7 is a diagram showing a maintenance reduction processing sequence in a conventional distributed processing system (high availability system).
Here, the network management device 5 is a device that manages the entire nodes 1 constituting the distributed processing system. Further, each node 1 constituting the cluster includes a reduction node 1b that is a node to be removed, an existing node 1a that is a node other than the reduction node 1b among the nodes 1 that constitute the cluster, and an existing node In 1a, there is a representative node 1A that manages the ID table and operates as a representative of the node 1.

  First, the representative node 1A determines the reduced node 1b as a node for maintenance reduction (step S1). The process for determining the node 1 to be removed is determined, for example, by receiving reduction instruction information from the network management device 5, or determined according to the state of each node 1 (contents of alarm notification and number of alarms). The failure probability received from an external device (details will be described later) can be used.

  Next, the representative node 1A transmits a distribution destination change request to the network management device 5 so that the signal is not distributed to the determined reduction node 1b (step S2). Then, the network management device 5 changes, for example, the setting of the distribution destination of the load balancer as the distribution source so that the signal is not distributed to the reduced node 1b among the nodes 1 constituting the distributed processing system. (Step S3). Subsequently, the network management device 5 transmits a distribution destination change completion notification to the representative node 1A (step S4).

  The representative node 1A that has received the distribution destination change completion notification generates a temporary ID table from which the reduced node 1b has been deleted (step S5). In this temporary ID table, a reduction flag “+1” is set indicating that the ID table is obtained by deleting the reduction node 1b from the current ID table. Subsequently, the representative node 1A notifies the generated temporary ID table to the existing node 1a and the reduced node 1b (step S6). Then, in the existing node 1a and the reduced node 1b, a temporary ID table is acquired and stored in the storage unit (step S7). Next, the existing node 1a and the reduced node 1b transmit a completion notification indicating that the temporary ID table has been acquired to the representative node 1A (step S8).

  When the representative node 1A receives a completion notification indicating that the temporary ID table has been acquired from all the existing nodes 1a and the reduced node 1b, the representative node 1A switches the current ID table to the temporary ID table (step S9). Then, the representative node 1A transmits a notification of switching to the temporary ID table to the existing node 1a and the reduced node 1b (step S10). Subsequently, the existing node 1a and the reduced node 1b switch the current ID table to the temporary ID table (step S11). Then, the existing node 1a and the reduced node 1b transmit a completion notification indicating that switching to the temporary ID table is completed to the representative node 1A (step S12).

  When the representative node 1A receives a completion notification indicating switching to the temporary ID table from all the existing nodes 1a and the reduced node 1b, the representative node 1A executes data relocation processing based on the temporary ID table (step S13). . Specifically, the representative node 1A checks whether or not it is necessary to rearrange the replicated data so as to maintain redundancy based on the temporary ID table, and executes the rearrangement of the replicated data if necessary. To do. This process is performed by the method described in Non-Patent Document 2, for example. Subsequently, the representative node 1A executes a process of promoting the copy data to the original data.

  Further, the representative node 1A transmits a rearrangement start notification indicating that the data rearrangement process is executed to the existing node 1a and the reduced node 1b (step S14). Here, the existing node 1a that has received the relocation start notification executes data relocation processing (relocation of replicated data, promotion processing of original data) (step S15). Further, the reduced node 1b that has received the rearrangement start notification executes data rearrangement processing (relocation of replicated data) (step S15).

Subsequently, when the rearrangement of the replicated data is completed, the existing node 1a and the reduced node 1b transmit a replicated data rearrangement completion notification (described as “duplication completion notification” in FIG. 7) to the representative node 1A ( Step S16). When the representative node 1A receives the replication data relocation completion notification from all the existing nodes 1a and the reduction node 1b, it transmits a replication completion confirmation notification to the existing node 1a and the reduction node 1b (step S17).
Next, the existing node 1a receives the copy completion confirmation notification, and further transmits the data relocation processing completion notification to the representative node 1A when the original data promotion processing is completed (step S18). Further, when receiving the copy completion confirmation notification, the reduction node 1b transmits a data relocation processing completion notification to the representative node 1A (step S18).

  When the representative node 1A receives the data rearrangement completion notification from all the existing nodes 1a and the reduction node 1b, the representative node 1A lowers the reduction flag “+1” in the temporary ID table, and uses the temporary ID table to determine the current ID table. Is updated (step S19). Subsequently, the representative node 1A transmits an ID table update notification to the existing node 1a (step S20). Thereby, the existing node 1a lowers the provisional ID table reduction flag “+1”, and updates the current ID table with the temporary ID table (step S21). Then, the existing node 1a transmits a completion notification indicating that the ID table has been updated to the representative node 1A (step S22). Receiving this completion notification, the representative node 1A transmits to the network management device 5 a cluster member update notification indicating the end of the process associated with the removal of the node 1 (step S23).

  On the other hand, when the representative node 1A updates its own ID table in step S19, the representative node 1A transmits a reduction notification to the reduction node 1b (step S24). Thereby, the reduction node 1b finishes all the processes (step S25), and is disconnected from the cluster.

  As shown in FIG. 7, during the transition period in this conventional maintenance reduction, the representative node 1A, each existing node 1a, and the reduction node 1b set the reduction flag “1” and store the temporary ID table in themselves. After that (the processes of steps S5 and S7), the data rearrangement process is completed and finally the ID table is updated (the processes of steps S19, S21, and S25).

David Karger, et al., “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, [online], 1997, ACM, [April 26, 2016 search], Internet <URL : http: //www.akamai.com/dl/technical_publications/ConsistenHashingandRandomTreesDistributedCachingprotocolsforrelievingHotSpotsontheworldwideweb.pdf> Eriko Iwasa and two others, “Study on autonomous data relocation method in high-availability server cluster”, The Institute of Electronics, Information and Communication Engineers, IEICE Communication Society Conference Proceedings 2012, B-6-71, 2012-08-28

  This maintenance reduction is required to be processed more quickly in consideration of disconnecting a node suspected of failure from the cluster. That is, it is required to shorten the transition period as much as possible. However, according to the above procedure, the maintenance reduction process is not completed unless the rearrangement of all the nodes constituting the cluster and the original promotion are completed. Also, if a node failure occurs during maintenance reduction, the maintenance reduction processing becomes invalid, and network costs and processing delays due to extra data replication and relocation between nodes due to rollback etc. Will occur.

  The present invention has been made in view of such a background, and the present invention provides a maintenance / removal system, a node, and a maintenance / removal method that reduce the processing time required for the maintenance / removal to separate a node from a cluster. Is an issue.

  In order to solve the above-described problem, the invention according to claim 1 is characterized in that a message is distributed to each of a plurality of nodes constituting a cluster and processed, and original data that is an original of the data processed as the message and a copy thereof. A maintenance / removal system provided in a high availability system for storing a certain replicated data in a different node, wherein a part of a plurality of nodes constituting the cluster is removed, A storage unit that stores distribution ID information that indicates the charge of data processed by each node in association with an ID that indicates node identification information, and a reduced node that indicates a node to be removed from the plurality of nodes A reduction node determination unit for determining the reduction node, and the distribution of the message to the reduction node is stopped, and the nodes of the nodes other than the reduction node are Replicated data that is promoted to the original data when the reduced node is removed by using a distribution destination change processing unit that generates temporary distribution ID information indicating the responsibility of the data and the temporary distribution ID information Is extracted as original promotion target data, and the limited synchronization processing unit executes synchronization processing between the extracted original promotion target data and the original data stored in the reduced node, and the limited synchronization After the synchronization processing by the processing unit, the original promotion target data is promoted to the original data, and then the current distribution ID information is updated using the temporary distribution ID information; A maintenance reduction system comprising: a data rearrangement processing unit that performs duplication and rearrangement of the data so as to maintain redundancy determined in the available system It was.

  According to a second aspect of the present invention, a message is distributed to each of a plurality of nodes constituting a cluster and processed, and the original data that is the original data processed as the message is different from the duplicate data that is a duplicate thereof. The node of the maintenance / reduction system, which is provided in the high availability system to be stored in the network, and in which a part of the plurality of nodes constituting the cluster is removed, corresponds to an ID indicating identification information of each node. In addition, a storage unit that stores distribution ID information indicating the charge of data processed by each node, and a reduction node determination unit that determines a reduction node indicating a node to be reduced from among the plurality of nodes, Temporary distribution indicating suspension of distribution of the message to the reduced node and indicating the data assignment of each node excluding the reduced node Using the distribution destination change processing unit that generates D information and the temporary distribution ID information, copy data that is promoted to the original data when the reduced node is removed is extracted as original promotion target data A limited synchronization processing unit that executes synchronization processing between the extracted original promotion target data and the original data stored in the reduced node, and after the synchronization processing by the limited synchronization processing unit, The original promotion target data is promoted to the original data, and thereafter, the provisional distribution ID information is used to update the current distribution ID information, and the redundancy determined in the high availability system is set. And a data rearrangement processing unit that executes the data duplication and rearrangement so as to maintain the node.

  According to a third aspect of the present invention, a message is distributed to each of a plurality of nodes constituting the cluster and processed, and the original data that is the original data processed as the message is different from the duplicate data that is a duplicate thereof. A maintenance / removal method for a maintenance / reduction system in which a part of a plurality of nodes constituting the cluster is removed, provided in a high-availability system stored in a storage system, wherein the node identifies each node In association with an ID indicating information, distribution ID information indicating the charge of data processed by each node is stored in the storage unit, and a reduction node indicating a node to be reduced is selected from the plurality of nodes. A step of deciding, and stopping the distribution of the message to the reduced node, and indicating the responsibility of the data of each node except the reduced node Generating the distribution ID information, and using the temporary distribution ID information, extracting duplicate data to be promoted to the original data when the reduced node is removed as original promotion target data, Executing the synchronization process between the extracted original promotion target data and the original data stored in the reduced node; and after the synchronization process, promoting the original promotion target data to the original data; Thereafter, using the provisional distribution ID information, updating the current distribution ID information, and executing replication and rearrangement of the data so as to maintain redundancy determined in the high availability system And a maintenance reduction method characterized in that

  As described above, in the maintenance reduction system, after the distribution destination change process, the synchronization process (limited synchronization process) is executed between the original promotion target data and the original data. Thereafter, the original is promoted, the ID table is updated, and finally data rearrangement is performed. This limited synchronization process synchronizes the latest state of the original data with the replicated data that is promoted after the disappearance of the original data (data to be promoted to the original data). Even in the case of a system that is not always up-to-date, the latest data can be retained when maintenance is reduced. In the maintenance reduction system, after the limited synchronization process, the original state promotion and the update of the ID table are performed, thereby terminating the transient state associated with the maintenance reduction process. Therefore, since the data rearrangement can be performed separately from the maintenance reduction, the transient state can be made shorter.

  According to the present invention, it is possible to provide a maintenance / reduction system, a node, and a maintenance / reduction method that reduce the processing time required for maintenance / removal for separating a node from a cluster.

It is a figure which shows the whole structure of the distributed processing system containing the maintenance reduction system which concerns on this embodiment. It is a functional block diagram which shows the structural example of the node which concerns on this embodiment. It is a figure which shows the data structural example of the node identifier management information which concerns on this embodiment. It is a figure which shows the example of a data structure of distribution ID information (ID table) which concerns on this embodiment. It is a figure which shows the data structural example of the failure probability information which concerns on this embodiment. It is a sequence diagram which shows the maintenance reduction process which the maintenance reduction system which concerns on this embodiment performs. It is a figure which shows the processing sequence of the maintenance reduction in the conventional distributed processing system (high availability system).

<Overall configuration>
First, a distributed processing system (high availability system) S including a maintenance reduction system 1000 according to a mode for carrying out the present invention (hereinafter referred to as “this embodiment”) will be described.
FIG. 1 is a diagram showing an overall configuration of a distributed processing system S including a maintenance / reduction system 1000 according to the present embodiment.

  The maintenance reduction system 1000 is composed of a plurality of nodes 1. Each node 1 is a physical device such as a computer or a logical device such as a virtual machine. The load balancer 3 distributes the message received from the client 2 by simple round robin or the like and transmits it to each node 1. Then, the distribution unit 13 of the node 1 distributes the message from the client 2 to the node 1 in charge of the message based on, for example, a consistent hash method. In the node 1 in charge of the message, the signal processing unit 14 performs signal processing and provides a service to the client 2.

  Note that the load balancer 3 does not exist, and a message can be transmitted from the client 2 to an arbitrary node 1 (distribution unit 13). Further, the distribution unit 13 and the signal processing unit 14 may exist on the same node 1 at the same time, or may exist on different nodes 1.

  Each node 1 constituting the cluster in the maintenance / reduction system 1000 includes a representative node 1A that manages an ID table (distribution ID information 200 and the like described later) and an existing node 1a that is a node 1 other than the representative node 1A. Is done. Here, the representative node 1A transmits / receives information to / from an external device such as a network management device and controls the entire maintenance / reduction system 1000. The network management apparatus is connected to the maintenance / reduction system 1000 and the load balancer 3 to manage the entire distributed processing system S.

<Overview>
As shown in FIG. 7, the conventional maintenance reduction processing is performed in the order of distribution destination change processing → data rearrangement → original promotion → ID table update. However, in this procedure, in the distribution destination changing process, it is a transitional period until the ID table is finally updated after shifting to the process using the temporary ID table excluding the removed node. That is, the maintenance reduction process is not completed unless the data rearrangement and the original promotion are completed.
On the other hand, in the maintenance / reduction system 1000 according to the present embodiment, a limited synchronization process (details will be described later) specific to the present invention is executed after the distribution destination change process. Thereafter, the original is promoted, the ID table is updated, and finally data rearrangement is performed. This limited synchronization process means a synchronization process for matching the data contents between the original data held by the reduced node and the replicated data newly promoted to the original data after the reduced node is removed. By this limited synchronization processing, the latest state of the original data is synchronized with the replicated data that is promoted after the original data disappears. Then, in the maintenance reduction system 1000, after the limited synchronization processing, the original state promotion and the update of the ID table are performed, thereby terminating the transient state associated with the maintenance reduction processing. That is, since the data rearrangement can be performed separately from the maintenance reduction, the transient state can be made shorter.
Hereinafter, the maintenance / reduction system 1000 including the node 1 according to the present embodiment will be specifically described.

≪Node≫
First, the node 1 which comprises the maintenance reduction system 1000 which concerns on this embodiment is demonstrated concretely. Note that the node 1 according to the present embodiment includes the node identifier management information 100 (see FIG. 3), which will be described later, among the plurality of nodes 1 constituting the maintenance / reduction system 1000, and the ID table (distribution ID information 200) described above. (See FIG. 4)) When the node becomes the representative node 1A that manages the existing node 1a (non-representative node) that receives the node identifier management information 100 and the distribution ID information 200 from the representative node 1A, and updates and stores each information There are cases where The processing performed by the representative node 1A will be described later. Further, in the maintenance / reduction system 1000, among the existing nodes 1a constituting the cluster, the node 1 to be reduced will be described as the reduction node 1b below. In addition, as an example of information used by the representative node 1A of the node 1 according to the present embodiment for determining the reduced node 1b, here, the representative node 1A receives failure probability information 300 (see FIG. 5) from an external device. An example will be described in which the reduced node 1b is determined based on the failure probability of each node 1 received.

FIG. 2 is a functional block diagram illustrating a configuration example of the node 1 according to the present embodiment.
As illustrated in FIG. 2, the node 1 includes a control unit 10, an input / output unit 20, and a storage unit 30.

  The input / output unit 20 inputs / outputs information to / from the load balancer 3, other nodes 1 other than itself, a network management device (not shown), other external devices (not shown), and the like. The input / output unit 20 also includes an input / output interface (input / output interface) that performs input / output between a communication interface (not shown) that transmits and receives information via a communication line and an input unit such as a keyboard and an output unit such as a monitor. (Not shown).

  The storage unit 30 includes storage means such as a hard disk, a flash memory, and a RAM (Random Access Memory). The storage unit 30 includes data 400 to be processed, node identifier management information 100 (see FIG. 3), and distribution ID information 200 (FIG. 4). The failure probability information 300 (FIG. 5) used when determining the reduced node 1b is stored. In this data 400, the original data stored by the node 1 itself and the copy of the original data stored by other nodes 1 other than the node 1 are stored as duplicate data. Details of each piece of information stored in the storage unit 30 will be described later.

  The control unit 10 controls the entire node 1, and includes a node identifier management unit 11, a reduced node determination unit 12, a distribution unit 13, a signal processing unit 14, and a maintenance reduction processing unit 15. In addition, this control part 10 is implement | achieved when CPU (illustration omitted) expand | deploys and executes the program stored in the memory | storage part 30 on RAM (illustration omitted), for example.

The node identifier management unit 11 manages the node information (IP address and the like) of each node 1 configuring the cluster and the ID space handled by each node 1 in the maintenance / reduction system 1000.
Specifically, the node identifier management unit 11 identifies, for example, identification information of the nodes 1 constituting the maintenance / removal system 1000 when the node 1 is removed (removed) in the maintenance / removal system 1000 to which the node identifier management unit 11 belongs. The node identifier management information 100 (FIG. 3) stored therein is updated.

FIG. 3 is a diagram illustrating a data configuration example of the node identifier management information 100 according to the present embodiment.
As shown in FIG. 3, the node identifier management information 100 stores the node identifier 101 and the address 102 (for example, IP address) of each node 1 constituting the maintenance reduction system 1000 in association with each other.

  The node identifier 101 is given by the node identifier management unit 11 of a specific node (for example, the representative node 1A set in ascending order of the node identifier 101) set in advance in the maintenance / removal system 1000, for example. It is distributed to each node 1 in the reduction system 1000.

The node identifier management unit 11 updates the node identifier management information 100 (reflects the removal of the node 1 or the like) based on the node ID change information received from the outside.
Furthermore, when the node identifier management unit 11 is the node identifier management unit 11 of the representative node 1A, the node to be removed (removal node 1b) is determined by the reduction node determination unit 12, and will be described later. When the distribution destination change processing unit 151 of the maintenance reduction processing unit 15 stops the allocation to the reduction node 1b, temporary allocation ID information 200k (temporary ID table) in which the reduction node 1b is deleted is generated. And transmitted to the existing nodes 1a other than the reduced node 1b.
Further, when the node identifier management unit 11 is the node identifier management unit 11 of the existing node 1a, the node identifier management unit 11 receives the temporary distribution ID information 200k (temporary ID table) from the representative node 1A, and stores its own storage unit 30. To remember. Further, when receiving the update notification of the distribution ID information 200 from the representative node 1A, the node identifier management unit 11 updates the current distribution ID information 200 using the temporary distribution ID information 200k (temporary ID table).

  FIG. 4 is a diagram illustrating a data configuration example of the distribution ID information 200 (ID table) according to the present embodiment. 4A shows the distribution ID information 200 before maintenance reduction (current state), and FIG. 4B shows the temporary distribution ID information 200k and distribution ID information after maintenance reduction (after update). 200 g is shown.

  As shown in FIG. 4A, the distribution ID information 200 stores an ID space 202 (area in charge) that the node 1 is responsible for in association with the node identifier 201. This node identifier 201 is the same information as the node identifier 101 of FIG. In the example shown in FIG. 4A, the total number of IDs in the ID space is 1000 from “0” to “999”. For example, the node 1 whose node identifier 201 is “A” is “ID space 202 in charge” 0 to 199 ". In the distribution ID information 200, the node ID on the ID space of the node 1 (node “A”) having the node identifier 201 “A” is “199”. Similarly, the node ID on the ID space of the node “B” is “399”. The node ID of the node “C” on the ID space is “599”. The node ID on the ID space of the node “D” is “799”. The node ID on the ID space of the node “E” is “999”. Then, the node identifier management unit 11 sorts the node IDs of the respective nodes 1 in ascending order in the distribution ID information 200 and manages them as a continuous ID space 202.

  FIG. 4B shows temporary allocation ID information 200k (generated by the node identifier management unit 11 when the node “D” is selected as the node 1 to be removed in the ID space of FIG. 200). The temporary distribution ID information 200k (temporary ID table) becomes the distribution ID information 200g (200) after the reduction of the reduction node 1b as a result of the maintenance reduction process.

  In FIG. 4B, the node “D” is deleted as the reduced node 1b, and the area that the node “D” was in charge of is divided into two equal parts, the node “C” and the node “E”. A shared example is shown. In FIG. 4B, the area in charge of the node “C” is changed to “400” to “699” inheriting half of the area in charge of the deleted node “D”. In addition, the area in charge of the node “E” is changed to “700” to “999”, inheriting half of the area in charge of the deleted node “D”. However, it is assumed that the logic on how to share the ID space area handled by the reduced node 1b in the existing node 1a is set in advance by the network administrator. When the reduced node determination unit 12 determines the reduced node 1b, the node identifier management unit 11 obtains temporary allocation ID information 200k (temporary ID table) based on the preset logic (predetermined logic). Generate.

  The node identifier management unit 11 of the representative node 1A in the maintenance reduction system 1000 transmits temporary allocation ID information 200k (temporary ID table) to each node 1. In addition, the node identifier management unit 11 of the representative node 1A updates the temporary distribution ID information 200k to the distribution ID information 200g (see FIG. 4B) after the maintenance reduction (after update). The node identifier management information 100 (see FIG. 3) from which the reduced node 1b is deleted is transmitted to each existing node 1a. Thus, the same node identifier management information 100 and the same distribution ID information 200 are held in each node 1 in the maintenance / reduction system 1000.

  For example, the representative node 1A is set to be the representative node 1A in order from the node 1 in the top row of the node identifier management information 100 (FIG. 3). When the node 1 newly becomes the representative node 1A, information indicating that it is the representative node 1A is transmitted to each existing node 1a and the like.

Returning to FIG. 2, the reduction node determination unit 12 determines a node (reduction node 1 b) to be reduced among the nodes 1 constituting the maintenance reduction system 1000. The determination of the reduced node 1b may be performed based on, for example, the designation information of the reduced node 1b obtained from a network management apparatus (not shown) for periodic inspection. Further, the representative node 1A may monitor the state of each existing node 1a and, for example, receive a warning notification that is greater than a predetermined value, thereby determining the node 1 that is suspected of being the failure node 1b. .
In the present embodiment, the failure probability information of (near) future due to a disaster such as an earthquake is acquired from an external device, and the reduced node determination unit 12 has a node 1 with a predetermined failure probability or higher (a failure occurs due to a disaster). A description will be given as an example of determining a node having a high probability as a reduction target node (reduction node 1b).

  This failure probability information is described, for example, in Non-Patent Document 3 (Hiroshi Saito, et al., “Proposal of Disaster Avoidance Control,” Proc. Of Telecommunications Network Strategy and Planning Symposium (Networks), 2014 16th International.). It is realized by the technology. In the technique described in Non-Patent Document 3, a failure probability is calculated based on the occurrence position of a disaster (such as an earthquake) and the positional relationship between each node 1 on the network. The reduced node determination unit 12 of the representative node 1A according to the present embodiment acquires failure probability information of each node 1 constituting the maintenance / removal system 1000 from an external device via, for example, a network management device (not shown). To do.

FIG. 5 is a diagram illustrating a data configuration example of the failure probability information 300 according to the present embodiment.
The failure probability information 300 is information indicating the probability of failure of each node 1 (server) at each site due to a large-scale disaster or the like that is predicted to occur in the future (probability of failure). Large-scale disasters include, for example, typhoons, storms, tornadoes, lightning strikes, heavy rains, river floods, tsunamis, earthquakes, etc., and in the areas where the bases are located several days or seconds to hours later, Information that indicates the predicted probability that the server will become unusable due to physical damage (including network disconnection, etc.), power outages, or failures such as the server administrator not getting close to the server installation facility It is.

The failure probability information 300 includes data items of a node identifier 301, a time 302, and a failure probability 303.
The node identifier 301 represents the identifier of each node 1 constituting the cluster, and is the same information as the node identifiers 101 and 201 in FIGS. 3 and 4.
The time 302 and the failure probability 303 are information stored in association with the node identifier 301. The failure probability 303 stores the failure probability (%) of the server at time 302 (predetermined time). FIG. 5 shows an example in which the time 302 is set every hour.
For example, as shown in the first line of the failure probability information 300, the node “A” of the node identifier 301 has a failure probability 303 of “1 June 2015” indicated at time 302 between 13:00 and 14:00. 10 (%) ”, the failure probability 303 between 14:00 and 15:00 is“ 30 (%) ”, and the failure probability 303 between 15:00 and 16:00 is“ 40 (%) ”.

  The reduced node determination unit 12 refers to the failure probability information 300 and selects a node 1 (node with a high probability of occurrence of a failure due to a disaster) that is equal to or higher than a predetermined failure probability as a reduction target node (removed node 1b). Determine as. For example, the reduced node determination unit 12 determines the node “D” whose failure probability 303 exceeds a predetermined failure probability (for example, 50% or more) as the reduced node 1b.

  Returning to FIG. 2, the distribution unit 13 obtains “hash (key)” based on the information (“distribution key”) in the message (signal) received from the client 2 via the load balancer 3 (FIG. 1) or the like. Calculate and refer to the distribution ID information 200 (FIG. 4) to extract the node 1 in charge of processing the message. Then, the distribution unit 13 acquires the extracted address information of the node 1 with reference to the node identifier management information 100 (FIG. 3), and distributes (transmits) the message to the extracted node 1.

The signal processing unit 14 performs signal processing of messages related to data handled by its own node 1. The processing executed by the signal processing unit 14 in response to this message is, for example, data registration, update, search, or deletion. In addition, when the signal processing unit 14 receives a message such as data registration or update, the signal processing unit 14 refers to the distribution ID information 200, and in the clockwise direction from the own node 1 in the ID space according to the redundancy. As a node, a node (duplicate node) that replicates data is determined (when the redundancy is “3”, two replica nodes are determined). Then, the signal processing unit 14 transmits the duplicate data obtained by duplicating the original data to the decided duplicate node, and stores the duplicate data.
The signal processing unit 14 embeds, as a distribution key, a hash value calculated based on “Call-id” in SIP (Session Initiation Protocol), for example, in a message sent after signal processing (for example, in To, for example, To (It is described in Tag of / From header.) As a result, when the distribution unit 13 receives a subsequent call of the message, the distribution ID information 200 (FIG. 4) is referred to using the hash value embedded as the distribution key, and the node 1 in charge of the subsequent call Can be specified.

The maintenance reduction processing unit 15 executes the maintenance reduction processing when the reduction node determination unit 12 determines a node to be removed from the cluster (reduction node 1b).
The maintenance reduction processing unit 15 includes a distribution destination change processing unit 151, a limited synchronization processing unit 152, an original promotion processing unit 153, and a data rearrangement processing unit 154.

The distribution destination change processing unit 151 prevents the signal from being distributed to the node 1 (removal node 1b) to be removed when the removal node determination unit 12 determines the reduction node 1b. The distribution destination change request is transmitted to the network management apparatus (not shown). Thereby, the network management device changes the setting of the load balancer 3 and the like, and changes the setting so that the message (signal) is not distributed to the reduced node 1b.
Also, the distribution destination change processing unit 151 notifies the node identifier management unit 11 that the signal distribution to the reduced node 1b has been stopped. This causes the node identifier management unit 11 to generate temporary allocation ID information 200k (temporary ID table) that is assumed to be reduced by the reduced node 1b based on a predetermined logic.
Then, the distribution destination change processing unit 151 notifies the generated temporary distribution ID information 200k (temporary ID table) to all of the existing nodes 1a and the reduced nodes 1b.

When the limited synchronization processing unit 152 is the limited synchronization processing unit 152 of the existing node 1a, the limited synchronization processing unit 152 refers to the temporary distribution ID information 200k (temporary ID table), and promotes the original copy of the copy data held by itself. The data to be replicated (hereinafter referred to as “original promotion target data”) is extracted. Then, the limited synchronization processing unit 152 executes synchronization processing between the original data held by the reduced node 1b and the original promotion target data.
Specifically, a synchronization request is transmitted from the limited synchronization processing unit 152 of the existing node 1a having the original data to be promoted to the limited synchronization processing unit 152 of the reduced node 1b. Then, the limited synchronization processing unit 152 of the reduced node 1b refers to the current distribution ID information 200, extracts the data (original data) that it owns as an original, and requests synchronization using the information of the original data as synchronization information. Is transmitted to the existing node 1a that has transmitted. At this time, the limited synchronization processing unit 152 of the reduced node 1b may transmit only the difference information between the original data and the duplicated data.

When the original promotion processing unit 153 is the original promotion processing unit 153 of the existing node 1a having the copy data (original promotion target data) that is the target of the original promotion, the original promotion processing data by the limited synchronization processing unit 152 When the synchronization processing between the original data and the original data is completed, the duplicated data (original promotion target data) is promoted to the original data.
Then, the original data promotion processing unit 153 transmits an original data promotion completion notification indicating that the copy data has been promoted to the original data to the representative node 1A.

Receiving the original promotion completion notification, the original promotion processing unit 153 of the representative node 1A updates the current distribution ID information 200 to the node identifier management unit 11 using the temporary distribution ID information 200k (temporary ID table). To execute the process to be executed.
Then, the original promotion processing unit 153 of the representative node 1A uses the temporary distribution ID information 200k (temporary ID table) for the existing node 1a when the own distribution ID information 200 is updated. ID table update notification, which is instruction information for updating the distribution ID information 200, is transmitted to the existing node 1a, and the current distribution ID information 200 is updated.
The original promotion processing unit 153 of the representative node 1A transmits an original promotion completion notification indicating that the original promotion process has been completed to the network management apparatus.

  When the data rearrangement processing unit 154 is the data rearrangement processing unit 154 of the representative node 1A, the data rearrangement processing unit 154 sends an update to the existing node 1a in response to the completion of the update processing of the distribution ID information 200 by the original promotion processing unit 153 In response, a data relocation start notification is transmitted. Thereby, in each existing node 1a, data requiring re-redundancy is extracted, and data rearrangement is executed by performing data migration or duplication based on the updated distribution ID information 200.

<Process flow>
Next, the flow of maintenance / reduction processing executed by the maintenance / reduction system 1000 according to the present embodiment will be described.
FIG. 6 is a sequence diagram showing maintenance / reduction processing executed by the maintenance / reduction system 1000 according to the present embodiment.

  First, the reduced node determination unit 12 of the representative node 1A determines a node (removal node 1b) to be reduced (step S101). Here, the reduced node determination unit 12 refers to the failure probability of each node 1 by receiving failure probability information 300 (see FIG. 5) from an external device, for example, and selects a node 1 having a predetermined failure probability or higher. The node to be removed is determined (removal node 1b).

  Subsequently, the distribution destination change processing unit 151 of the representative node 1A transmits a distribution destination change request to the network management device 5 so that a message (signal) is not distributed to the determined reduction node 1b (Step S1). S102). Then, the network management device 5 includes, for example, a distribution destination of the load balancer 3 serving as a distribution source so that signals are not distributed to the reduction node 1b among the nodes 1 constituting the maintenance reduction system 1000. The setting is changed (step S103). Subsequently, the network management device 5 transmits a distribution destination change completion notification to the representative node 1A (step S104).

  Next, the distribution destination change processing unit 151 of the representative node 1A notifies the node identifier management unit 11 that the signal distribution to the reduced node 1b has been stopped, and the reduced node 1b has been reduced. Temporary allocation ID information 200k (temporary ID table) to be generated is generated (step S105). In this temporary distribution ID information 200k, a reduction flag “+1” indicating that the deletion node 1b is deleted from the current distribution ID information 200 is set.

  Subsequently, the distribution destination change processing unit 151 of the representative node 1A notifies the generated temporary distribution ID information 200k (temporary ID table) to the existing node 1a (step S106). Then, the existing node 1a acquires the temporary distribution ID information 200k (temporary ID table) and stores it in the storage unit 30 (step S107).

Next, limited synchronization processing is executed between the existing node 1a and the reduced node 1b.
Specifically, the limited synchronization processing unit 152 of the existing node 1a refers to the temporary distribution ID information 200k (temporary ID table), and among the copy data held by itself, the copy data (original data) that is the target of the original data promotion. Data to be promoted) is extracted (step S108).
Subsequently, a synchronization request is transmitted from the limited synchronization processing unit 152 of the existing node 1a having the original data to be promoted to the limited synchronization processing unit 152 of the reduced node 1b (step S109). Then, the limited synchronization processing unit 152 of the reduced node 1b refers to the current distribution ID information 200, extracts the data (original data) that it owns as an original, and requests synchronization using the information of the original data as synchronization information. Is transmitted to the existing node 1a that has transmitted (step S110). As a result, the synchronization process is executed between the original data of the reduced node 1b and the original promotion target data.

When the synchronization processing between the original data and the original promotion target data by the limited synchronization processing unit 152 is finished, the original promotion processing unit 153 of the existing node 1a having the original promotion target data receives the duplicate data that is the original promotion target data. Then, an original promotion process for promoting the original data is executed (step S111).
Subsequently, the original promotion processing unit 153 of the existing node 1a that has executed the original promotion process transmits an original promotion completion notification indicating that the original promotion of replicated data has been completed to the representative node 1A (step S112).

Next, when receiving the original promotion completion notification, the original promotion processing unit 153 of the representative node 1A lowers the reduction flag “+1” of the temporary distribution ID information 200k (temporary ID table) to the node identifier management unit 11. At the same time, the temporary distribution ID information 200k is used to instruct execution of a process for updating the current distribution ID information 200 (step S113).
Then, the original promotion processing unit 153 of the representative node 1A uses the temporary distribution ID information 200k (temporary ID table) for the existing node 1a when the own distribution ID information 200 is updated. ID table update notification, which is instruction information for updating the distribution ID information 200, is transmitted to the existing node 1a (step S114). Thereby, in the existing node 1a, the reduction flag “+1” of the temporary distribution ID information 200k is lowered, and the distribution ID information 200 is updated (step S115).
Also, the original promotion processing unit 153 of the representative node 1A transmits an original promotion completion notification indicating that the original promotion process has been completed to the network management device (step S116).
Since the original promotion and the update of the distribution ID information 200 are completed when the original promotion completion notification is transmitted to the network management apparatus, each node 1 can end the transient state. Therefore, the network administrator can execute the disconnection of the reduced node 1b from the cluster at this point.

  Subsequently, the data relocation processing unit 154 of the representative node 1A transmits a data relocation start notification to the existing node 1a (step S117). In addition, the maintenance / reduction processing unit 15 of the representative node 1A transmits a maintenance / reduction processing completion notification to the network management device 5 (step S118).

In the existing node 1a that has received the data relocation start notification from the representative node 1A, the data relocation processing unit 154 extracts data that requires re-redundancy based on the updated distribution ID information 200, and migrates the data. Then, data rearrangement is executed by performing duplication (step S119). Then, when the data rearrangement is completed, the data rearrangement processing unit 154 of the existing node 1a transmits a completion notification to that effect to the representative node 1A (step S120).
The maintenance reduction processing unit 15 of the representative node 1A transmits a cluster member update completion notification indicating that all cluster member updates have been completed to the network management device 5 (step S121), and ends the process.

  As described above, according to the maintenance / reduction system 1000, the node 1, and the maintenance / reduction method according to the present embodiment, after the distribution destination change process, the limited synchronization process peculiar to the present invention is executed, and then the original Promotion is performed, the ID table is updated, and finally data rearrangement is performed (see FIG. 6). By doing in this way, the period of a transient state can be shortened compared with the procedure of the prior art shown in FIG. By shortening the period of the transient state, the removal node 1b can be disconnected from the cluster more quickly. Even if the process is interrupted for some reason at the stage of data relocation of the replicated data performed after the original promotion and update of the ID table, if the original is promoted and rewinded immediately after the ID table is updated Therefore, MTTR (average repair time) can be shortened.

1 node 1A representative node 1a existing node 1b reduced node 2 client 3 load balancer 10 control unit 11 node identifier management unit 12 reduced node determination unit 13 distribution unit 14 signal processing unit 15 maintenance reduction processing unit 20 input / output unit 30 storage Section 100 Node identifier management information 151 Distribution destination change processing section 152 Limited synchronization processing section 153 Original promotion processing section 154 Data relocation processing section 200 Distribution ID information 300 Failure probability information 400 Data 1000 Maintenance reduction system S Distributed processing system (High Available system)

Claims (3)

Provided in a high availability system that distributes and processes messages to each of a plurality of nodes constituting a cluster, and stores the original data that is the original data processed as the message and the replicated data that is a duplicate thereof in different nodes. , A maintenance / reduction system for removing some of the plurality of nodes constituting the cluster,
The node is
A storage unit for storing distribution ID information indicating the charge of data processed by each node in association with an ID indicating identification information of each node;
A reduction node determination unit for determining a reduction node indicating a node to be reduced from among the plurality of nodes;
A distribution destination change processing unit that stops the distribution of the message to the reduced node and generates temporary distribution ID information indicating the data assignment of each node excluding the reduced node;
Using the temporary distribution ID information, duplicate data that is promoted to the original data when the reduced node is removed is extracted as original promotion target data, the extracted original promotion target data, and the reduction A limited synchronization processing unit that performs synchronization processing with the original data stored in the installation node;
After the synchronization processing by the limited synchronization processing unit, the original promotion processing unit that promotes the original promotion target data to the original data and then updates the current distribution ID information using the temporary distribution ID information When,
A data relocation processing unit for performing replication and relocation of the data so as to maintain the redundancy determined in the high availability system;
A maintenance reduction system characterized by comprising: Provided in a high availability system that distributes and processes messages to each of a plurality of nodes constituting a cluster, and stores the original data that is the original data processed as the message and the replicated data that is a duplicate thereof in different nodes. The node of the maintenance / removal system for removing a part of the plurality of nodes constituting the cluster,
A storage unit for storing distribution ID information indicating the charge of data processed by each node in association with an ID indicating identification information of each node;
A reduction node determination unit for determining a reduction node indicating a node to be reduced from among the plurality of nodes;
A distribution destination change processing unit that stops the distribution of the message to the reduced node and generates temporary distribution ID information indicating the data assignment of each node excluding the reduced node;
Using the temporary distribution ID information, duplicate data that is promoted to the original data when the reduced node is removed is extracted as original promotion target data, the extracted original promotion target data, and the reduction A limited synchronization processing unit that performs synchronization processing with the original data stored in the installation node;
After the synchronization processing by the limited synchronization processing unit, the original promotion processing unit that promotes the original promotion target data to the original data and then updates the current distribution ID information using the temporary distribution ID information When,
A data relocation processing unit for performing replication and relocation of the data so as to maintain the redundancy determined in the high availability system;
A node characterized by comprising: Provided in a high availability system that distributes and processes messages to each of a plurality of nodes constituting a cluster, and stores the original data that is the original data processed as the message and the replicated data that is a duplicate thereof in different nodes. A maintenance / removal method for a maintenance / removal system that removes some of the plurality of nodes constituting the cluster,
The node is
In association with the ID indicating the identification information of each node, the distribution ID information indicating the charge of data processed by each node is stored in the storage unit,
Determining a reduction node indicating a node to be reduced from among the plurality of nodes;
Stopping the distribution of the message to the reduced node, and generating temporary allocation ID information indicating the data assignment of each node excluding the reduced node;
Using the temporary distribution ID information, duplicate data that is promoted to the original data when the reduced node is removed is extracted as original promotion target data, the extracted original promotion target data, and the reduction Executing a synchronization process with the original data stored in the installation node;
After the synchronization processing, the original promotion target data is promoted to the original data, and then the current distribution ID information is updated using the temporary distribution ID information.
Performing replication and relocation of the data so as to maintain the redundancy defined in the high availability system;
The maintenance reduction method characterized by performing.

Images