JP2010198442A - Distributed system with failover function and failover method in the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To ensure real-time property by detecting a computer in deadlock state and failing over to a standby computer. <P>SOLUTION: In each of n active computers 100-i among m computers 100-i connected via a network, a consistency part 262 of an ordered multicast part 2 performs ordered-multicasting of input data by establishing consistency of the input data among (n-t) or more computers. A system configuration change detection part 63 detects a computer in deadlock state at a fixed level or higher in comparison with the computer based on the operation condition of the ordered-multicasting by the consistency part 262, and establishes consistency on the detection among (n-t) or more computers including the deadlock computer. When consistency is established on the detection of the deadlock computer, a system configuration setting part 64 suspends the deadlock computer and activates the standby computer. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention synchronously operates n computers (m is an integer satisfying 4 <n <m) among m computers (m is an integer greater than 5) connected by a network, and (n−t ) Related to a distributed system that guarantees multiplexing over t (t is an integer satisfying 3t <n), and particularly when any of the n computers is stagnant, the stagnating computer The present invention relates to a distributed system having a failover function for failing over to one of the remaining (mn) computers and a failover method in the system.

  In recent years, computer technology and network technology have been remarkably improved, and along with this, computerization of business has been widely performed. In addition, depending on the contents of the business, there are many things that cannot be interrupted due to a failure or the like, and recently, it is becoming common to construct a distributed system in which a plurality of computers are connected via a network. One of the distributed system operation techniques is multiplexing of execution of a deterministic program using ordered multicast.

  The ordered multicast is a mechanism for delivering the input to the distributed system to all computers, and guarantees that the arrival order of data is the same on all computers.

  As a multiplexing method in a distributed system, for example, Patent Document 1 is based on the principle of split brain in a distributed system in which n computers (n is an integer of 4 or more) (that is, 4 or more computers) are connected by a network. And a method that does not cause interruption of processing when a failure occurs due to a timeout.

  In the conventional multiplexing method (prior art) disclosed in Patent Document 1, when a distributed system composed of n computers (n is an integer of 4 or more) is taken as an example, the n computers are synchronized. In order to guarantee multiplexing in (n−f) computers (where f is the maximum integer satisfying 3f <n), of the n computers, Each of the n computers includes input candidate collection means and input candidate selection control means (first input candidate selection control means). The input candidate collecting means collects input data selected as candidates to be processed next by each of the n computers via the network. When there are (n−f) or more input data collected by the input candidate collecting unit, the input candidate selection control unit determines whether or not there are (n−f) or more input data having the same contents. When there are (n−f) or more, the input data is determined as a target to be processed next. As a result, the input data is arranged and multicast. In this way, the input candidate selection control means determines that there are (n−f) or more input data having the same content among the (n−f) or more collected input data. f) It is nothing but an agreement on input data with more than one computer. That is, the input candidate selection control means functions as an agreement means.

  For example, in Patent Document 2, t is used as a variable corresponding to the maximum allowable failure number f. In this case, the agreement of input data is obtained by (n−t) or more computers, and t failures (so-called Byzantine failure) among the n computers are allowed. Such an algorithm is called a t-fault tolerant Byzantine consensus algorithm, a t-fault tolerant Byzantine general algorithm, or simply a t-fault tolerant algorithm. In the following description, the above-mentioned algorithm is referred to as a t-fault tolerance algorithm, and “t” is used as the maximum allowable failure number in order to prevent complexity even with respect to the prior art disclosed in Patent Document 1. Further, in the following description, the above-mentioned n computers may be referred to as computers constituting the multiplexing or computers to be operated synchronously.

  In the t-fault tolerance algorithm, t may be an integer larger than 0 that satisfies 3t <n, and does not necessarily need to be the maximum integer that satisfies 3t <n as described in Patent Document 1.

Japanese Patent No. 3655263 JP-A-6-83611

  According to the prior art described in Patent Document 1, in a distributed system composed of n computers (n is an integer of 4 or more), that is, in a distributed system with n computers constituting multiplexing, up to t computers Computer failure is acceptable.

  Here, it is assumed that there are computers that are constantly delayed (that is, whose operations are stagnant) among the n computers. In this case, if the number of delayed computers is t, the remaining (nt) computers can be agreed, so that real-time performance can be ensured.

  Next, it is assumed that t computers have failed and there are (nt) computers operating. In such a state, if there is a delayed computer among the (nt) computers in operation, it is necessary to obtain an agreement including the delayed computer. In such a case, the processing speed in the distributed system becomes the processing speed of the computer that is delayed.

  That is, in the prior art, a failure of up to t computers is allowed, but if t computers fail and there are (nt) computers that are delayed, the distributed system The overall processing speed is lowered.

  The present invention has been made in consideration of the above circumstances, and its purpose is to detect a computer that is stagnant among computers constituting a multiplex and to fail over to a computer in a standby state, thereby realizing real-time performance. It is an object to provide a distributed system having a failover function capable of ensuring the above and a failover method in the system.

  According to one aspect of the present invention, n (n is an integer satisfying 4 ≦ n <m) computers among m computers (m is an integer of 5 or more) connected via a network are set to an operating state. Thus, a distributed system having a failover function that operates synchronously by application of the t-fault tolerance algorithm is provided. Each of the m computers constituting the distributed system stores system configuration information indicating whether the m computers are in an operating state to be operated synchronously or in a standby state. The system configuration storage means and (nt) computers (n is 3t <n) among the n computers indicated to be in the operating state by the system configuration information stored in the system configuration storage means. In order to guarantee multiplexing at an integer greater than or equal to 0), an agreement means for arranging and multicasting the input data by agreeing the input data with the (nt) or more computers, and the n units Computers that are stagnant more than a predetermined level compared to itself, are arranged multicast by the agreement means When detecting based on the execution status and detecting the stagnant computer, the detection of the stagnant computer is performed by (nt) or more computers including itself among the n computers. According to the agreement, the detection means for confirming the detection of the stagnant computer and the confirmed stagnant computer are put into a standby state in accordance with the confirmation of the detection of the stagnant computer, and the standby System configuration for failing over from the stagnated computer to the computer in the standby state by updating the system configuration information information stored in the system configuration storage means so that the computer in the state is in the operating state Setting means.

  According to the present invention, when a stagnant computer is detected, an agreement is reached between (nt) or more computers including the computer that has detected the stagnant computer regarding the detection of the stagnant computer. Thus, it can be confirmed that the stagnant computer has been correctly detected. Further, according to the present invention, in response to this confirmation, by making a system change in which the confirmed stagnated computer is set in a standby state and the computer in a standby state is in an operating state, that is, from a stagnant computer By failing over to a computer in a standby state, it is possible to ensure real-time performance.

1 is a block diagram showing a configuration of a distributed system according to an embodiment of the present invention. 2 is an exemplary block diagram showing a functional configuration of a computer constituting the distributed system of the embodiment. FIG. The figure which shows the example of a data structure of the input packet data piled up in the input reception queue part of the embodiment. The figure which shows the layout of the alignment multicast protocol data transmitted / received between the alignment multicast protocol data transmission / reception parts of the computer which comprises the distributed system of the embodiment. The figure which shows the example of a data structure of the system configuration | structure storage part in the embodiment. The figure which shows the example of a data structure of the largest definite input sequence number history storage part in the embodiment. The figure which shows the layout of the system configuration change agreement protocol data transmitted / received between the system configuration change agreement protocol data transmission / reception parts of the computer which comprises the distributed system of the embodiment. The 1st flowchart which shows the operation | movement procedure of the basic part which performs one delivery of the ordered multicast which each computer performs in the same embodiment. The 2nd flowchart which shows the operation | movement procedure of the basic part which performs one delivery of the ordered multicast which each computer performs in the same embodiment. The 1st flowchart which shows the operation | movement procedure for eliminating the delay of multiplexing execution which each computer performs in the same embodiment. The 2nd flowchart which shows the operation | movement procedure for eliminating the delay of multiplexing execution which each computer performs in the same embodiment. The 3rd flowchart which shows the operation | movement procedure for eliminating the delay of multiplexing execution which each computer performs in the same embodiment. FIG. 10 is a fourth flowchart showing an operation procedure for eliminating a delay in multiplexing execution, which is executed by each computer in the embodiment. The flowchart which shows the procedure of the process for searching for the computer which has stagnated which each computer performs in the embodiment. 6 is an exemplary flowchart illustrating a processing procedure executed when each computer receives report-type system configuration change agreement protocol data in the embodiment. 6 is an exemplary flowchart illustrating a procedure of processing performed when each computer receives agreement type or non-agreement type system configuration change agreement protocol data executed by each computer in the embodiment. 6 is an exemplary flowchart illustrating a processing procedure executed by each computer in the embodiment when a confirmed system configuration change request packet is received. 6 is an exemplary flowchart illustrating a processing procedure executed by each computer in the embodiment when system configuration notification protocol data is received.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a distributed system according to an embodiment of the present invention. In FIG. 1, the distributed system 10 includes, for example, five computers 100-1 (# 1) to 100-5 (# 5). Of the computers 100-1 to 100-5, it is assumed that more than the number of computers necessary for guaranteeing multiplexing by the t-fault tolerance algorithm, for example, four computers 100-1 to 100-4 are in operation. . The computers 100-1 to 100-4 in this operating state constitute multiplexing. Of the computers 100-1 to 100-5, the remaining computer 100-5 is in a standby state.

  Here, the number of computers constituting the distributed system 10 is represented by m, and the number of computers currently multiplexed in the distributed system 10 (in an operating state) is represented by n. n is an integer of 4 or more as in Patent Document 1, and m is an integer larger than n. That is, n is an integer that satisfies 4 ≦ n <m. In this case, assuming that t is an integer greater than 0 that satisfies 3t <n, the t-fault tolerance algorithm allows a maximum of t failures of n computers. In the example of the distributed system of FIG. 1, n is 4 and m is 5. When n is 4, t is 1, and one computer is allowed to fail. When t is 1, the minimum value of n necessary to tolerate the failure of t computers, that is, the minimum number of computers required to guarantee multiplexing by the t-fault tolerance algorithm is 4. The example of the distributed system in FIG. 1 corresponds to this case.

  The computers 100-1 to 100-5 are connected to the client device 200 via the network A. The computers 100-1 to 100-5 are also connected to a client device (not shown) other than the client device 200 via the network A. In the present embodiment, the network A is a public network (external network). The computers 100-1 to 100-5 are connected via a network B. In the present embodiment, the network B is a private network (internal network).

  The computers 100-1 to 100-4 in operation, that is, the computers 100-1 to 100-4 constituting multiplexing in the distributed system 10, are the same as the computers in the distributed system described in Patent Document 1. The input packet (input) received from the client device 200 via the network A is processed in the same order as other computers. An input packet from the client device 200 is input to any one of the computers 100-1 to 100-4.

  Each of the computers 100-1 to 100-5 has the same application program 3 (see FIG. 2). Among the computers 100-1 to 100-5, the computers 100-1 to 100-4 constituting the multiplexing start from the same initial state. Thereafter, data input from the client device 200 to the distributed system is always delivered to the computers 100-1 to 100-4 in the same order through ordered multicast. Thereby, each application program 3 is executed in the computers 100-1 to 100-4.

  The input data string to the application program 3 that each of the computers 100-1 to 100-4 has has the same order by the ordered multicast. For this reason, the states of the computers 100-1 to 100-4 are kept the same and the output data strings are all the same due to the characteristics of the deterministic program as described in Patent Document 1. That is, program execution is multiplexed.

  FIG. 2 is a block diagram showing a configuration of the computer 100-i (i = 1, 2,... -5) shown in FIG. In FIG. 2, an input packet transmitted from the client apparatus 200 to the computer 100-i via the network A and received by the input reception queue unit (hereinafter referred to as reception queue) 1 of the computer 100-i is Stacked in the reception queue 1 in the order of reception. The input packet loaded in the reception queue 1 is delivered to the application program 3 or the system configuration management unit 6 described later by the ordered multicast unit 2 (the agreement unit 262 included in the input packet determination unit 26). In addition to the input packet from the client device 200, the input packet loaded in the reception queue 1 includes a specific input packet requesting a system configuration change sent from the system configuration change detection unit 63 of the system configuration management unit 6 ( System configuration change request packet).

  The application program 3 receives the delivered input packet, processes the input packet according to the state stored in the program state management unit 4, and generates an output packet. The generated output packet is selected by the output filter unit 5 and then returned to the client device 200 via the network A (output).

  Next, the configuration of the ordered multicast unit 2 of the computer 100-i will be described. Similar to the ordered multicast unit described in Patent Document 1, the ordered multicast unit 2 includes an input sequence number storage unit 21, an input packet journal storage unit 22, an ordered multicast protocol data transmission / reception unit 23, a step number storage unit 24, a candidate This includes well-known configurations of a packet storage unit 25, an input packet determination determination unit 26, a maximum determination input sequence number storage unit 27, a delay storage unit 28, and a skip determination unit 29.

  The input sequence number storage unit 21 stores the sequence number of the next input packet to be delivered to the computer 100-i by the ordered multicast (that is, the latest sequence number assigned serially to the ordered multicast). The input packet journal storage unit 22 stores a certain amount of the input packet sequence that is confirmed to be delivered to the computer 100-i by the ordered multicast from the latest one.

  The ordered multicast protocol data transmission / reception unit 23 refers to a system configuration storage unit 61 (to be described later) of the system configuration management unit 6, and transmits protocol data via the ordered multicast protocol data transmission / reception unit 23 of the computer in another operating state and the network B. Exchange (aligned multicast protocol data).

  In this embodiment, the network to be used is switched between data exchange between the client apparatus 200 and the computer 100-i and data exchange between the computers 100-i. This reduces the network load. However, the configuration may be such that data exchange between the client device 200 and the computer 100-i and data exchange between the computers 100-i are performed via the network A, for example. Further, the network A is not necessarily a public network.

  The step number storage unit 24, the candidate packet storage unit 25, and the input packet decision determination unit 26 are used in an algorithm that determines the next input packet to be delivered to the computer 100-i by ordered multicast and changes the system configuration.

  The step number storage unit 24 stores a step number indicating a protocol step. The candidate packet storage unit 25 stores a total of n input packets that are “input candidates” for each computer in that step.

  The input packet determination determination unit 26 determines the determination of the input packet from the information in the candidate packet storage unit 25 and determines the “input candidate” in the next step. Unlike the input packet determination determination unit described in Patent Document 1, the input packet determination determination unit 26 passes the input packet to either the application program 3 or the system configuration management unit 6 (the system configuration setting unit 64). To decide. For this determination, information indicating the processing type (processing type information) is added to the input packet loaded in the reception queue 1. The input packet confirmation determination unit 26 includes an input candidate collection unit 261 and an agreement unit 262.

  FIG. 3 shows an example of the data structure of data (input packet data) accumulated in the reception queue 1. As shown in FIG. 3, the input packet data includes fields of processing type and input packet. An input packet is stored (set) in the input packet field, and process type information is stored (set) in the process type field.

  In the present embodiment, the processing type information indicates whether the input packet stored in the input packet field is to be passed to the application program 3 or the system configuration management unit 6 (internal system configuration setting unit 64). The processing type for the (consensus unit 262) to determine is shown. Therefore, the process type indicated by the process type information is divided into (1) application and (2) configuration. When the processing type is “application”, it also indicates that the input packet is input from the external client device 2000. When the processing type is “configuration”, the input packet constitutes one of the distributed system 10 It also indicates that the information is input from the computer system configuration management unit 6 (internal system configuration change detection unit 63) to the reception queue 1 of any one of the computers.

  Referring to FIG. 2 again, the maximum confirmed input sequence number storage unit 27 includes the largest input sequence number that is known to have been confirmed to be delivered, including other computers that are configured in multiplexing (in an operating state). Is stored. The delay storage unit 28 has (n−1) delay flags (n = 4) indicating whether or not the delay storage is delayed from the other (n−1) computers (n = 4) constituting the multiplexing. In this embodiment, three flags) are stored. The skip determination unit 29 determines the necessity of the skip operation from the information in the delay storage unit 28 and executes the skip operation.

  In the following description, the input sequence number stored in the input sequence number storage unit 21 is referred to as a corresponding input sequence number, and the step number stored in the step number storage unit 24 is referred to as a corresponding step number. Of the n “input candidates” stored in the candidate packet storage unit 25 included in the ordered multicast unit 2 of the computer 100-i, the “input candidates” corresponding to the computer 100-i itself (own computer) Is called a self-candidate, and “input candidates” other than the self-candidate are called other candidates.

Next, the ordered multicast protocol data transmitted / received by the ordered multicast protocol data transmitting / receiving unit 23 will be described.
FIG. 4 is a diagram showing a layout of the ordered multicast protocol data. As shown in FIG. 4, the ordered multicast protocol data transmitted / received by the ordered multicast protocol data transmitting / receiving unit 23 includes type, sender, input order number, step number (ordered multicast step number), maximum confirmed input order number, processing. Includes fields for type and input packet. The aligned multicast protocol data shown in FIG. 4 is different from the protocol data described in Patent Document 1 in that the processing type field is added.

The ordered multicast protocol data is used in the following three types depending on the type field at the top.
(1) Candidate type The input order number field, the step number field, and the input packet field respectively store a corresponding input order number, a corresponding step number, and a self-candidate at the time of transmission by the sender (transmission side computer).

(2) Determined type Indicates that the input packet corresponding to the input sequence number (the input sequence number stored in the input sequence number field) is in the input packet journal storage unit 22 at the time of transmission by the sender. The input packet is stored in the field. In this case, the step number field is not used.

(3) Delay type Indicates that there is no input packet corresponding to the input sequence number in the input packet journal storage unit 22 at the time of transmission by the sender. In this case, the step number field and the input packet field are not used.

  In any type of ordered multicast protocol data, the maximum determined input sequence number field stores the corresponding maximum determined input sequence number at the time of transmitting the ordered multicast protocol data from the sender (sender computer). In addition, the corresponding maximum confirmed input sequence number in the receiving computer of the ordered multicast protocol data is the sequence number of the input packet determined in the receiving computer and the maximum determined in the ordered multicast protocol data received in the receiving computer. The input sequence number is updated to the largest one.

  In the present embodiment, unlike the computer described in Patent Document 1, the computer 100-i in the distributed system 10 dynamically changes the computer constituting the multiplexing according to the situation of the system 10. Includes new configurations. That is, the computer 100-i further includes a system configuration management unit 6 as shown in FIG. The system configuration management unit 6 manages and determines the configuration of the distributed system 10. The determination of the configuration of the distributed system 10 is performed based on an agreement between the computers constituting the multiplexing.

  The system configuration management unit 6 includes a system configuration storage unit 61, a maximum confirmed input sequence number history storage unit 62, a system configuration change detection unit 63, a system configuration change setting unit 64, a system configuration change agreement protocol data transmission / reception unit 65, and a system configuration notification The protocol data transmission / reception unit 66 is configured.

  The system configuration storage unit 61 stores system configuration information related to the configuration of the distributed system 10. The system configuration information includes information indicating the computers constituting the distributed system 10 and the state of the computers. The state of the computer is either an operating state in which the computer is operating (that is, a state that configures multiplexing) or a standby state that is waiting (that is, a state that does not configure multiplexing). The system configuration information further includes information (priority information) indicating the priority of the computers constituting the distributed system 10. This priority is used to determine a computer to be preferentially used when a failover is performed in response to detection of a stagnation of a computer in an operating state.

  FIG. 5 shows an exemplary data structure of the system configuration storage unit 61. In the example of FIG. 5, a computer name is used as information indicating the computers constituting the distributed system 10. In this embodiment, if the computer names of the computers 100-1 to 100-5 are computers # 1 to # 5, the system configuration storage unit 61 shown in FIG. -5.

  The system configuration storage unit 61 shown in FIG. 5 indicates that the computers 100-1 to 100-4 are in an operating state and the computer 100-5 is in a standby state. That is, the system configuration storage unit 61 shown in FIG. 5 shows the configuration of the distributed system 10 in the state shown in FIG.

  In the system configuration storage unit 61 shown in FIG. 5, the priorities of the computers 100-1, 100-2, 100-3, 100-4, and 100-5 are 1, 2, 3, 4, and 5 respectively. Indicates. In the present embodiment, the lower the priority value, the higher the priority. Therefore, in the example of FIG. 5, the computer 100-1 has the highest priority, and the computers 100-2, 100-3, 100-4, and 100-5 are in the following order. However, in the present embodiment in which n is 4 and m is 5, only the computer 100-5 is in the standby state, and therefore when performing failover, the computer 100-5 is irrelevant to the priority. Used for. If m is larger than 5, and therefore there are a plurality of computers in the standby state including the computer 100-5, the highest priority among the plurality of computers in the standby state when performing failover. A high computer is used.

  The maximum confirmed input sequence number history storage unit 62 stores the maximum input sequence number (maximum confirmed input sequence number) for which it is known that the delivery has been confirmed for each computer constituting the multiplexing (in operation state). Store history. More specifically, the maximum confirmed input sequence number history storage unit 62 is a time stamp indicating the time each time the maximum confirmed input sequence number stored in the maximum confirmed input sequence number storage unit 27 of the own computer is changed. In addition, the maximum input sequence number of each computer that constitutes multiplexing (computers 100-1 to 100-4 in the example of FIG. 1) whose delivery has been confirmed is stored in chronological order. Used to do.

  FIG. 6 shows an example of the data structure of the maximum confirmed input sequence number history storage unit 62. In the example of FIG. 6, the maximum confirmed input sequence number in the computer 100-4 (# 4) after a certain time is smaller than that of the other computers 100-1 (# 1) to 100-3 (# 3) and is delayed. It is shown that.

  The system configuration change detection unit 63 detects that the computer configuration is stagnant based on the maximum confirmed input sequence number history stored in the maximum confirmed input sequence number history storage unit 62, thereby changing the system configuration. Detect (determine). When the system configuration change detection unit 63 detects a stagnation computer, the system configuration change agreement protocol data transmission / reception unit transmits the detected stagnation computer to all computers in other operating states. Report via 65. For this report, a report type system configuration change agreement protocol data is used.

  In response to the above-described report, the system configuration change detection unit 63 also includes (n−t−1) or more (n = 4, t = 1 in this embodiment) among the computers in other operating states. From the above, if an agreement is returned via the system configuration change agreement protocol data transmission / reception unit 65, an agreement can be reached on (nt) or more units including the own computer regarding the detection of the stagnant computer. Judge that

  The system configuration change detection unit 63 also reports the report based on the maximum confirmed input sequence number history stored in the maximum confirmed input sequence number history storage unit 62 when a stagnated computer is reported from another computer. Determine whether you agree with. The system configuration change detection unit 63 notifies this determination result (that is, whether to agree or disagree) to the computer that reported the stagnating computer. This notification uses agreement type or non-agreement type system configuration change agreement protocol data.

  The system configuration change detection unit 63 also confirms the detection of a stagnant computer when an agreement has been reached on (nt) or more units including its own computer regarding the detection of the stagnant computer. Further, the system configuration change detection unit 63 sets the confirmed stagnated computer in a standby state and sets the computer in the standby state in an operating state in response to confirmation of detection of the stagnating computer. A system configuration change request packet for requesting a configuration change is passed to the reception queue 1 as an input packet.

  The system configuration setting unit 64 sends the system configuration change request packet from the agreement unit 262 as a result of the agreement of the system configuration change request packet by the agreement unit 262, that is, as a result of the system configuration change request packet being confirmed by the agreement unit 262. If so, the configuration of the distributed system requested by the system configuration change request packet is changed.

  The system configuration change agreement protocol data transmission / reception unit 65 communicates system configuration change agreement protocol data for forming an agreement with another computer when a stagnant computer is detected by the system configuration change detection unit 63. .

  FIG. 7 is a diagram showing a layout of system configuration change agreement protocol data transmitted / received by the system configuration change agreement protocol data transmission / reception unit 65. As shown in FIG. 7, the system configuration change agreement protocol data includes the type, sender, proposer, proposal time stamp, and stagnation confirmation target computer name (the name of the computer suspected of stagnation). Contains each field. The stagnation confirmation target computer name field corresponds to the input packet field of the ordered multicast protocol data. That is, in this embodiment, the contents of the stagnation confirmation target computer name field are treated as an input packet.

The system configuration change agreement protocol data is divided into the following three types according to the type field at the top.
(1) Report type This indicates that the proposer (computer) indicated in the proposer field has detected another computer that is constantly stagnating. In the sender field and the proposal time stamp field, the sender (sender computer) of the system configuration change agreement protocol data (that is, the report type system configuration change agreement protocol data) including the type field indicating the report type, respectively. The computer name to be shown and the time stamp when the stagnating computer is detected (time stamp at the time of proposal) are stored. In the stagnation confirmation target computer name field, the computer name of the computer that needs to be confirmed regarding the detection of stagnation (that is, the computer subject to stagnation confirmation) is stored. The sender indicated by the sender field matches the proposer indicated by the proposer field.

(2) Consent type Indicates that the computer that received the report type system configuration change agreement protocol data has detected an equivalent stagnation. The sender field stores the computer name indicating the sender of the system configuration change agreement protocol data (that is, the agreement type system configuration change agreement protocol data) including the type field indicating the agreement type. For the proposer field, the proposal time stamp field, and the stagnation confirmation target computer name field, copies of the contents of the corresponding fields of the report type system configuration change agreement protocol data are used.

(3) Disagreement type Indicates that the computer that received the report type system configuration change agreement protocol data has not detected an equivalent stagnation. The sender field stores the computer name indicating the sender of the system configuration change agreement protocol data including the type field indicating the non-agreement type (that is, the system configuration change agreement protocol data of the non-agreement type). For the proposer field, the proposal time stamp field, and the stagnation confirmation target computer name field, copies of the contents of the corresponding fields of the report type system configuration change agreement protocol data are used.

  The system configuration notification protocol data transmission / reception unit 66 receives system configuration notification protocol data for putting a computer in a standby state into an operating state at a timing when the system configuration change detection unit 63 detects that the system configuration should be changed. Send and receive. The system configuration notification protocol data includes data (system configuration information) stored in the system configuration storage unit 61 itself (that is, dump information of the system configuration storage unit 61).

Next, the operation principle of the computer in the operating state in the distributed system 10 will be specifically described.
(Initialization process)
First, the system configuration storage unit 61 of the computers 100-1 to 100-5 stores information indicating the initial state of the distributed system 10. FIG. 5 shows an example of the data structure of the system configuration storage unit 61 at this time. Here, the computers 100-1 to 100-4 are in an operating state, and the computer 100-5 is in a standby state.

  The input sequence number storage unit 21 of the computer 100-j (j = 1 to 4) in the operating state stores the input sequence number in the initial state, that is, the initial input sequence number (for example, 1). The input packet journal storage unit 22 and the candidate packet storage unit 25 of the computer 100-j are empty.

  The step number storage unit 24 of the computer 100-j stores an initial step number (for example, 1), and the maximum determined input sequence number storage unit 27 stores the initial input sequence number. All the (n-1) flags (here, three flags) in the delay storage unit 28 of the computer 100-j are reset, and the maximum confirmed input sequence number history storage unit 62 stores the current time stamp and the computer 100. The initial input sequence number for each of -1 to 100-4 is stored.

  The process of determining the input packet delivered to each computer by the ordered multicast executed by the ordered multicast unit 2 of the computer 100-j is realized by algorithms 1-9.

  First, algorithms 1 to 4 will be described with reference to FIGS. 8 and 9. FIG. 8 and FIG. 9 are flowcharts showing an operation procedure of a basic part that performs one delivery of ordered multicast. Step A1 in the flowchart of FIG. 8 (first flowchart) represents the entire algorithms 1 to 3, and the remaining steps A2 to A13 represent the algorithm 4. In the flowchart of FIG. 9 (second flowchart), steps B1 to B4 indicate algorithm 1, steps B5 to B8 indicate algorithm 2, and steps B9 and B10 indicate algorithm 3.

(Algorithm 1)
The input candidate collection unit 261 included in the input packet confirmation determination unit 26 in the ordered multicast unit 2 is an input selected as a candidate (input candidate) to be processed next by each of the computers 100-1 to 100-4 (n = 4). A part (steps B1 to B4 in FIG. 9) of the candidate list creation process (step A1 in FIG. 8) for collecting packets (input data) is executed as follows.

  When the corresponding step number stored in the step number storage unit 24 is an initial value (YES in step B1 in FIG. 9), the input candidate collection unit 261 determines whether an input packet exists in the reception queue 1 (Step B2 in FIG. 9).

  If there is an input packet (YES in step B2 in FIG. 9), the input candidate collection unit 261 advances the corresponding step number stored in the step number storage unit 24 (step B3 in FIG. 9). Then, the input candidate collection unit 261 stores the input packet as a self-candidate in the candidate packet storage unit 25, and sorts the candidate type protocol data (aligned multicast protocol data) in which the self-candidate is set in the input packet field. The protocol data transmitting / receiving unit 23 transmits the data to all other computers (that is, all computers except the computer 100-j among the computers 100-1 to 100-4) via the network B (step B4 in FIG. 9). In step B4, the input candidate collection unit 261 empties all other candidates in the candidate packet storage unit 25.

(Algorithm 2)
On the other hand, when the corresponding step number is not the initial value (NO in step B1 in FIG. 9) or there is no input packet in the reception queue 1 (NO in step B2 in FIG. 9), the input candidate collection unit 261 receives the input sequence number. Whether the ordered multicast protocol data transmitting / receiving unit 23 receives candidate type of ordered multicast protocol data having an input order number (input order number field in which the input order number matches the corresponding input order number stored in the storage unit 21). Is determined (step B5 in FIG. 9). If it is received (YES in step B5 in FIG. 9), the input candidate collection unit 261 is set in the (step number field) in the received ordered multicast protocol data (received ordered multicast protocol data). It is determined whether the step number is larger than the corresponding step number (step B6 in FIG. 9).

  If the step number in the reception aligned multicast protocol data is larger than the corresponding step number (YES in step B6 in FIG. 9), the input candidate collection unit 261 stores the corresponding step number stored in the step number storage unit 24. Is updated to the step number in the reception aligned multicast protocol data (step B7 in FIG. 9).

  Next, the input candidate collection unit 261 stores the input packet (set in the input packet field) in the received aligned multicast protocol data as a self-candidate in the candidate packet storage unit 25, and the self-candidate is stored in the input packet field. Aligned multicast protocol data of a candidate type set to (that is, including a copy of the input packet field in the received aligned multicast protocol data) is transmitted to all other computers via the network B by the aligned multicast protocol data transmitting / receiving unit 23 (Step B8 in FIG. 9). In this step B8, the input candidate collection unit 261 sends the input packet in the received aligned multicast protocol data (in this case, the input packet determined as its own candidate) to the sender (indicated by the sender field) of the received aligned multicast protocol data. The candidate packet storage unit 25 stores the corresponding other candidate. That is, in step B8, the input packet in the received aligned multicast protocol data is set as its own candidate and also set as another candidate corresponding to the sender of the received aligned multicast protocol data. At this time, the input candidate collection unit 261 discards (empties) all other candidates in the candidate packet storage unit 25 other than the other candidates corresponding to the sender of the received aligned multicast protocol data.

(Algorithm 3)
On the other hand, if the step number in the reception aligned multicast protocol data is equal to the corresponding step number (NO in step B6 in FIG. 9, YES in step B9), the input candidate collection unit 261 includes the received alignment multicast protocol data. Are stored in the candidate packet storage unit 25 as other candidates corresponding to the sender of the received aligned multicast protocol data (step B10 in FIG. 9).

  When the input candidate collection unit 261 executes Step B6 or Step B10, the number of candidates stored in the candidate packet storage unit 25 is (nt) or more (in the example of n = 4, t = 1, 3 or more). (Step B11 in FIG. 9).

  If the number of candidates does not reach (nt) or more (NO in step B11 in FIG. 9), the input candidate collection unit 261 executes the processing from step B1 again. On the other hand, if the number of candidates is (nt) or more (YES in step B11 in FIG. 9), the input candidate collection unit 261 ends the candidate list creation process. Even when the determination in step B5 or step B9 is NO, the processing from step B1 is executed again.

(Algorithm 4)
When the candidate list creation process (step A1 in FIG. 8) is completed, that is, when the number of candidates (input candidates) (number of non-empty candidates) stored in the candidate packet storage unit 25 becomes (nt) or more, the input packet The agreement unit 262 in the decision determination unit 26 functions as a first input candidate selection control unit, and whether or not (nt) or more identical candidates exist in the candidate packet storage unit 25, that is, (n− t) It is determined whether or not there is a candidate for which an agreement has been reached (an agreement has been formed) on more than one computer (step A2 in FIG. 8).

  If there are (nt) or more (same content) candidates (YES in step A2 in FIG. 8), the agreement unit 262 determines the candidates as input packets in the corresponding input sequence number. (Step A3 in FIG. 8). In step A <b> 3, the agreement unit 262 updates the corresponding maximum confirmed input sequence number stored in the maximum confirmed input sequence number storage unit 27 to the corresponding input sequence number. In step A <b> 3, if the confirmed input packet exists in the reception queue 1, the agreement unit 262 deletes the input packet from the reception queue 1.

  As described above, when the input packet at the corresponding input sequence number is confirmed, that is, when the input packet at the corresponding input sequence number is agreed and the ordered multicast is confirmed, the agreement unit 262 proceeds to step A4 in FIG. In step A4, the agreement unit 262 functions as a candidate output destination switching unit, and whether or not the processing type indicated by the processing type information attached to the determined input packet is “configuration” (“application”). In other words, it is determined whether the determined input packet is a system configuration change request packet for requesting a system configuration change. Details of the system configuration change request packet will be described in the processing of the algorithm 12 described later.

  If the processing type is “configuration”, that is, if the confirmed input packet is a system configuration change request packet (YES in step A4 in FIG. 8), the agreement unit 262 determines the confirmed input packet as The input packet is transferred to the system configuration setting unit 64 in the system configuration management unit 6 and the input packet is stored in the input packet journal storage unit 22 (step A5 in FIG. 8).

  When the system configuration setting unit 64 receives the input packet determined from the agreement unit 262, is the system configuration change request packet requesting that the packet change the computer 100-j itself (own computer) to a standby state? Is determined (step A6 in FIG. 8). If it is a system configuration change request packet for requesting that the own computer be changed to a standby state (YES in step A6 in FIG. 8), the system configuration setting unit 64 performs the initialization process described above to A standby state is set (step A7 in FIG. 8). The system configuration management unit 6 of the computer in the standby state waits for the system configuration notification protocol data for notifying the change from the standby state to the operating state from the computer in the operating state. On the other hand, the aligned multicast unit 2 of the computer that has entered the standby state stops its operation to save power, for example.

  On the other hand, when the determined input packet is a system configuration change request packet requesting that another computer be changed to a standby state (NO in step A6 in FIG. 8), the system configuration setting unit 64 The system configuration information stored in the configuration storage unit 61 is updated to indicate the system configuration after the change, and system configuration notification protocol data including the updated system configuration information (that is, the system configuration after the change is notified) System configuration notification protocol data) is transmitted to all other computers constituting the distributed system by the system configuration notification protocol data transmitting / receiving unit 66 (step A8 in FIG. 8). In this case, the agreement unit 262 advances the corresponding input sequence number stored in the input sequence number storage unit 21 to the next step (increment by 1), and is stored in the step number storage unit 24. The corresponding step number is initialized (step A9 in FIG. 8). In step A9, the agreement unit 262 discards (empties) all candidates stored in the candidate packet storage unit 25, and sets all (n−1) delay flags stored in the delay storage unit 28. Reset.

  On the other hand, if the processing type is not “configuration”, that is, if the determined input packet is not a system configuration change request packet (NO in step A4 in FIG. 8), the agreement unit 262 delivers the input packet to the application program 3 At the same time, the input packet is stored in the input packet journal storage unit 22 (step A10 in FIG. 8). Here, the determined input packet is stored so as to be positioned behind the column of input packets currently stored in the input packet journal storage unit 22. If the input packet sequence stored in the input packet journal storage unit 22 reaches a certain amount, the first input packet (that is, the oldest input packet) is discarded. Thereafter, the agreement unit 262 executes Step A9 to shift to the next process.

  On the other hand, when there are not (nt) or more identical candidates (NO in step A2 in FIG. 8), the agreement unit 262 functions as a second input candidate selection control unit, and this time, candidates It is determined whether or not a majority of the same candidates exist in the packet storage unit 25 (step A11 in FIG. 8). If more than half of the same candidates exist (YES in step A11 in FIG. 8), the agreement unit 262 selects the candidate and stores it as a candidate in the candidate packet storage unit 25 (that is, the candidate). And the candidate type protocol data (aligned multicast protocol data) in which the self candidate is set in the input packet field via the network B by the aligned multicast protocol data transmitting / receiving unit 23. To all other computers (step A12 in FIG. 8). In step A12, the agreement unit 262 discards all other candidates stored in the candidate packet storage unit 25.

  On the other hand, if more than a majority of the same candidates do not exist (NO in step A11 in FIG. 8), the agreement unit 262 functions as a third input candidate selection control unit and is stored in the candidate packet storage unit 25. A candidate (input packet) is randomly selected from among the input candidates, and the protocol data of the candidate type (aligned multicast protocol data) in which the self candidate is set in the input packet field is aligned multicast protocol data The transmission / reception unit 23 transmits the data to all other computers via the network B (step A13 in FIG. 8). In step A13, the agreement unit 262 discards all other candidates stored in the candidate packet storage unit 25.

  When step A12 (operation as the second input candidate selection control means) or step A13 (operation as the third input candidate selection control means) is completed, the processing from step A1 is executed again. On the other hand, when step A7 or step A9 ends, one delivery process of the ordered multicast ends.

  With the above procedure, each computer 100-j constituting the multiplexing, that is, the computers 100-1 to 100-4, confirms the coincidence of input packets in (nt) computers (three computers in this case). While proceeding.

  Next, algorithms 5 to 9 will be described with reference to FIGS. 10 to 13. FIG. 10 to FIG. 13 are flowcharts showing the procedure of processing for eliminating the delay of multiplexing execution. Steps C1 and C2 in the flowchart of FIG. 10 (first flowchart) represent algorithm 5, and step C3 represents algorithm 7. The flowchart of FIG. 11 (second flowchart) shows the algorithm 6, the flowchart of FIG. 12 (third flowchart) shows the algorithm 8, and the flowchart of FIG. 13 (first flowchart) shows the algorithm 9.

(Algorithm 5)
The agreement unit 262 in the ordered multicast unit 2 responds to the input sequence number when the ordered multicast protocol data transmitting / receiving unit 23 receives candidate type of ordered multicast protocol data having an input sequence number smaller than the corresponding input sequence number. It is determined whether or not an input packet to be present exists in the input packet journal storage unit 22 (step C1 in FIG. 10).

  If an input packet corresponding to an input sequence number smaller than the corresponding input sequence number exists in the input packet journal storage unit 22 (YES in step C1 in FIG. 10), that is, because of a short multiplexing execution delay, If an input packet corresponding to an input sequence number smaller than the input sequence number is left in the input packet journal storage unit 22, the agreement unit 262 determines the type of ordered multicast protocol in which the input packet is set in the input packet field. The ordered multicast protocol data transmitting / receiving unit 23 returns the data to the sender of the received ordered multicast protocol data via the network B (step C2 in FIG. 10).

(Algorithm 6)
In addition, when the ordered multicast protocol data having a definite type having an input sequence number that matches the input sequence number is received by the ordered multicast protocol data transmitting / receiving unit 23, the agreement unit 262 The input packet is determined as the input packet (step D1 in FIG. 11). In step D <b> 1, if the confirmed input packet exists in the reception queue 1, the agreement unit 262 deletes the input packet from the reception queue 1. In step D1, if the maximum confirmed input sequence number in the received aligned multicast protocol data is greater than the corresponding maximum confirmed input sequence number stored in the maximum confirmed input sequence number storage unit 27, the corresponding maximum confirmed input sequence number. Is updated to the maximum definitive input sequence number in the receive aligned multicast protocol data.

  After step D1, processes D2 to D7 corresponding to steps A4 to A9 in algorithm 4 described above are executed as follows. First, the agreement unit 262 determines whether the processing type indicated by the processing type information attached to the confirmed input packet is “configuration” (“application”), that is, the confirmed input packet is a system configuration change. Is determined as a system configuration change request packet (step D2 in FIG. 11).

  If the processing type is “configuration”, and therefore the confirmed input packet is a system configuration change request packet (YES in step D2 in FIG. 11), the agreement unit 262 converts the confirmed input packet into the system configuration. While passing to the system configuration setting unit 64 in the management unit 6, the input packet is stored in the input packet journal storage unit 22 (step D3 in FIG. 11).

  When the system configuration setting unit 64 receives the input packet determined from the agreement unit 262, the system configuration setting unit 64 determines whether the packet is a system configuration change request packet for requesting the computer to be changed to a standby state (FIG. 11). Step D4). If it is a system configuration change request packet requesting that the own computer be changed to a standby state (YES in step D4 in FIG. 11), the system configuration setting unit 64 performs initialization processing and puts the own computer in a standby state. (Step D5 in FIG. 11).

  On the other hand, when the determined input packet is a system configuration change request packet that requests that another computer be changed to a standby state (NO in step D4 in FIG. 11), the system configuration setting unit 64 The system configuration information stored in the configuration storage unit 61 is updated to indicate the system configuration after the change, and system configuration notification protocol data including the updated system configuration information (that is, the system configuration after the change is notified) System configuration notification protocol data) is transmitted to all other computers constituting the distributed system 10 by the system configuration notification protocol data transmitter / receiver 66 (step D6 in FIG. 11). In this case, the agreement unit 262 advances the corresponding input sequence number stored in the input sequence number storage unit 21 to the next step (increment by 1), and is stored in the step number storage unit 24. The corresponding step number is initialized (step D7 in FIG. 11). In step D7, the agreement unit 262 discards all candidates stored in the candidate packet storage unit 25 and resets all (n−1) delay flags stored in the delay storage unit 28.

  On the other hand, if the processing type is not “configuration”, that is, if the determined input packet is not an input packet for requesting a system configuration change (NO in step D2 in FIG. 11), the agreement unit 262 determines that the input packet is an application program. 3 and the input packet is stored in the input packet journal storage unit 22 (step D8 in FIG. 11). Thereafter, the agreement unit 262 executes Step D7 to shift to the next process.

(Algorithm 7)
On the other hand, when the candidate type of ordered multicast protocol data having an input sequence number smaller than the corresponding input sequence number is received, an input packet corresponding to the input sequence number smaller than the corresponding input sequence number exists in the input packet journal storage unit 22 If not (NO in step C1 in FIG. 10), the agreement unit 262 proceeds to step C3. That is, when an input packet corresponding to an input sequence number smaller than the corresponding input sequence number has already been discarded from the input packet journal storage unit 22 due to a long multiplexing execution delay (NO in step C1 in FIG. 10). The agreement unit 262 causes the arranged multicast protocol data transmission / reception unit 23 to return the delay type arranged multicast protocol data to the sender of the received arranged multicast protocol data via the network B (step C3 in FIG. 10).

(Algorithm 8)
On the other hand, the skip determining unit 29 in the ordered multicast unit 2 stores the delay when the ordered multicast protocol data transmitting / receiving unit 23 receives a delay type of ordered multicast protocol data having an input sequence number that matches the corresponding input sequence number. Among the (n−1) delay flags stored in the unit 28, a delay flag corresponding to the sender of the protocol data is set (step E1 in FIG. 12).

(Algorithm 9)
The skip determination unit 29 includes the number of set delay flags among the (n−1) delay flags stored in the delay storage unit 28 and the number of input candidates stored in the candidate packet storage unit 25. It is monitored whether the sum of and has reached (nt) or more (step F1 in FIG. 13). If the number has reached (n−t) or more (YES in step F1 in FIG. 13), the skip determination unit 29 determines that the number of input candidates stored in the candidate packet storage unit 25 is (nt). It is determined whether the number is less than the number (step F2 in FIG. 13). If the number is less than (nt) (YES in step F2 in FIG. 13), the skip determination unit 29 performs a skip operation (step F3 in FIG. 13). That is, the skip determination unit 29 updates the corresponding input sequence number stored in the input sequence number storage unit 21 to the corresponding maximum determined input sequence number stored in the maximum determined input sequence number storage unit 27, and the step number storage unit The corresponding step number stored in 24 is set as the initial step number. The skip determination unit 29 empties the candidate packet storage unit 25, resets all delay flags in the delay storage unit 28, and notifies the system configuration management unit 6 and the program state management unit 4 of the skip.

  When the skip is notified from the skip determination unit 29, the system configuration setting unit 64 of the system configuration management unit 6 uses the system configuration information stored in the system configuration storage unit 61 of another computer as the system configuration storage unit of its own computer. 61 is copied (step F4 in FIG. 13). Based on the copied system configuration information, the system configuration setting unit 64 determines whether it is necessary to change the own computer to a standby state (step F5 in FIG. 13).

  If it is necessary to change the own computer to the standby state (YES in step F5 in FIG. 13), the system configuration setting unit 64 performs the initialization process described above to place the own computer in the standby state (FIG. 13). Step F6). On the other hand, if it is not necessary to change the own computer to the standby state (NO in step F5 in FIG. 13), the system configuration setting unit 64 does nothing and ends the process.

  On the other hand, when the skip is notified from the skip determination unit 29, the program state management unit 4 copies the state immediately before the corresponding input sequence number from the program state management unit 4 of another computer (step F7 in FIG. 13). For this reason, the program state management unit 4 holds the state immediately before each input sequence number by a certain amount from the latest one.

  In the flowchart of FIG. 13, the process by the program state management unit 4 (step F7) is executed after the process by the system configuration setting unit 64. However, the order of the processes may be reversed, or both processes may be executed in parallel.

Next, processing for system configuration change executed by the system configuration management unit 6 will be described. This processing is realized by algorithms 10 to 14.
(Algorithm 10)
First, the algorithm 10 will be described with reference to FIG. FIG. 14 is a flowchart showing a procedure of processing for searching for a stagnant computer executed by the system configuration change detection unit 63 of the system configuration management unit 6.

The system configuration change detection unit 63 performs the following processing at regular time intervals Δt1 (step G1 in FIG. 14).
First, the system configuration change detection unit 63 refers to the maximum confirmed input sequence number history storage unit 62, and among the maximum confirmed input sequence number history stored in the storage unit 62, the latest time Δt2 (Δt2> Δt1). Information on the maximum confirmed input order number added in between is acquired (step G2 in FIG. 14). The information on the maximum confirmed input sequence number added during the latest time Δt2 is specified based on the time stamp.

  Next, the system configuration change detection unit 63 determines that other computers in the operating state have a maximum confirmed input sequence number that is less than or equal to Δi (Δi is an integer greater than 1) than the maximum confirmed input sequence number of the own computer (that is, the own computer Whether or not the number of times satisfying the condition that the computer is delayed by a time corresponding to Δi from the computer) is more than a certain number is determined based on the acquired information on the maximum confirmed input sequence number (step G3 in FIG. 14). ). The system configuration change detection unit 63 repeats the determination in step G3 for all other computers in the operating state (step G4 in FIG. 14). As a result, the system configuration change detection unit 63 searches for a computer in an operating state in which the number of times satisfying the above condition is equal to or greater than a certain number.

  If such a computer is found (YES in step G3 in FIG. 14), the system configuration change detection unit 63 determines that the computer is stagnant (steadily stagnating). Therefore, the system configuration change detection unit 63, for example, a report type system that targets the certified computer as a stagnation confirmation target for computers in other operating states except for the computer that is certified to be stagnant. The configuration change agreement protocol data is transmitted by the system configuration change agreement protocol data transmission / reception unit 65 (step G5 in FIG. 14).

(Algorithm 11)
Next, the algorithm 11 will be described with reference to FIG. FIG. 15 is a flowchart illustrating a processing procedure executed by the system configuration change detection unit 63 when report type system configuration change agreement protocol data is received from another computer.

  Now, the system configuration change detection unit 63 uses the report type system configuration change agreement protocol data transmission / reception unit 65 of another computer as the system configuration change agreement protocol data transmission / reception unit 65 of its own computer. It is assumed that it was received via Then, the system configuration change detection unit 63 refers to the maximum confirmed input sequence number history storage unit 62 and is added during the latest time Δt2 in the maximum confirmed input sequence number history stored in the storage unit 62. Information on the maximum confirmed input order number is acquired (step H1 in FIG. 15).

  Next, the system configuration change detecting unit 63 satisfies the condition that the stagnation confirmation target computer reported in the received system configuration change agreement protocol data satisfies a condition that the computer is less than Δi compared to the maximum confirmed input sequence number of the own computer. Is determined based on the acquired information on the maximum confirmed input sequence number (step H2 in FIG. 15). That is, the system configuration change detection unit 63 detects whether or not the reported stagnation confirmation target computer is stagnant.

  If the determination in step H2 is YES, that is, if a stagnation of the computer subject to the reported stagnation confirmation is detected, the system configuration change detection unit 63 reports the system configuration change agreement protocol data of the report type. The system configuration change agreement protocol data transmission / reception unit 65 returns the consent type system configuration change agreement protocol data to the transmission source computer (step H3 in FIG. 15). On the other hand, if the determination in step H2 is NO, that is, if no stagnation of the computer subject to the reported stagnation confirmation is detected, the system configuration change detection unit 63 changes the report type system configuration change. The system configuration change agreement protocol data transmission / reception unit 65 of the non-agreement type is returned to the computer that has transmitted the agreement protocol data by the system configuration change agreement protocol data transmission / reception unit 65 (step H4 in FIG. 15).

(Algorithm 12)
Next, the algorithm 12 will be described with reference to FIG. FIG. 16 shows a case where consent type or non-agreement type system configuration change agreement protocol data is received from another computer as a response to the report type system configuration change agreement protocol data executed by the system configuration change detection unit 63. It is a flowchart which shows the procedure of a process.

  Now, the system configuration change detection unit 63 uses the system configuration change agreement protocol data of the own computer as the agreement type or non-agreement type system configuration change agreement protocol data returned from the system configuration change agreement protocol data transmission / reception unit 65 of the other computer. It is assumed that the data is received via the data transmission / reception unit 65. Then, the system configuration change detection unit 63 refers to the proposer field and the proposal time stamp field of the received system configuration change agreement protocol data of the consent type or the non-agreement type (step I1 in FIG. 16).

  Next, the system configuration change detection unit 63 includes the system configuration change protocol data received this time, and both the proposer and the proposal time stamp are the same agreement type or non-agreement type as the report type system configuration change agreement protocol data. It is determined whether (nt-1) or more system configuration change agreement protocol data has been received (step I2 in FIG. 16).

  If (nt-1) or more of such system configuration change agreement protocol data is received (YES in step I2 in FIG. 16), the system configuration change detection unit 63 (nt-t-) 1) It is determined whether or not the number of pieces of data is agreement type system configuration change agreement protocol data (step I3 in FIG. 16). That is, in step I3, the system configuration change detection unit 63 obtains (nt−t−) the system configuration change agreement protocol data of the same agreement type as the report type system configuration change agreement protocol data. 1) It is determined whether or not it has been received. This determination is equivalent to determining whether or not an agreement has been reached for computers in which stagnation has been detected among computers in the operating state of (nt) or more.

  If the determination in step I3 is YES, that is, if it is determined that an agreement has been reached between computers in the operating state of (nt) or more computers with stagnation detected, the system configuration change detection unit 63 proceeds to step I4. In step I4, the system configuration change detection unit 63 refers to the system configuration storage unit 61 and determines whether there is a computer in a standby state. That is, the system configuration change detection unit 63 determines whether there is a computer that can fail over from the computer in which the stagnation is detected (step I4 in FIG. 16).

  If the determination in step I4 is YES, the system configuration change detection unit 63 generates a system configuration change request packet and places the system configuration change request packet in the reception queue 1 as an input packet (step I5 in FIG. 16). The system configuration change request packet (input packet) is attached with processing type information indicating “configuration”. The system configuration change request packet loaded in the reception queue 1 is processed by the algorithm 4 described above as an input packet.

The system configuration change request packet includes system configuration information. The system configuration change detection unit 63 generates the system configuration information using a copy of the system configuration information currently stored in the system configuration storage unit 61 of the own computer. That is, the system configuration change detection unit 63 updates the information indicating the state of the computer that has been agreed that the stagnation is detected among the copies of the system configuration information so as to indicate the standby state, and prioritizes the computer. The information is updated so as to indicate the operating state, and the information indicating the state of the computer in the standby state with the highest priority is updated. The updated copy of the system configuration information is used for the system configuration change request packet. Therefore, the system configuration change request packet requests that the computer that has been agreed that the stagnation has been detected be set in the standby state, and sets the priority of the computer to be lower than those in other standby computers. The computer in the standby state with the highest priority is requested to be in the operating state. That is, the system configuration change request packet is a packet for requesting failover from the computer that has been agreed that the stagnation has been detected to the computer in the standby state with the highest priority.
On the other hand, if the determination in step I2, I3, or I4 is NO, the system configuration change detection unit 63 ends the process.

(Algorithm 13)
Next, the algorithm 13 will be described with reference to FIG. FIG. 17 is a flowchart showing a processing procedure executed by the system configuration setting unit 64 when a system configuration change request packet determined from the agreement unit 262 of the ordered multicast unit 2 is received.

  When the system configuration setting unit 64 receives the confirmed system configuration change request packet from the agreement unit 262, the system configuration setting unit 64 executes the following processing for all the computers constituting the distributed system 10 (step J1 in FIG. 17).

  First, the system configuration setting unit 64 stores system configuration information (hereinafter referred to as new system configuration information) included in the system configuration change request packet passed from the agreement unit 262 in the current system configuration storage unit 61. The system configuration information (hereinafter referred to as current system configuration information) is compared (step J2 in FIG. 17). Based on the result of this comparison, the system configuration setting unit 64 determines whether a change from the standby state to the operating state is requested for the current target computer (step J3 in FIG. 17). Note that when a change from the operating state to the standby state is requested, processing is performed by algorithm 4 (specifically, step A4 and subsequent steps).

  If the current target computer is requested to change from the standby state to the operating state (YES in step J3 in FIG. 17), the system configuration setting unit 64 transmits / receives the system configuration notification protocol data of its own computer. New system configuration information is transmitted from the unit 66 to the system configuration notification protocol data transmission / reception unit 66 of the current target computer (that is, the computer requested to change from the standby state to the operating state) using the system configuration notification protocol data. (Step J4 in FIG. 17). The transmission of the system configuration notification protocol data (new system configuration information) notifies the transmission destination computer of the change from the standby state to the operating state.

  Further, the system configuration setting unit 64 initializes information related to the computer requested to be changed to the operating state, which is stored in each storage unit in the own computer (step J5 in FIG. 17). Here, among the (n−1) delay flags in the delay storage unit 28, a delay flag newly associated with the computer requested to change to the operating state, and the maximum confirmed input sequence number history storage unit Of the maximum confirmed input sequence number histories in 62, the maximum confirmed input sequence number history newly associated with the computer requested to change to the operating state is initialized. In step J5, the current system configuration information stored in the system configuration storage unit 61 is updated to new system configuration information.

(Algorithm 14)
Next, the algorithm 14 will be described with reference to FIG. FIG. 18 is a flowchart showing a processing procedure executed by the system configuration setting unit 64 when system configuration notification protocol data is received from the system configuration notification protocol data transmission / reception unit 66.

  Assume that the system configuration notification protocol data transmission / reception unit 66 of the own computer has received the system configuration notification protocol data transmitted from the system configuration notification protocol data transmission / reception unit 66 of another computer. The system configuration notification protocol data transmission / reception unit 66 passes the system configuration notification protocol data received by itself to the system configuration setting unit 64 in its own computer.

Upon receiving the system configuration notification protocol data from the system configuration notification protocol data transmission / reception unit 66, the system configuration setting unit 64 determines whether the computer is in a standby state (step K1 in FIG. 18). If the own computer is in a standby state (YES in step K1 in FIG. 18), the system configuration setting unit 64 indicates that the received system configuration notification protocol data indicates that the own computer is changed to an operating state. (Step K2 in FIG. 18)
If the system configuration notification protocol data indicates that the own computer is to be changed to the operating state (YES in step K2 in FIG. 18), the system configuration setting unit 64 is stored in the system configuration storage unit 61. The system configuration information is updated to the system configuration information included in the protocol data (step K3 in FIG. 18). Then, the system configuration setting unit 64 performs initialization processing, and puts the ordered multicast unit 2 into an operable state (step K4 in FIG. 18).

  On the other hand, if the determination in step K1 or K2 is NO, the system configuration setting unit 64 ends the process. For example, in the system configuration change detection unit 63 of the computer that is requested to change from the operating state to the standby state by the system configuration change request packet, the determination in step K1 is NO. In this case, the system configuration change detection unit 63 ends the process without updating the system configuration information stored in the system configuration storage unit 61 to new system configuration information. Even if it does in this way, since the computer requested | required to change to a standby state is disregarded from the computer in another operating state, it will not pose any problem. The system configuration information may be updated in a computer that is requested to change from the operating state to the standby state.

  The order of the algorithms 1 to 14 described above is not necessarily executed in this order. That is, these are executed independently if the operating condition is satisfied.

  In the present embodiment, the system configuration change detection unit 63 in the system configuration management unit 6 arranges the computers in the latest fixed time (Δt2) among the information stored in the maximum confirmed input sequence number history storage unit 62. Based on information reflecting the processing status of the multicast unit 2, a computer whose maximum confirmed input sequence number is lower than its own computer by Δi (Δi is an integer greater than 1) or more (more specifically, a certain number of times that the number is less than Δi) The above computer is detected as a stagnant computer (algorithm 10). The reason why Δi is an integer larger than 1 is that in the distributed system to which the t-fault tolerance algorithm is applied, the algorithm can execute the following processing if there is an agreement of (nt) computers. That is, when the t-fault tolerance algorithm is applied, in principle, for a certain computer in the distributed system, t computers appear to be in a state where the input sequence number is delayed by one. Therefore, by making Δi an integer greater than 1, erroneous detection of a stagnant computer can be prevented.

  Further, in this embodiment, by setting the time range Δt2 for detecting a stagnant computer, it is possible to prevent a problem that a temporarily stagnating computer is detected as a stagnating computer.

  In the present embodiment, the detection of a stagnant computer is triggered by a failover (algorithm 12) that an agreement is reached between (nt) or more computers. Accordingly, it is possible to prevent the occurrence of split brain by determining that each computer is stagnating with the communication partner computer only because the computers cannot communicate with each other.

  In this embodiment, the system configuration change request packet is used as an input packet for ordered multicast (algorithm 12). As a result, when the computer that has been changed to the standby state because of the stagnation is restored, the process before the change is executed, and the process after the change is not executed.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. For example, in the above-described embodiment, the number n of computers set in the operating state (computers constituting the multiplexing) is 4, and t is 1. However, n may be an integer equal to or greater than 4, that is, four or more computers may be set in the operating state, and t may be an integer greater than 0 that satisfies 3t <n. In this case, the t-fault tolerance algorithm can tolerate a maximum of t faults among the n computers.

  In particular, if {n, t} is set so that the above condition is satisfied and the value of (n−t) is the minimum value, failure of a maximum of t computers out of n computers is allowed. In addition, an agreement can be reached with the smallest number of computers. Here, if the condition 3t <n of t is modified, it can be said that t is an integer larger than 0 that satisfies t <n / 3. From this, the value of (nt) has a relationship such that (nt)> 2n / 3 with respect to n. Therefore, if {n, t} is set so that the above condition is satisfied and the value of (nt) exceeds 2n / 3, that is, {n, t} If it is set so that an agreement of / 3 is obtained, an agreement can be reached with the smallest number of computers. {N, t} at which the value of (nt) exceeds 2n / 3 is {4, 1}, {7, 2}, {10, 3}, {13, 4}. .

  In the above embodiment, the system configuration change detection unit 63 detects the stagnant computer, and the maximum confirmed input sequence number added during the most recent time Δt2 to the maximum confirmed input sequence number history storage unit 62. No particular consideration is given to the number of information (ie the number of samples). However, when the number of samples is equal to or less than a certain number, that is, when the change of the maximum confirmed input sequence number at time Δt2 is equal to or less than a certain number of times, detection of a stagnant computer may be suppressed. In this way, erroneous detection can be prevented by eliminating the case where the number of samples is small for detection of a stagnant computer.

  Further, instead of using the information on the maximum confirmed input sequence number added to the maximum confirmed input sequence number history storage unit 62 during Δt2, information on a certain number of maximum confirmed input sequence numbers may be used. Specifically, the number of information of the changed maximum confirmed input sequence number for the own computer (that is, the number of samples for the own computer) is set as a fixed number, and the other computers corresponding to these maximum confirmed input sequence numbers are determined. Comparing with the maximum confirmed input sequence number, a computer in which the number (that is, the ratio) of which the maximum confirmed input sequence number is lower than the own computer by Δi or more is a certain number or more is detected as a stagnant computer. Good. In this way, erroneous detection can be prevented by setting the number of samples to a fixed number.

  Further, based on the information of the maximum determined input sequence number of each computer added to the maximum determined input sequence number history storage unit 62 during the time Δt2, the maximum determined input sequence number is lower than the own computer by Δi or more, and the number A computer in which the difference between them increases with the passage of time may be detected as a stagnant computer. In this way, a stagnant computer can be detected with high accuracy.

  In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment.

  DESCRIPTION OF SYMBOLS 1 ... Input reception queue part (reception queue), 2 ... Alignment multicast part, 3 ... Application program, 4 ... Program state management part, 5 ... Output filter part, 6 ... System configuration management part, 10 ... Distributed system, 21 ... Input Sequence number storage unit, 22 ... input packet journal storage unit, 23 ... aligned multicast protocol data transmission / reception unit, 24 ... step number storage unit, 25 ... candidate packet storage unit, 26 ... input packet decision determination unit, 27 ... maximum decision input order Number storage unit, 28 ... delay storage unit, 29 ... skip determination unit, 61 ... system configuration storage unit, 62 ... maximum confirmed input sequence number history storage unit, 63 ... system configuration change detection unit, 64 ... system configuration setting unit, 65 ... System configuration change agreement protocol data transmission / reception unit, 66 ... System configuration notification protocol data transmission / reception Part, 100-1~100-5,100-i ... computer, 200 ... client device, 261 ... input candidate collection unit, 262 ... agreement portion.

Claims (6)

By setting n computers (n is an integer satisfying 4 ≦ n <m) out of m computers (m is an integer of 5 or more) connected by a network, and applying the t-fault tolerance algorithm A distributed system with failover function that operates synchronously,
Each of the m computers is
System configuration storage means for storing system configuration information indicating whether each of the m computers is in an operating state to be operated synchronously or in a standby state;
(N−t) computers (n is an integer larger than 0 satisfying 3t <n) among n computers indicated to be in an operating state by the system configuration information stored in the system configuration storage unit. In order to guarantee the multiplexing described above, an agreement means for arranging and multicasting the input data by agreeing the input data with the (nt) computers or more,
Among the n computers, a computer stagnating more than a predetermined level compared to itself is detected based on the execution status of the ordered multicast by the agreement means, and the stagnating computer is detected. In this case, regarding the detection of the stagnant computer, the detection of the stagnant computer is confirmed by agreeing with (nt) or more computers including itself among the n computers. Detection means;
In response to the confirmation of the detection of the stagnant computer, the confirmed stagnant computer is set in the standby state, and the computer in the standby state is stored in the system configuration storage means so as to be in the operating state. And a system configuration setting unit that performs failover from the stagnating computer to the computer in the standby state by updating the system configuration information information. When the detection unit confirms the stagnated computer, the detection unit changes the configuration of the distributed system so that the confirmed stagnant computer is set in a standby state and the computer in the standby state is set in an operating state. Providing the system configuration change request data for requesting the agreement means as the input data to align and multicast the system configuration change request data;
The system configuration setting unit performs a change to the configuration of the distributed system indicated by the system configuration change request data when the system configuration change request data is subjected to ordered multicast by the agreement unit. A distributed system having a failover function according to Item 1. The agreement means manages the input data to be agreed by a sequence number serially attached to the input data,
Each of the m computers stores, for each of the n computers, a sequence number history for storing a history of the maximum sequence numbers attached to input data agreed with other computers in operation. A storage means;
The detection means detects the stagnated computer based on a history of maximum sequence numbers for each of the n computers stored in the sequence number history storage means. Item 3. A distributed system having a failover function according to item 1 or 2.   4. The distributed system having a failover function according to claim 3, wherein the predetermined level is a difference between the sequence numbers greater than 1. When the detection means detects the stagnated computer, the report type protocol data for reporting the detected stagnant computer to all computers in other operating states, other operating System configuration change agreement protocol that transmits agreement type protocol data indicating that the report type protocol data transmitted from the computer in the state agrees, and non-agreement type protocol data indicating that the report is not agreed System configuration change agreement protocol data transmission / reception means for receiving the report type protocol data, the agreement type protocol data and the non-agreement type protocol data transmitted from a computer in another operating state Further comprising means,
The detection means detects the stagnated computer, and the system configuration change agreement protocol data transmission / reception means transmits the report type protocol data to another computer in the operating state, and the report type protocol data. On the other hand, when (n-t-1) or more computers return the protocol data of the consent type, the (n-t) or more computers including itself are related to the detection of the stagnant computer. 5. A distributed system having a failover function according to any one of claims 1 to 4, wherein an agreement is reached. By setting n computers (n is an integer satisfying 4 ≦ n <m) out of m computers (m is an integer of 5 or more) connected by a network, and applying the t-fault tolerance algorithm In a distributed system that operates synchronously, when the operation of any one of the n computers is stagnant, a failover method for failing over from the stagnating computer to a computer in a standby state Because
Each of the m computers includes system configuration storage means for storing system configuration information indicating whether the m computers are in an operation state to be operated synchronously or in a standby state, respectively. An agreement means, a detection means, and a system configuration setting means,
The failover method is:
(N−t) of the n computers in which the agreement means of each of the m computers is shown to be in operation by the system configuration information stored in the system configuration storage means. (T is an integer greater than 0 satisfying 3t <n) In order to guarantee multiplexing at or above, the input data is agreed by the (n−t) or more computers, and the input data is arranged and multicast. Steps,
The detecting means detecting, among the n computers, a computer that has stagnated more than a predetermined level in comparison with itself based on an execution status of the ordered multicast by the agreement means;
When the stagnant computer is detected, the detection means agrees on (nt) or more computers including itself among the n computers with respect to detection of the stagnant computer. Confirming the detection of the stagnant computer;
In response to the confirmation of the detection of the stagnant computer, the confirmed stagnant computer is set in the standby state, and the computer in the standby state is stored in the system configuration storage means so as to be in the operating state. And failover to the computer in the standby state by updating the system configuration information information.

Images