JP5292351B2 - Message queue management system, lock server, message queue management method, and message queue management program - Google Patents

Description

  The present invention relates to a message queue management system, a lock server, a message queue management method, and a message queue management program, and more particularly, in a network computer cluster environment with low reliability using inexpensive hardware, a network failure and a server failure. The present invention relates to a message queue management system, a lock server, a message queue management method, and a message queue management program.

As a first conventional technique, there is a technique for supporting a long-time transaction in an unreliable network computer cluster environment using inexpensive hardware. This technology realizes fault tolerance against a server failure and fault tolerance against a client terminal failure due to a transaction by replicating data in preparation for a master change (for example, see Non-Patent Document 1). )
Further, as a second conventional technique, there is a lock service corresponding to both a network failure and a server failure. This technique is a technique that can cope with any of a network failure, a server failure, and a client failure by performing an operation for locking a file (for example, see Non-Patent Documents 2 and 3).

http://www.ibm.com/software/jp/websphere/integration/wmq/ Mike Burrows, "The Chubby lock service for loosely-coupled distributed systems", 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2006. November 8. (http://research.google.com/archive/chubby-osdi06 .pdf) Tushar Chandra et al., "Paxos Made Live-An Engineering Perspective," PODC'07, 2007. (http://labs.google.com/papers/paxos_made_live.html)

  However, in the first prior art (Non-Patent Document 1), when the network is divided, there is a possibility that a multi-master state in which a plurality of masters exist is generated. Further, since the transaction is rolled back, there is a problem that the task processing takes a long time in the embodiment of the task queue. Paxos (a set of protocols for solving consensus issues on a single outcome in a network of unreliable processors) can be guaranteed that long-running transactions will not lose elements in the queue. In some cases, data consistency cannot be guaranteed. In addition, although the elements in the queue are not lost, there is a problem that the entries in the queue remain in the server permanently if the client fails.

  The second prior art (Non-Patent Documents 2 and 3) is a lock service that can cope with network failures and server failures, and can perform operations such as lock / file processing. When the corresponding distributed lock is a primitive operation, it is not possible to guarantee consistency in the information in the queue when the server fails and fails over only by the combination. For example, in order to operate queue information after locking a file, multiple operations of file locking and reading / writing must be performed for one queue operation (push, pop, etc.) Sufficient performance cannot be expected.

  The present invention has been made in view of the above points. 1) When a client fails, entries in the queue on the server are prevented from remaining permanently. 2) Network failure, server failure, client failure If any of these failures occur simultaneously, as long as a majority of servers can communicate with each other, they can continue to operate as servers and maintain their queue status normally. 3) Between multiple synchronized servers When the network is distributed (Split Brain state), the read queue state should not be inconsistent. 4) Even if some servers in the cluster fail, the queue transaction state is introduced. Message queue management system, lock server, and message queue management method capable of retaining the state in a system An object of the present invention is to provide a message queue management program.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (claim 1), network-in a computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, a large-scale including a plurality of lock server having an operating mode of the Master or Replica In a distributed processing system, the client terminal, the worker, the primary server, and the backup server are the client 1, and the message queue management system includes the client and the plurality of lock servers 300.
Client 1
Means for transmitting a connection request to all master candidates, obtaining a session ID from the master candidate, and performing a queue operation request for the session ID;
Means for transmitting a reconnection request by transmitting the session ID to all master candidates when a message cannot be received within a predetermined time from the master candidate of the session ID transmission source;
The lock server 300
A database 309 storing a queue and information of the queue;
Session information storage means 304 storing session information;
Master adjustment means 301 for determining whether or not the device is a master using Paxos, which is a majority protocol that guarantees that there is always one master,
When the master is changed as a result of the master adjustment unit 301, a valid session is confirmed based on the queue information previously copied in the database 309 and the reconnection request from the client 1, and an intermediate state of the operation on the queue Replication means 307 for requesting data replication to a non-master if the own apparatus becomes the master, and replicating data in the own apparatus in response to a data restoration request from the master if the own apparatus is the master,
A session management unit 303 that transmits a session ID to the client after receiving the connection request from the client 1 and stores the session ID together with the information of the client 1 in the session information storage unit 304;
When the client 1 fails, all the queues corresponding to the session ID are acquired from the session information storage unit 304, and when the session ID exists in the records of all the queues, the session information is left as the session ID. Automatic unmarking means 3031 for updating the storage means.

The present invention (Claim 2) is the message queue management system according to Claim 1,
A deletion waiting session storage means storing session information waiting for deletion;
Further comprising
The replication means 307
If the device itself has not been a master before, the session information before the master change is read from the session information storage unit 304, stored in the session storage unit waiting for deletion, the session information in the session information storage unit is deleted, When there is a session in the session storage unit waiting for deletion after a lapse of time, a session ID is obtained from the session storage unit waiting for deletion, and the session information storage unit is set with the session ID of the session information storage unit blank. First session control means for updating;
In response to the reconnection request from the client 1, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID is A reconnection request processing means to be transmitted to the client, and if the request from the client is other than the connection request and the reconnection request, the lock that operates as a replica performs processing for the queue in the database 309 according to the request content Request processing means for transmitting to the server, and second session control means.

The present invention (Claim 3) is the request processing means of the second session control means of Claim 2,
When the request from the client 1 is a queue push operation request, a push unit that performs a push operation on the queue of the database 309 using the acquired queue ID, key, and key value;
When the request from the client 1 is a queue pop operation request, pop means for performing a pop operation on the queue of the database 309 based on the acquired queue ID;
If the request from the client 1 is a deletion request, a deletion unit that performs a deletion operation on the queue of the database 309 using the acquired queue ID and key;
When the request from the client 1 is a mark request that is a request for restoring the mid-operation state of the queue, the queue ID and the session ID are acquired, and the first record of the queue corresponding to the queue ID is obtained from the database 309. And if the session ID is not set in the record, mark means for setting the acquired session ID;
If the request from the client 1 is an unmark request that is a request that does not restore the mid-operation state of the queue, the queue ID and key are acquired, and the database 309 obtains the key from the record list corresponding to the queue ID. Unmarking means for acquiring a corresponding record and deleting the session ID of the record.

The present invention (claim 4), network-in a computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, a large-scale including a plurality of lock server having an operating mode of the Master or Replica In a distributed processing system, a lock server in a system in which the client terminal, the worker, the primary server, and the backup server are clients,
A queue and a database storing information about the queue;
Session information storage means for storing session information;
Master adjustment means for determining whether or not the device is a master using Paxos, which is a majority protocol that guarantees that there is always one master,
If the master changes as a result of the master adjustment means, the queue information that has been replicated in the database in advance and the valid session are confirmed by the reconnection request from the client, and the intermediate state of the operation on the queue is restored. A replication means for requesting data replication to a non-master if the own device becomes a master, and for replicating data in the own device in response to a data restoration request from the master if the own device is not a master;
Session management means for transmitting a session ID to the client after receiving a connection request from the client, and storing the session ID together with the client information in a session information storage means;
When the client fails, all the queues corresponding to the session ID are acquired from the session information storage means, and when the session ID exists in the records of all the queues, the session information storage means with the session ID as blank And automatic unmarking means for updating.

The present invention (Claim 5) is the lock server according to Claim 4,
A deletion waiting session storage means storing session information waiting for deletion;
Further comprising
Replication means
If the device itself has not been a master before, the session information before the master change is read from the session information storage means, stored in the deletion waiting session storage means, the session information in the session information storage means is deleted, and a predetermined time When there is a session in the session storage unit waiting for deletion after the elapse of time, the session ID is acquired from the session storage unit waiting for deletion and the session information storage unit is updated with the session ID of the session information storage unit blank. First session control means for
In response to the reconnection request from the client, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID is added to the client If the request from the client and the request from the client are other than the connection request and the reconnection request, processing is performed on the queue in the database according to the request content, and the lock server operating as a replica And a second session control means including a request processing means for transmission.

Moreover, this invention (Claim 6) is the lock server of Claim 5,
The request processing means of the second session control means is:
If the request from the client is a queue push operation request, a push means for performing a push operation on the database queue using the acquired queue ID, key, and key value;
If the request from the client is a queue pop operation request, pop means for performing a pop operation on the database queue based on the acquired queue ID;
If the request from the client is a deletion request, a deletion unit that performs a deletion operation on the database queue using the acquired queue ID and key;
When the request from the client is a mark request that is a request for restoring the mid-operation state of the queue, the queue ID and the session ID are obtained, and the first record of the queue corresponding to the queue ID is obtained from the database. If the session ID is not set in the record, a mark means for setting the acquired session ID;
When the request from the client is an unmark request that is a request that does not restore the intermediate state with respect to the queue, the queue ID and key are obtained, and the record corresponding to the key is retrieved from the record list corresponding to the queue ID from the database. And unmarking means for acquiring and deleting the session ID of the record.

The present invention (claim 7), network-in a computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, a large-scale including a plurality of lock server having an operating mode of the Master or Replica In a distributed processing system, a message queue management method in which the client terminal, the worker, the primary server, and the backup server are clients, and any one of the plurality of lock servers is a master,
The client
Sending connection requests to all master candidate lock servers, obtaining a session ID from the master candidate, and performing a queue operation request for the session ID;
If a message cannot be received within a predetermined time from the master candidate of the session ID transmission source, a step of transmitting a reconnection request by transmitting the session ID to all master candidates is performed.
The lock server
A master adjustment step of determining whether or not the own device is a master using Paxos, which is a majority protocol that guarantees that the master is always one;
If the master is changed as a result of the master adjustment step, the queue information that has been duplicated in the database in advance and the valid session are confirmed by the reconnection request from the client, and the intermediate state of the operation on the queue is restored. A replication step of requesting data replication to a non-master if the own device becomes a master, and replicating data in the own device in response to a data restoration request from the master if the own device is not the master;
A session management step of transmitting a session ID to the client after receiving a connection request from the client, and storing the session ID together with the information of the client in session information storage means;
When the client fails, all the queues corresponding to the session ID are acquired from the session information storage means, and when the session ID exists in the records of all the queues, the session information storage means with the session ID as blank And an automatic unmarking step for updating.

Further, the present invention (Claim 8) is a replication step according to Claim 7,
If your device was not the master before,
Reads the session information before the master change from the session information storage means storing the session information, stores the session information waiting for deletion in the deletion waiting session storage means, deletes the session information in the session information storage means, If there is a session in the session storage unit waiting for deletion after the elapse of time, the session ID is acquired from the session storage unit waiting for deletion, and the middle of the queue held by the client corresponding to the session ID in the database A first session control step of invalidating the state;
In response to the reconnection request from the client, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID is added to the client To
If the request from the client is other than the connection request and the reconnection request, a second session control step of processing the queue in the database according to the request content and transmitting to the other lock server is included.

The present invention (Claim 9) is characterized in that, in the second session control step of Claim 8,
When the request from the client is a queue push operation request, a push operation is performed on the database queue using the acquired queue ID, key, and key value.
If the request from the client is a queue pop operation request, perform a pop operation on the database queue based on the acquired queue ID,
If the request from the client is a delete request, perform a delete operation on the database queue using the acquired queue ID and key,
When the request from the client is a mark request that is a request for restoring the mid-operation state of the queue, the queue ID and the session ID are obtained, and the first record of the queue corresponding to the queue ID is obtained from the database. If the session ID is not set in the record, a mark operation for setting the acquired session ID is performed.
When the request from the client is an unmark request that is a request that does not restore the intermediate state with respect to the queue, the queue ID and key are obtained, and the record corresponding to the key is retrieved from the record list corresponding to the queue ID from the database. Obtaining and performing an unmark operation for deleting the session ID of the record.

  The present invention (Claim 10) is a message queue management program for causing a computer to function as each means constituting the lock server according to any one of Claims 4 to 6.

  As described above, according to the present invention, in a network computer cluster environment with low reliability using inexpensive hardware, a multi-master is selected by selecting a unique master by majority from predetermined candidates. Avoid getting into a state.

  In addition, even if the master fails due to replication and failover, the operation is continued just by taking over the role of the new master. Therefore, even if some servers in the cluster fail, the queue transaction state is introduced. In a system where the information in the queue is inconsistent.

  Even when network partitioning occurs between a plurality of synchronized servers, it is possible to prevent a problem that the read queue states do not match. That is, it is possible to prevent a problem in which the information in the queue is inconsistent when the server holding the master queue fails and fails over.

  When a client fails, it is possible to prevent the entries in the queue on the master from remaining permanently.

  In addition, even if a network failure, server failure, or client failure occurs at the same time, as long as the majority of servers can communicate with each other, they continue to operate as servers and maintain the queue status normally. can do.

  Even when some servers in the cluster fail, the state can be maintained in a system in which the transaction state of the queue is introduced. That is, when applied to a task queue, the processing efficiency of the system is improved by preventing task re-execution.

It is a principle block diagram of this invention. 1 is a system configuration diagram to which the present invention is applied. It is a system configuration figure of the principal part in one embodiment of the present invention. It is an example of the table of the session list memory | storage part in one embodiment of this invention. It is an example of the table of the deletion waiting session list memory | storage part in one embodiment of this invention. It is an example of the table of DB in one embodiment of this invention. It is a flowchart of operation | movement of the client in one embodiment of this invention. It is a flowchart of the lock server in one embodiment of the present invention. It is a flow chart of session restoration processing (S1060) with a client in one embodiment of the present invention. It is a flowchart of the request reception start process (S1070) from the client in one embodiment of this invention. It is a flowchart of push operation of the queue control part in one embodiment of this invention. It is a flowchart of the pop operation of the queue control unit in one embodiment of the present invention. It is a flowchart of delete operation of the queue control part in one embodiment of the present invention. It is a flowchart of mark operation of the queue control part in one embodiment of this invention. It is a flowchart of unmark operation in one embodiment of this invention. It is a flowchart of the request | requirement reception | blocking process (S1100, S2130) from the client in one embodiment of this invention. It is a flowchart of the session deletion process (S5070, S2170) in one embodiment of this invention. It is a flowchart of automatic unmark operation (S4020) in one embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows a system configuration to which the present invention is applied.

  As shown in FIG. 2, the present invention provides a primary server 100 that performs data management and control of a distributed file system, a backup server 200, a master that provides basic functions for increasing the availability and consistency of the distributed system, or A lock server 300 having an operation mode as a replica, a plurality of client terminals 10 having client terminal libraries linked to an application using a distributed file system of a distributed system, and a plurality of workers 20 that physically hold data. This technology is applied to a large-scale distributed processing system.

  The primary server 100 acquires a file meta-information acquisition / update request from the client terminal 10, copies the updated meta-information and transmits it to the backup server 200, and manages data for the worker 20 based on the meta-information. Take control. Further, the state of the distributed file master is stored in the lock server master 300.

  The client terminal 10 acquires the location information of the primary server 100 from the lock server 300, acquires the meta information of the file from the primary server 100, and the meta information of the worker 20 that manages the data passed from the primary server 100. Based on the above, data is read or data is written.

  The worker 20 registers and provides data by being accessed from the client terminal 10.

  The lock server (master) 300 manages the distributed file master status and the location information of the distributed master, receives lock acquisition requests from the primary server 100 and the backup server 200, performs exclusive control between the servers, and monitors the alive state. For example, when a failure occurs in the primary server 100, the alive monitoring file of the primary server 100 is deleted. The lock server (replica) 300 holds the copied information of the lock server 300 that is the master.

FIG. 3 shows the system configuration of the main part in an embodiment of the present invention. In the figure, the configuration of the lock server 300 is shown as the main part in the present embodiment.
The “client” shown in the figure refers to a communication partner viewed from the lock server 300, and includes not only the client terminal 10 shown in FIG. 2 but also any of the primary server 100, backup server 200, and worker 20 of the distributed file master. Shall point to.

  The lock server 300 shown in the figure includes a master arbitration unit 301, a master management unit 302, a session management unit 303, a session list storage unit 304, a deletion waiting session list storage unit 305, a request processing unit 306, a replication management unit 307, a database ( DB) 309 and a transaction management unit 310.

  The session list storage unit 304 and the deletion waiting session list storage unit 305 are storage media such as a memory or a hard disk.

  The session list storage unit 304 and the deletion waiting session list storage unit 305 are tables that are referred to and updated from the session management unit 303. As shown in FIGS. 4 and 5, client information such as a session ID and an IP address, And a table of queue entries that are marked. The table in the deletion waiting session list storage unit 305 is held only immediately after the master change.

  As shown in FIG. 6, the DB 309 has a table of a queue list (FIG. (A)) and a record list for each queue ID (FIG. (B)).

  First, the operation of the client 1 will be described.

  FIG. 7 is a flowchart of the operation of the client according to the embodiment of the present invention.

  Step 101) The client 1 transmits a connection request to all master candidates (the lock server 300).

  Step 102) The session ID is received from the lock server 300 operating as the master.

  Step 103) A reception confirmation is transmitted to the master of the session ID transmission source.

  Step 104) The alive monitoring timer is set.

  Step 105) It is determined whether a response has been received from the master candidate lock server 300 before the alive monitoring timer times out. If received, the process proceeds to Step 103. If not received, the process proceeds to Step 106. To do.

  Step 106) Set the reconnection timer.

  Step 107) A reconnection request including the session ID is transmitted to all master candidate lock servers 300.

  Step 108) It is determined whether a reply is obtained from the lock server 300 operating as the master before the reconnection timer times out. If it is obtained, the process proceeds to Step 103, and if not, the process is terminated. To do.

  FIG. 8 is a flowchart of the lock server according to the embodiment of the present invention.

  The processing shown in the figure includes master selection processing by Paxos, data redundancy by replication and failover, and processing for a loss prevention queue.

  Step 1010) When the operator activates the system, the master arbitration unit 301 performs a master selection process by Paxos. For example, Reference 1 “Leslie Lamport,“ Paxos Made Simpke, ”ACM SIGACT News (Distributed Computing Column) 32, 4 (Whole Number 121, December 2001) 51-58”, Reference 2 “Cary G. Gray and Davide R. Cheriton, “Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Coherency,” Procs of 12th ACM SOSP '89, 202-210, 1989 ” adopt.

  In this method, a unique master is selected by a majority vote from among the lock servers 300 of predetermined master candidates on the system. Specifically, servers that cannot be synchronized with the master are stopped. This is not a multi-master state. This is because Paxos ensures that the majority decision (= new master) converges to one value for one agenda item.

  Step 1020) The master management unit 302 determines whether the server 300 has been elected as a master by the processing of Step 1010 described above. If so, the process proceeds to Step 1030. If not, the process proceeds to Step 1110. .

  Step 1030) Set the master lease timer. Here, the “master lease” refers to a time limit for continuing the master state.

  Step 1040) A timer for selecting the next master is set.

  Step 1050) The master management unit 302 proceeds to step 1070 when the own device has been the master and the previous master lease timer has not timed out, and is not the master computer before or the previous master If the lease timer has timed out, that is, if the master has changed, the process proceeds to step 1060.

  Step 1060) The session management unit 303 restores the session with the client 1, and proceeds to Step 1070. This operation is a process when the master is changed, and is a process necessary for starting the operation as a master when the lock server 300 becomes a master from a state that is not a master. This detailed operation is shown in FIG.

  FIG. 9 is a flowchart of a session restoration process with a client terminal according to an embodiment of the present invention.

    Step 5010) When restoring the session with the client 1, the session management unit 303 reads the session information before the master change from the session information storage unit 304 shown in FIG. Write to the storage unit 305.

    Step 5020) The session management unit 303 deletes the previous information in the table of the session list storage unit 304 of FIG.

    Step 5030) The session management unit 303 sets a session deletion timer.

    Step 5040) If the session deletion timer times out, the session management unit 303 proceeds to step 5050.

    Step 5050) The session management unit 303 ends the processing if the table in the deletion waiting session list storage unit 305 is empty, and proceeds to step 5060 if the table is not empty.

    Step 5060) The session management unit 303 selects any one session in the deletion waiting session list storage unit 305.

    Step 5070) The session management unit 303 deletes the selected session. Specifically, it will be described in detail with reference to FIG.

  Step 1070) The request processing unit 306 starts accepting a request from the client 1 when the processing of the above steps 5010 to 5070 is completed.

  Hereinafter, a process for accepting a request from the client 1 will be described in detail with reference to FIG. FIG. 10 is a flowchart of a request reception start process from a client according to an embodiment of the present invention.

    Step 2020) The request processing unit 306 proceeds to Step 2030 when there is a request from the client 1, and waits when there is no request.

    Step 2030) Upon obtaining the request from the client 1, the request processing unit 306 determines whether or not the request is a (first) connection request. If the request is a connection request, the request processing unit 306 proceeds to Step 2040; Goes to Step 2070.

    Step 2040) The session management unit 303 issues a session ID and additionally stores it in the session list storage unit 304.

    Step 2050) The request processing unit 306 notifies the client 1 of the session ID issued by the session management unit 303.

    Step 2060) Start the client monitoring process and go to Step 2140.

    Step 2070) In step 2030, if the acquired request is a reconnection request, the process proceeds to step 2080. Otherwise, the process proceeds to step 2100.

    Step 2080) When the received request is a reconnection request, the session management unit 303 deletes the session ID corresponding to the request from the deletion waiting session list storage unit 305.

    Step 2090) The session management unit 303 additionally stores the session ID in the session list storage unit 304, and proceeds to step 2050.

    Step 2100) When the request processing unit 306 determines in Step 2070 that the request from the client 1 is a request other than the reconnection request, the queue control unit 308 of the replication management unit 307 determines whether the request is a request. , Processing for the queue in the DB 309 shown in FIG. 6 is executed. Various processes for the queue will be described with reference to FIGS.

    Step 2140) When the client monitoring process is started, after waiting for a predetermined time, a ping command (command for node reachability confirmation) is executed to the client 1.

    Step 2150) Set the timer.

    Step 2160) If there is a reply to the ping command from the client 1 before the timeout, the procedure goes to Step 2140, and if there is no reply, the procedure goes to Step 2170.

    Step 2170) Session deletion processing is performed. Details will be described later with reference to FIG.

<Push operation>
FIG. 11 shows a push operation to the queue by the queue control unit 308.

      Step 11010) When the queue ID, key (K), and value (V) are input as a push operation request from the client 1, the queue control unit 307 lists the queues in the DB 309 based on the queue ID (FIG. 6A). ) To get the last record (Last) in the queue.

      Step 11020) A record is created using the input key (K) and input value (V) shown below, and is inserted into the record list (FIG. 6B) in association with the queue ID. Also, the last record is updated using the key next to the last key in the queue list (FIG. 6A) as an input key.

      Step 11030) The last key of the queue in the queue list in DB 309 (FIG. 6A) is updated with the input key (K).

<Pop operation>
Next, the pop operation of the queue control unit 307 is shown in FIG.

      Step 12010) When a queue ID is input as a pop operation request from the client 1, the queue control unit 308 uses the queue ID to search from the queue list in the DB 309 (FIG. 6A) for the first record (Front) of the queue. To get.

      Step 12020) The head record (Front) is deleted from the queue list of DB 309 (FIG. 6A).

      Step 12030) The head key of the queue corresponding to the queue ID on the queue list (FIG. 6A) is updated with the key “Front.Next key” next to the head key.

<Delete operation>
Next, the delete operation of the queue control unit 308 is shown in FIG.

      Step 13010) When the queue ID and the key (K) are input as a delete operation request from the client 1, the queue control unit 308 determines the key corresponding to the queue ID from the record list in the DB 309 (FIG. 6B). Record C is acquired.

      Step 13020) Based on the previous key “Prev” and next key “Next” of record C, record B and record A are acquired.

      Step 13030) Update the next key of record B with the key of record A, update the next key of record A with the key of record B, and delete record C.

<Mark operation>
Next, the mark operation of the queue control unit 308 is shown in FIG. Here, the “mark operation” refers to restoration of an intermediate state for the queue.

      Step 14010) When the session ID and the queue ID are input from the client 1 as a mark operation request from the client 1, the queue control unit 308 acquires the first record of the queue from the queue list of the DB 309 (FIG. 6A), To do.

      Step 14020) If the session ID of the acquired current record is blank, the process proceeds to step 14040. If the session ID is not blank, the process proceeds to step 14030.

      Step 14030) Based on the next key of the current record, the next record is obtained from the record list of the queue (FIG. 6B), made the current record, and the process proceeds to Step 14020. If the next key is blank, it is determined to be a failure and the process is terminated.

  Step 14040) The session ID of the current record is updated with the input session ID.

<Unmark operation>
Next, the unmark operation of the queue control unit 308 is shown in FIG. Here, the “unmark operation” is to solve the problem that entries in the queue remain permanently when the client 1 fails without restoring the intermediate state of the queue.

      Step 15010) When the key and the queue ID are input from the client 1 as the unmark operation request from the client 1, the queue control unit 308 obtains the record corresponding to the input key from the record list corresponding to the queue ID (FIG. 6B). To do.

      Step 15020) The session ID of the record is updated to a blank.

    Step 2110) The replication management unit 307 transmits the processing result of any one of the above push operation, pop operation, delete operation, mark operation, and unmark operation to the replica 30.

    Step 2120) If a reply is obtained from the replica 30, the process proceeds to step 2020, and if not, the process proceeds to step 2130.

    Step 2130) The request processing unit 306 blocks reception of the request from the client 1. Details of this step will be described in detail with reference to FIG.

  Step 1080) It is determined whether or not the next master selection timer has timed out. If timed out, the process proceeds to Step 1090, and if not timed out, the process is repeated.

  Step 1090) It is determined whether the master lease has timed out. If timed out, the process proceeds to Step 1100, and if not timed out, the process is repeated.

  Step 1100) The request processing unit 306 blocks the request reception from the client 1. This process will be described with reference to FIG.

    Step 3010) The process for accepting requests from the client 1 is stopped.

    Step 3020) Stop all client terminal monitoring processes.

    Step 3030) All the session information is deleted from the session list storage unit 305.

  Step 1110) Next, in the above-described Step 1020, when the device itself is not elected as the master, the master lease timer is set.

  Step 1120) Start receiving a replication request from the master. This process is described in Reference 3 “Jim Gray et al.,“ The Recovery Manager of the System R Database Manager ”, Computing Surveys, Vol. 13, No. 2, 1981.” and Reference 4 “David K. Gifford,“ Weighted ”. Voting for Replicated Data ", Proc of SIGOPS, pp. 150-162, 1979."

  Step 1130) If the master lease timer times out, go to Step 1140.

  Step 1140) The reception of the replication request from the master is blocked, and the process proceeds to Step 1010. This processing is the same as that described in Documents 3 and 4 above.

  Next, the processing for blocking the reception of requests from clients in step 1100 in FIG. 8 and 2130 in FIG. 10 will be described.

  FIG. 16 is a flowchart of a process for blocking acceptance of requests from clients (S1100, S2130) according to an embodiment of the present invention.

  Step 3010) The request processing unit 306 stops the request reception process from the client.

  Step 3020) Stop all client monitoring processes.

  Step 3030) Delete all session information.

  Next, the session deletion process in step 5070 of FIG. 9 and step 2170 of FIG. 10 will be described.

  FIG. 17 is a flowchart of session deletion processing (S5070, S2170) according to an embodiment of the present invention.

  Step 4010) The session management unit 303 acquires all the queues related to the session ID from the session list in the session list storage unit 304.

  Step 4020) The session management unit 303 performs an automatic unmark operation on all the acquired queues as a loss prevention queue operation by the process shown in FIG.

  FIG. 18 is a flowchart of the automatic unmark operation (S4020) according to the embodiment of the present invention.

  Step 16010) The session management unit 303 acquires the first record in the queue from the session list storage unit 304 based on the session ID acquired in step 4010.

  Step 16020) If the session ID of the record is the session ID acquired in step 4010, the process proceeds to step 16030. Otherwise, the process proceeds to step 16040.

  Step 16030) The session ID of the acquired record is null (blank).

  Step 16040) If the next record exists in the session list of the session list storage unit 304, the process proceeds to step 16050, and if not, the process ends.

  Step 16050) The record next to the current record is acquired from the session list storage unit 304, and the process proceeds to Step 16020.

  In the series of flowcharts described above, in steps 1010 to 1100 in FIG. 8, it is guaranteed that there is always one master by performing master selection using the majority vote protocol Paxos. This ensures that the queue data structure managed by the system is not updated by multiple masters even if the network breaks down due to a network failure (Split Brain state). Can be maintained.

  Further, even when the master no longer exists due to a master failure, the function of selecting the next master in place of this master is included, so that the new master can take over the operation.

  When the master takes over the operation, the processing shown in step 1070 in FIG. 8 and the flowchart in FIG. 10 is performed. Here, when the server becomes a master from a state where it is not the master, processing necessary to start the operation as the master is performed.

  The new lock server 300 becomes effective by the reconnection process (step 107) from the client shown in FIGS. 4, 5, 6 and 7 that has been restored in advance in steps 2100 to 2120 of FIG. Check the session. As a result, the data in the queue (marked in FIG. 6B) is valid (= marked, but the marked session does not exist). Record) and recover from the server failure state and network failure state.

  At this time, as in Non-Patent Document 1 described above, if a transaction is used to indicate that a data change is in progress, the state in the middle of the transaction must be restored (replicated). However, it is difficult to improve performance if the intermediate state of the transaction (all lock states for various data) is restored for general purposes. Therefore, in the present invention, as shown in FIG. 6B, FIG. 14 and FIG. 15, the intermediate state of the transaction is defined as “mark state” and is limited to one entry in the queue. State restoration was made easier. As a result, it is possible to restore the mid-operation state in the queue. Details of FIGS. 6B, 14 and 15 are as shown in steps 14010 to 14040 and steps 1510 and 1520 of the above flowchart.

  Furthermore, by performing the processing shown in steps 2140 to 2170, FIG. 17 and FIG. 18 in FIG. 10, when the client 1 fails and the session ID remains in the record of the session list, the processing is automatically performed. By performing an automatic unmark operation that sets the session ID to blank (null), it is possible to prevent the entries in the queue on the lock server from remaining, and in the system in which the transaction state of the queue is introduced, Can hold.

  The process of restoring (replicating) data in preparation for the master change is shown in steps 2070 to 2080 of FIG. 10 in the above flowchart. The logical processing here is as shown in FIGS. Behind the processing of FIG. 11 to FIG. 15, the data is made permanent using the same algorithm as that of Reference 3, and the data is restored using the same algorithm as that of Reference 4. Thereby, the fault tolerance to the master change resulting from the server failure and the network failure is secured.

  Only when the session is in a normal state (steps 103 to 105 shown in FIG. 7), the client 1 transmits a queue operation request of push, pop, delete, mark, or unmark to the master. Based on the operation request 1, the queue operations in steps 2100 to 2120 shown in FIG. 10 are performed. The queue operation is defined by the data structure of FIG. 6B and FIGS.

In this way, the logical data structure of the queue is realized by associating one record in the key-value format with one entry of the queue for the queue. Note that the processing of the lock server 300 is performed as a program. It can be constructed and installed on these computers for execution, or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

1 Client 10 Client Terminal 20 Worker 100 Primary Server 200 Backup Server 300 Lock Server 301 Master Adjustment Unit 302 Master Management Unit 303 Session Management Unit 304 Session List Storage Unit 305 Delete Waiting Session List Storage Unit 306 Request Processing Unit 307 Replication Unit, Replication Unit 309 database

Claims (10)

In network computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, in a large distributed processing system including a plurality of lock server having an operating mode of the Master or replica, said client terminal, A message queue management system comprising the worker, the primary server, and the backup server as a client, the client and the plurality of lock servers,
The client
Means for transmitting a connection request to all master candidates, obtaining a session ID from the master candidate, and performing a queue operation request for the session ID;
Means for transmitting a reconnection request by transmitting the session ID to all master candidates when a message cannot be received within a predetermined time from the master candidate of the session ID transmission source;
The lock server is
A queue and a database storing information about the queue;
Session information storage means for storing session information;
Master adjustment means for determining whether or not the device is a master using Paxos, which is a majority protocol that guarantees that there is always one master,
If the master is changed as a result of the master adjustment means, the queue information that has been replicated in the database in advance and the reconnection request from the client are checked for a valid session, and the queue is in the middle of operation. Replication means for requesting data replication to a non-master if the own device becomes the master, and replicating data in the own device in response to a data restoration request from the master if the own device is the master,
Session management means for transmitting a session ID to the client after receiving the connection request from the client, and storing the session ID together with the client information in the session information storage means;
When the client fails, all the queues corresponding to the session ID are acquired from the session information storage unit, and when the session ID is present in the records of all the queues, the session information is left blank as the session ID. Automatic unmarking means for updating the storage means;
A message queue management system comprising: A deletion waiting session storage means storing session information waiting for deletion;
Further comprising
The replication means includes
If the device itself was not previously a master, read the session information before the master change from the session information storage means, store it in the deletion waiting session storage means, delete the session information of the session information storage means, If there is a session in the deletion waiting session storage means after a predetermined time has elapsed, a session ID is acquired from the deletion waiting session storage means, and the session information storage means stores the session ID as a blank. First session control means for updating the means;
In response to the reconnection request from the client, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID is If the request from the reconnection request processing means to be transmitted to the client and the request from the client is other than the connection request and the reconnection request, the queue is processed according to the request content and operates as a replica Second session control means including request processing means for transmitting to the lock server;
The message queue management system according to claim 1, comprising: The request processing means of the second session control means is:
If the request from the client is a queue push operation request, push means for performing a push operation on the queue of the database using the acquired queue ID, key, and key value;
If the request from the client is a queue pop operation request, pop means for performing a pop operation on the database queue based on the acquired queue ID;
If the request from the client is a deletion request, a deletion unit that performs a deletion operation on the queue of the database using the acquired queue ID and key;
When the request from the client is a mark request that is a request for restoring an intermediate state of an operation on the queue, a queue ID and a session ID are acquired, and the first record of the queue corresponding to the queue ID is obtained from the database. And if the session ID is not set in the record, mark means for setting the acquired session ID;
When the request from the client is an unmark request that is a request that does not restore the mid-operation state of the queue, the queue ID and key are acquired, and the key is obtained from the record list corresponding to the queue ID from the database. Unmarking means for acquiring a corresponding record and deleting the session ID of the record;
The message queue management system according to claim 2, comprising: In network computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, in a large distributed processing system including a plurality of lock server having an operating mode of the Master or replica, said client terminal, A lock server in a system in which the worker, the primary server, and the backup server are clients,
A queue and a database storing information about the queue;
Session information storage means for storing session information;
Master adjustment means for determining whether or not the device is a master using Paxos, which is a majority protocol that guarantees that there is always one master,
If the master is changed as a result of the master adjustment means, the queue information that has been replicated in the database in advance and the reconnection request from the client are checked for a valid session, and the queue is in the middle of operation. Replication means for requesting data replication to a non-master if the own device becomes the master, and replicating data in the own device in response to a data restoration request from the master if the own device is the master,
Session management means for transmitting a session ID to the client after receiving a connection request from the client and storing the session ID together with the client information in the session information storage means;
When the client fails, all the queues corresponding to the session ID are acquired from the session information storage unit, and when the session ID is present in the records of all the queues, the session information is left blank as the session ID. Automatic unmarking means for updating the storage means;
A lock server characterized by comprising: A deletion waiting session storage means storing session information waiting for deletion;
Further comprising
The replication means includes
If the device itself was not previously a master, read the session information before the master change from the session information storage means, store it in the deletion waiting session storage means, delete the session information of the session information storage means, If there is a session in the deletion waiting session storage means after a predetermined time has elapsed, a session ID is acquired from the deletion waiting session storage means, and the session information storage means stores the session ID as a blank. First session control means for updating the means;
In response to the reconnection request from the client, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID A reconnection request processing means for transmitting to the client, and if the request from the client is other than the connection request and the reconnection request, it performs processing for the queue in the database according to the request content and operates as a replica Second session control means including request processing means for transmitting to the lock server
The lock server according to claim 4, including: The request processing means of the second session control means is:
If the request from the client is a queue push operation request, push means for performing a push operation on the queue of the database using the acquired queue ID, key, and key value;
If the request from the client is a queue pop operation request, pop means for performing a pop operation on the database queue based on the acquired queue ID;
If the request from the client is a deletion request, a deletion unit that performs a deletion operation on the queue of the database using the acquired queue ID and key;
When the request from the client is a mark request that is a request for restoring an intermediate state of an operation on the queue, a queue ID and a session ID are acquired, and the first record of the queue corresponding to the queue ID is obtained from the database. And if the session ID is not set in the record, mark means for setting the acquired session ID;
When the request from the client is an unmark request that is a request that does not restore an intermediate state with respect to the queue, the queue ID and key are obtained, and the corresponding key is obtained from the record list corresponding to the queue ID from the database. Unmarking means for acquiring a record and deleting the session ID of the record;
The lock server according to claim 5, comprising: In network computer cluster environment, a plurality of client terminals, a plurality of workers, the primary server, a backup server, in a large distributed processing system including a plurality of lock server having an operating mode of the Master or replica, said client terminal, A message queue management method in which the worker, the primary server, and the backup server are clients, and any one of the plurality of lock servers is a master,
The client
Transmitting a connection request to all the master candidates as the lock server, obtaining a session ID from the master candidate, and performing a queue operation request for the session ID;
If a message cannot be received within a predetermined time from the master candidate of the session ID transmission source, a step of transmitting a reconnection request by transmitting the session ID to all master candidates, and
The lock server is
A master adjustment step of determining whether or not the own device is a master using Paxos, which is a majority protocol that guarantees that the master is always one;
As a result of the master adjustment step, when the master is changed, a valid session is confirmed by information on the queue that has been replicated in the database in advance and a reconnection request from the client, and an intermediate state of the operation on the queue is confirmed. A replication step for requesting data replication to a non-master if the local device becomes a master, and replicating data in the local device in response to a data recovery request from the master if the local device is not the master;
A session management step of transmitting a session ID to the client after receiving a connection request from the client, and storing the session ID together with the client information in the session information storage means;
When the client fails, all the queues corresponding to the session ID are acquired from the session information storage unit, and when the session ID is present in the records of all the queues, the session information is left blank as the session ID. An automatic unmark step for updating the storage means;
A message queue management method characterized by: In the replication step,
If the device is not the master before,
Reads the session information before the master change from the session information storage means storing the session information, stores the session information waiting for deletion in the deletion waiting session storage means, deletes the session information in the session information storage means, If there is a session in the deletion-waiting session storage means after the elapse of time, a session ID is obtained from the deletion-waiting session storage means, and the queue of the queue held by the client corresponding to the session ID in the database is acquired. A first session control step for invalidating the intermediate state;
In response to the reconnection request from the client, the session ID corresponding to the reconnection request is deleted from the deletion waiting session information storage means, the session ID is added to the session information storage means, and the session ID is Send to the client,
If the request from the client is other than the connection request and the reconnection request, a second session control step of processing the queue in the database according to the request content and transmitting to the other lock server;
The message queue management method according to claim 7, comprising: In the second session control step,
When the request from the client is a queue push operation request, a push operation is performed on the queue of the database using the acquired queue ID, key, and key value,
If the request from the client is a queue pop operation request, perform a pop operation on the database queue based on the acquired queue ID;
If the request from the client is a delete request, perform a delete operation on the queue of the database using the acquired queue ID and key,
When the request from the client is a mark request that is a request for restoring an intermediate state of an operation on the queue, a queue ID and a session ID are acquired, and the first record of the queue corresponding to the queue ID is obtained from the database. If the session ID is not set in the record, a mark operation is performed to set the acquired session ID.
When the request from the client is an unmark request that is a request that does not restore an intermediate state with respect to the queue, the queue ID and key are obtained, and the corresponding key is obtained from the record list corresponding to the queue ID from the database. Obtain a record and perform an unmark operation to delete the session ID of the record.
The message queue management method according to claim 8.   A message queue management program for causing a computer to function as each means constituting the lock server according to any one of claims 4 to 6.

Images