WO2012042607A1

WO2012042607A1 - Distributed computing system

Info

Publication number: WO2012042607A1
Application number: PCT/JP2010/066906
Authority: WO
Inventors: 渡邊典孝
Original assignee: 株式会社トライテック
Priority date: 2010-09-29
Filing date: 2010-09-29
Publication date: 2012-04-05
Also published as: JPWO2012042607A1

Abstract

[Problem] To provide a distributed computer system which can prevent service provision being suspended due to the incidence of faults in the connection window with a cell client. [Solution] In this distributed computing system (1), a control unit (6) in a server (4) constituting a cell (2) receives a processing request from a control unit (5) of a client (3), and notifies another server (4) that there has been a processing request and of a client ID indentifying said client (3), and also receives a processing request from the control unit (5) of the client (3), or, receives a notification that there has been a processing request from the control unit (6) of the other server (4) which received a processing request, and then initiates a session with the client (3) on the basis of the client ID, and carries out the processing for the same.

Description

Distributed computing system

The present invention relates to a distributed computing system including a cell and a client including a plurality of replicas that are synchronized with each other, and more particularly to a session management technique between the cell and the client.

For example, the following replication technology is known for a distributed computing system including a cell including a plurality of replicas and a client.

In a cluster system using a load balancer (load balancer), the virtual IP address of the load balancer serves as a connection window with the client (see Patent Documents 1 and 2), and requests from the client are transmitted to the replica via the load balancer. The response from the replica is sent to the client via the load balancer.

In a highly available cluster system, the production system (primary system) and standby system (standby system) replicas are provided (see Patent Documents 3 and 4), and the production system replica that provided services to the client fails. For example, the standby replica in the hot standby state takes over the processing of the production replica and continues to provide the service.

In the virtual machine, hardware level virtualization, that is, by constructing a plurality of virtual hardware in one piece of hardware and making each one function as a replica (see Patent Document 5). The same availability as the above highly available cluster system is realized.

RAID devices provide a virtualized interface for disks, and store data multiplexed and redundant by mirroring or pooling.

JP 2010-86145 A JP-A-9-218842 JP 2009-265690 A JP-A-10-105424 JP 2010-33280 A

In each of the above replication technologies, the replica group can be seen as one virtual system from the client by the virtualization technology, and the distributed cooperative method between replicas is made into a black box. In other words, there is one connection window between the client and the group of replicas, and if a failure occurs at this connection window, the client loses the access method to the replica inside the black box. Even if the service is provided, the service provision by the cell stops.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a distributed computing system that can prevent service stoppage due to the occurrence of a failure at a connection window with a cell client. .

In order to solve the above-mentioned problem, the invention according to claim 1 includes a cell including a plurality of replicas synchronized with each other and a client, and manages a session between the cell and the client in the client. First connection means for storing connection information for the client to connect to each replica and a client ID for identifying the client as session information for the connection to any one of the plurality of replicas Processing request means for performing a predetermined processing request and notification of the client ID based on the information, and receiving a processing request from the processing request means to each of the replicas, and indicating that there has been a processing request and the client ID Process request notifying means for notifying other replicas, and receiving a process request from the process request means, A processing execution unit that receives a notification from the processing request notification unit of the replica that has received the processing request and establishes the session with the client based on the client ID and performs the predetermined processing And a distributed computing system.

The invention according to claim 2 is the distributed computing system according to claim 1, wherein the predetermined process is an update process.

The invention according to claim 3 is the distributed computing system according to claim 1, wherein the predetermined processing request is a session opening request for causing each replica to open the session.

According to a fourth aspect of the present invention, in the distributed computing system according to any one of the first to third aspects, when the first storage means selects one of the plurality of replicas as a master. , Storing a master ID for specifying which replica is the master, and the processing requesting unit performs the predetermined processing on the replica specified by the master ID stored in the first storage unit. And a notification of the client ID.

The invention according to claim 5 is the distributed computing system according to claim 4, wherein the first storage means is selected when one of the plurality of replicas is selected as a reference to be used for reference processing. A reference ID for identifying whether the replica of the first reference is a reference is stored, and the processing requesting unit requests a reference processing for the replica specified by the reference ID stored in the first storage unit. It is characterized by that.

According to a sixth aspect of the present invention, in the distributed computing system according to the fourth aspect of the present invention, in each of the replicas, a response generation unit that generates a response to the request for the predetermined process, and the master ID in the first storage Second storage means for storing the response generated independently of the means and storing the response generated by the response generation means; and a response transmission means capable of transmitting the response stored in the second storage means to the client And when the response transmission means determines that its replica is the master based on the master ID stored in the second storage means, the response stored in the second storage means It transmits to the said client, It is characterized by the above-mentioned.

According to a seventh aspect of the present invention, in the distributed computing system according to the fifth aspect, a response generation means for generating a response to the request for the predetermined processing in each replica, and the master ID in the first storage Second storage means for storing the response generated independently of the means and storing the response generated by the response generation means; and a response transmission means capable of transmitting the response stored in the second storage means to the client And when the response transmission means determines that its replica is the master based on the master ID stored in the second storage means, the response stored in the second storage means It transmits to the said client, It is characterized by the above-mentioned.

The invention according to claim 8 is the distributed computing system according to claim 6, wherein the response transmission means determines that its replica is not a master based on the master ID stored in the second storage means. Sometimes, information on the master ID is transmitted to the client instead of the response stored in the second storage means.

The invention according to claim 9 is the distributed computing system according to claim 7, wherein the response transmission unit determines that its replica is not a master based on a master ID stored in the second storage unit. Sometimes, information on the master ID is transmitted to the client instead of the response stored in the second storage means.

According to a tenth aspect of the present invention, in the distributed computing system according to the fourth aspect, when there is a master change among the plurality of replicas in each of the replicas, information on the master ID of the new master is sent to the client. A master change notification means for notifying is provided.

According to an eleventh aspect of the present invention, in the distributed computing system according to the fifth aspect, when there is a master change among the plurality of replicas in each of the replicas, information on a master ID of a new master is transmitted to the client. A master change notification means for notifying is provided.

The invention according to claim 12 is the distributed computing system according to claim 6, wherein when each master has a master change among the plurality of replicas, information on a master ID of a new master is sent to the client. A master change notification means for notifying is provided.

According to a thirteenth aspect of the present invention, in the distributed computing system according to the seventh aspect, when there is a master change among the plurality of replicas in each of the replicas, information on a master ID of a new master is transmitted to the client. A master change notification means for notifying is provided.

The invention according to claim 14 is the distributed computing system according to claim 8, wherein when each master has a master change among the plurality of replicas, information on a master ID of a new master is sent to the client. A master change notification means for notifying is provided.

According to a fifteenth aspect of the present invention, in the distributed computing system according to the ninth aspect, when there is a master change among the plurality of replicas in each of the replicas, information on a master ID of a new master is transmitted to the client. A master change notification means for notifying is provided.

The invention according to claim 16 is the distributed computing system according to claim 8, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when receiving information on the master ID from the response transmission means to the client, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. Is provided.

The invention according to claim 17 is the distributed computing system according to claim 9, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when receiving information on the master ID from the response transmission means to the client, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. Is provided.

The invention according to claim 18 is the distributed computing system according to claim 10, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master stored in the first storage means Change the master ID without replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. Means are provided.

The invention according to claim 19 is the distributed computing system according to claim 11, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master stored in the first storage means Change the master ID without replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. Means are provided.

The invention according to claim 20 is the distributed computing system according to claim 12, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master stored in the first storage means Change the master ID without replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. Means are provided.

The invention according to claim 21 is the distributed computing system according to claim 13, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master stored in the first storage means Change the master ID without replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. Means are provided.

The invention according to claim 22 is the distributed computing system according to claim 14, wherein the first storage means stores a master table for storing a master ID before change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when receiving information on the master ID from the response transmission means or the master change notification means to the client, the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID stored in the first storage means is used as the master ID. A master ID changing means for replacement is provided.

The invention according to claim 23 is the distributed computing system according to claim 15, wherein the first storage means stores a master table that stores a master ID before the change when the master ID is changed. If the master ID is the same as the master ID stored in the master table when receiving information on the master ID from the response transmission means or the master change notification means to the client, the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID stored in the first storage means is used as the master ID. A master ID changing means for replacement is provided.

The invention according to claim 24 is the distributed computing system according to claim 16, wherein when the master ID changing unit receives information on the master ID from the response transmitting unit, the master ID is a replica of the transmission source Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

The invention according to claim 25 is the distributed computing system according to claim 17, wherein when the master ID changing unit receives information on the master ID from the response transmitting unit, the master ID is a replica of the transmission source. Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

The invention according to claim 26 is the distributed computing system according to claim 18, wherein when the master ID changing means receives information on the master ID from the response sending means, the master ID is a replica of the sending source. Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

According to a twenty-seventh aspect of the present invention, in the distributed computing system according to the nineteenth aspect, when the master ID changing unit receives information on the master ID from the response transmitting unit, the master ID is a replica of the transmission source Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

The invention according to claim 28 is the distributed computing system according to claim 20, wherein when the master ID changing means receives information on the master ID from the response transmitting means, the master ID is a replica of the transmission source. Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

The invention according to claim 29 is the distributed computing system according to claim 21, wherein when the master ID changing unit receives information on the master ID from the response transmitting unit, the master ID is a replica of the transmission source. Or the master ID stored in the first storage means is replaced when the master ID is the same as the master ID stored in the first storage means. It is characterized by not.

The invention according to claim 30 is the distributed computing system according to claim 22, wherein the master ID changing means receives the master ID information from the response sending means or the master change notifying means. If the ID is the same as the ID of the replica of the transmission source, or if the master ID is the same as the master ID stored in the first storage means, the ID is stored in the first storage means. The master ID is not replaced.

The invention according to claim 31 is the distributed computing system according to claim 23, wherein when the master ID changing means receives information on the master ID from the response sending means or the master change notifying means, the master ID changing means If the ID is the same as the ID of the replica of the transmission source, or if the master ID is the same as the master ID stored in the first storage means, the ID is stored in the first storage means. The master ID is not replaced.

According to the present invention, a processing request notification means for receiving a processing request from a client at each replica constituting the cell and notifying other replicas of a processing request and a client ID for identifying the client, and the client The processing request is received from the client, or the processing request notification means of the replica that has received the processing request is notified that the processing request has been received, and a predetermined process is performed by establishing a session with the client based on the client ID Since the processing execution means is provided, each replica establishes the same session with the client based on the client ID and performs processing, so that there are a plurality of connection windows with the cell client, and the connection is established. It is possible to prevent the service provision from being stopped due to the occurrence of a failure at the window.

According to the present invention, it is possible to prevent the service provision from being stopped due to the occurrence of a failure at the connection window with the cell client.

It is explanatory drawing which shows the distributed computing system which concerns on the form for inventing. It is explanatory drawing which shows the example of a request packet. It is explanatory drawing which shows the example of a response packet. It is explanatory drawing which shows the example of a master notification packet. 2 is a flowchart showing processing in a client in the distributed computing system of FIG. 2 is a flowchart showing threads when a request packet is received at a server in the distributed computing system of FIG. 1. 2 is a flowchart showing threads when an update processing execution notification in a server is received in the distributed computing system of FIG. 1. 2 is a flowchart showing threads when transmitting a response packet in a server in the distributed computing system of FIG. 1. It is a flowchart which shows a session establishment process as an example of the update process of FIG. It is a flowchart which shows a session closing process as an example of the update process of FIG. FIG. 8 is a flowchart showing a file writing process as an example of the updating process of FIG. 7. FIG. It is a flowchart which shows an event acquisition start process as an example of the update process of FIG. It is a flowchart which shows the process at the time of event generation after event management start. It is a flowchart which shows an event acquisition request process as an example of the update process of FIG. It is a flowchart which shows an event acquisition completion process as an example of the update process of FIG. It is explanatory drawing which shows the example of a master selection arrangement | sequence.

Embodiments for carrying out the present invention will be described with reference to the drawings.
[Configuration of distributed computing system]
FIG. 1 shows a distributed computing system according to this embodiment. The distributed computing system 1 includes a cell 2 and a client 3, and the cell 2 includes a plurality of replicas that are synchronized with each other. Here, as a replica, three servers 4 that form an agreement by Paxos and synchronize (in the case of distinguishing each server 4 and in the drawings, each server is represented by reference numerals 4a, 4b, and 4c, respectively) are exemplified. However, replicas are not limited to this.

The servers 4a, 4b, and 4c are connected to each other via a dedicated line or an Internet line, and are connected to the client 3 via an Internet line. When a request for task processing (application processing) or the like is received from the application of the client 3, the processing is performed as described later, and when a response is generated, the master described below transmits the response to the client 3. The request from such an application includes an update request for updating the data stored in the server 4 and changing its state, and a reference request for referring to the data stored in the server 4 and not changing its state. Yes, the server 4 performs an update process or a reference process in response to each request.

One of the servers 4a, 4b, and 4c serves as a master mainly responsible for communication with the client 3 at the time of an update request, and one of the servers 4a, 4b, and 4c serves as a reference from the client 3 at the time of a reference request (hereinafter referred to as "reference"). It is said.) Since the servers 4a, 4b, and 4c form an agreement by Paxos, the Paxos leader may be the master. The Paxos leader may be determined by any known method, or by a method in which the oldest server (the server with the longest startup time and the longest continuous operation time in the cell 2) is used as the leader. Further, since the servers 4a, 4b, and 4c generally have different loads, in this embodiment, a load value is calculated and a value having a small value is used as a reference.

Here, the client 3 and the server 4 are general-purpose computers, respectively, and reference numerals 5 and 6 are assigned to control units such as CPUs, and reference numerals 7 and 8 are assigned to storage devices such as memories (each control part 6 and each server 4). In the case of distinguishing the storage devices 8 and in the drawings, each control unit is represented by reference numerals 6a, 6b, and 6c, and each storage device is represented by reference numerals 8a, 8b, and 8c, respectively.

In the storage device 7 of the client 3, an area (session structure) for storing various session information related to the session between the cell 2 and the client 3 is secured, and this is referred to as a client session 9. Similarly, an area (session structure) for storing various session information is created and secured in the storage device 8 of the server 4 and is referred to as a server session 10 (when each server session 10 is distinguished or in the drawings). Represents each server session by reference numerals 10a, 10b, and 10c, respectively).

The client session 9 stores connection information, client ID, transaction sequence number, master ID, reference ID, master selection array as a master table, update transaction sequence number, and the like.

The “connection information” is connection information with the server 4, specifically, each IP address of the servers 4 a, 4 b, 4 c and each connection state between the client 3 and the servers 4 a, 4 b, 4 c (whether or not there is a connection) ).

“Client ID” is a unique ID that identifies a session, and includes a client 3 IP address, a process ID assigned to each process by the client 3, and a tag ID assigned to each tag.

“Transaction serial number” is a serial number in a session attached to a transaction issued by the client 3.

“Master ID” is the ID of the server 4 that the client 3 recognizes as the master.

“Reference ID” is the ID of the server 4 that the client 3 recognizes as a reference.

The “master selection array” is used to avoid circulation when the master is changed, and details will be described later.

“Update transaction sequence number” is a number that is replaced with a transaction sequence number when the request from the client 3 is an update request and this is properly processed in the server 4.

When the client 3 requests a predetermined process from the server 4, a request packet is transmitted from the client 3 to the server 4. The request packet includes a client ID, a transaction sequence number, an update transaction sequence number, as shown in FIG. It includes a command indicating request contents and data (request data) necessary for processing the command.

On the other hand, the server session 10 stores a client ID, a session state, a request start transaction sequence number, a request end transaction sequence number, a response transaction sequence number, a hold response flag, a hold response list, an event wait flag, a hold event list, and the like. 8 stores a master ID and a reference ID separately from the server session 10.

“Master ID” is the ID of the server 4 that the server 4 recognizes as the master, and is independent of the master ID stored in the client session 9.

“Reference ID” is an ID of the server 4 that the server 4 recognizes as a reference. The servers 4a, 4b, and 4c calculate their own load values by a predetermined method, and exchange the information with each other to recognize the server 4 having the minimum load value as a reference, and store the ID as a reference ID. Store in the devices 8a, 8b, 8c. This reference ID is transmitted from the server 4 to the client 3 by a response packet or a master notification packet described later. The reference ID in the client session 9 is replaced with the reference ID transmitted from the server 4 when it becomes “unknown” indicating that the reference is unknown. The reference ID is set to “unknown” as an initial value, and is also set to “unknown” when the operation of the corresponding server 4 cannot be confirmed.

“Client ID” is the client ID in the client session 9 conveyed by the request packet. In the server 4, the server session 10 is managed using a hash or the like so that it can be identified by the client ID.

“Session state” indicates the state of the server session 10, and is opened when a session establishment request is sent from the client 3 to the server 4, and closed when a session closure request is made.

The “request start transaction sequence number” is a number that is replaced with the transaction sequence number in the request packet when the server 4 starts the update process or the reference process.

The “request end transaction sequence number” is a number that is replaced with the transaction sequence number or the request start transaction sequence number in the request packet when the server 4 finishes the update process or the reference process.

The “response transaction sequence number” is a number that is replaced with the transaction sequence number in the request packet or the request start transaction sequence number when the server 4 generates the response packet described below.

When the server 4 responds to the client 3, a response packet is transmitted from the server 4 to the client 3. As shown in FIG. 3, the response packet includes an ID (replica ID) of the server 4 that transmits the packet, Reference ID, packet type (response type) indicating that the packet is a response packet, client ID for identifying the client 3 and session to which the packet is transmitted, response transaction sequence number, and data indicating response content (response data) including. If the server 4 is not the master, the server 4 may transmit a master notification packet that notifies the master instead of a response packet. This master notification packet is a server that transmits the packet as shown in FIG. 4 ID (replica ID), reference ID, packet type (master type) indicating that the packet is a master notification packet, and master ID.

The “pending response flag” is a flag indicating whether or not the server 4 may transmit a response packet to the client 3 or whether transmission should be held as a hold response. If this flag is “true”, the server 4 regards it as a hold instruction, holds the generated response packet as a pending response, and if this flag is “false”, regards it as a release instruction, and transmits the generated response packet.

The “pending response list” is one or a plurality of response packets stored as a pending response.

The “event waiting flag” is a flag indicating whether or not the generated event may be transmitted to the client 3 when the server 4 manages the event, or whether transmission should be suspended as a suspended event. In the distributed computing system 1, an event can be managed by the server session 10, and the server 4 that is instructed to manage the event by the client 3 sets the event waiting flag to “true” if an event acquisition request is received from the client 3. The generated event is transmitted to the client 3. Further, when there is no event acquisition request from the client 3, the event waiting flag is set to “false”, and the generated event is not immediately transmitted to the client 3 but is held as a hold event. Any event can be used. For example, when adding or deleting a file to a directory path (changing a directory under the addition or deletion of a file) is used as an event, the directory path is used as an event resource. . Events and event resources are defined by the application.

The “pending event list” is one or a plurality of events stored as a holding event.
[Distributed computing method]
5 to 15 show a distributed computing method by the distributed computing system 1, that is, a procedure in which an application of the client 3 transmits a reference request or an update request to the cell 2, and the reference process or the update process is performed in the cell 2. (See FIG. 2).

[Request packet transmission]
As shown in FIG. 5, when there is a request from an application (step 101 (denoted as “S.101” in the drawing, the same applies hereinafter)), the control unit 5 of the client 3 increments the transaction serial number (step 102). It is determined whether or not it is an update request (step 103). If it is an update request, the master ID of the client session 9 is referred to (step 104). If it is not an update request, the reference ID is referred to as a reference request (step 105), and the server 4 specified by the referenced ID is sent. The request packet is transmitted to the control unit 6 (step 106). Here, the client 3 performs synchronous transaction communication with the server 4 by sending a request packet and waiting for a response packet. Therefore, the order of the packets is guaranteed by the client 3.

The control unit 5 of the client 3 that has transmitted the request packet waits for a response packet or a master notification packet to arrive from the control unit 6 of the server 4, and if there is no response after a certain time, refers to the master ID or reference ID again. The request packet is transmitted (steps 107 to 109).

[Reception of request packet]
As shown in FIG. 6, the control unit 6 of the server 4 that has received the request packet (step 201) acquires the server session 10 from a hash or the like based on the client ID included in the request packet (corresponding to the client ID). If there is no server session 10 to be created, it is newly created: step 202).

Next, the control unit 6 determines whether or not it is an update request based on a command included in the request packet (step 203).

If it is an update request, the control unit 6 checks whether or not its own server 4 is a master (step 204), and if it is a master, the contents of the request packet are sent to all the servers 4 (including the master itself). Notification is made (step 205), and an agreement is consulted by Paxos on whether or not the update processing can be performed (step 206). Further, the control unit 6 determines that the master recognized by the client 3 and the server 4 is different if it is not the master in step 204 (for example, if there is a master change but it has not been transmitted to the client 3). In such a case, it is determined that the control unit 6 is not a master.), A master notification packet is transmitted to the control unit 5 of the client 3 (step 207). Upon receiving the master notification packet, the control unit 5 replaces the client ID of the client session 9 with the master ID in the master notification packet, and transmits the request packet again (see steps 109 and 110 in FIG. 5).

In step 206, if the consent of the majority of servers 4 is obtained and an agreement is formed by Paxos, the master control unit 6 that has received the request packet controls the control units 6 of all the servers 4 (including the master itself). In response to this, an update processing execution notification is transmitted (step 208). As a result, all servers 4 respond to the update request. If no agreement is formed by Paxos in Step 206, the control unit 6 proceeds to Step 201 and enters a standby state for the next request packet.

On the other hand, if the request is not an update request in step 203, the server 4 that has received the request packet responds to the reference request.

[Response to update requests]
As shown in FIG. 7, the control unit 6 of the server 4 that has received the update processing execution notification (step 301) acquires the server session 10 from the hash or the like based on the client ID in the request packet (the client ID). If there is no server session 10 corresponding to, a new one is created: Step 302).

Subsequently, the control unit 6 confirms whether or not the session state of the server session 10 acquired in step 302 is open (step 303), and if not open, transmits an error response to the control unit 5 of the client 3 ( Step 304), the process proceeds to Step 301 to enter a standby state for notification of execution of the next update process.

If the session state is open in step 303, the control unit 6 determines whether the transaction sequence number in the request packet is the same as the response transaction sequence number stored in the server session 10 (described as “response sequence number” in the drawing). It is determined whether or not (step 305). In the case of one request and one response, if the transaction sequence number is the same as the response transaction sequence number, the response packet stored as a pending response including the response transaction sequence number is generated corresponding to the request packet including the transaction sequence number. (In other words, the server 4 has already responded to the same request as this time, and the current request is considered to be a duplicate request.) Therefore, if the transaction sequence number and the response transaction sequence number are the same in step 305, the control unit 6 confirms that the pending response flag is “false” (step 306) and that its own server 4 is also the master at that time. (Step 307) The response packet held as a hold response is transmitted to the control unit 5 of the client 3 without generating a response packet again (step 308). If the pending response flag is “true” in step 306 or if its own server 4 is not the master in step 307, the control unit 6 proceeds to step 301 and enters a standby state for notification of execution of the next update process. enter.

If the transaction sequence number and the response transaction sequence number are not the same in step 305, the control unit 6 describes the transaction sequence number in the request packet and the request start transaction sequence number stored in the server session 10 (described as “request start sequence number” in the drawing). )) Is the same (step 309). If the transaction sequence number and the request start transaction sequence number are the same, the server 4 is considered to be processing the same request as this time, so the control unit 6 ignores the current request to avoid duplicate processing. .

In step 309, if the transaction sequence number and the request start transaction sequence number are not the same, the control unit 6 replaces the request start transaction sequence number with the transaction sequence number (step 310). Then, an update process is performed based on the command and request data in the request packet (step 311). When this update process is completed, the request end transaction sequence number (described as “request end sequence number” in the drawing) is used as the transaction sequence number. Replace (step 312).

There are many examples of update processing. For example, a request for opening a server session transmitted from the client 3 to the server 4 or a request for closing an opened server session is an update request. The process is an update process. The file writing process is also an update process.

That is, if the command included in the request packet is a server session establishment command, a server session establishment process as shown in FIG. 9 is performed as an update process in step 311 (at this time, the control unit 6 performs step 303). (Omitted, the process proceeds from step 302 to step 305).

First, the control unit 6 opens the session state of the server session 10 acquired in step 302 (step 501), initializes values of each transaction serial number and the like (step 502), and when these processes are completed, Is sent to the control unit 5 of the client 3 (step 503), and step 401 of FIG. 8 is called to perform transmission processing of the response packet (step 504).

In the figure, the control unit 6 determines whether or not it is necessary to transmit a response packet for the current update process (step 401). If not necessary, the control unit 6 proceeds to step 312 (the lock or the like as described later). If so, the control unit 6 may not generate a response packet. In such a case, although one request is one response, the response is delayed until the lock is released.

If it is necessary to send a response packet, the control unit 6 checks the hold response flag (step 402). If the flag is “true”, the response packet generated in step 503 is added to the hold response list to hold the response packet. The response list is updated (step 403). On the other hand, if the flag is “false”, the pending response list is updated by replacing the pending response list with the response packet generated in step 503 (step 404).

Next, the control unit 6 replaces the response transaction sequence number with the transaction sequence number (step 405), and confirms whether or not its own server 4 is a master (step 406). If it is a master, the control part 6 transmits the response packet of a pending | holding response list (step 407), and if it is not a master, it will transfer to step 312 without transmitting a response packet.

Then, when the server session establishment process is performed in all the servers 4, the same session is established between the client 3 and all the servers 4.

If the command included in the request packet is a server session closing command, a server session closing process as shown in FIG.

The control unit 6 cancels the open state of the server session 10 opened in step 501 (closes the session state) (step 511), and when this processing is completed, a response packet for notifying the client 3 of that fact is sent. 8 is generated (step 512), and step 401 is called to perform the response packet transmission processing of FIG. 8 (step 513).

If the command included in the request packet is a file write command, the request data in the request packet includes the file path name, offset, data to be written, and the length of the data to be written. As the update process in step 309, a file writing process as shown in FIG. 11 is performed.

The control unit 6 opens the file (step 521), writes data from the offset (step 522), and closes the file (step 523). Then, a response packet including the written length is generated (step 524), and step 401 is called to perform response packet transmission processing in FIG. 8 (step 525).

[Response to reference requests]
When the server 4 proceeds with the process by forming an agreement with Paxos, the server 4 leaked from the majority at the time of the Paxos agreement may not have performed the most recent update process, and the client 3 may be such a server. It is necessary to avoid the situation of referring to 4.

Therefore, in the reference process, as shown in FIG. 6, the control unit 6 determines whether or not the request end transaction sequence number is equal to or greater than the update transaction sequence number in the request packet (step 209), and the request end transaction sequence number is updated. If the transaction sequence number is greater than or equal to the transaction number, the server 4 confirms that the session state has been opened and proceeds to the reference process (step 210), assuming that the update is not delayed and can be used for reference. If the session state is not open, the control unit 6 transmits an error response to the control unit 5 of the client 3 (step 211).

If the session state is open in step 210, the control unit 6 replaces the request start transaction sequence number with the transaction sequence number in the request packet (step 212), and performs reference processing based on the command and request data in the request packet. (Step 213) When this reference process is completed, the request end transaction sequence number is replaced with the transaction sequence number (step 214). Also in the reference process of step 213, the control unit 6 transmits a response packet including the reference contents to the control unit 5 of the client 3, but the method is the same as the response packet transmission process of FIG. And go to step 214).

In step 209, if the request end transaction sequence number is less than the update transaction sequence number, the control unit 6 interrupts the process and enters a standby state. Then, if the update delay is resolved and the request end transaction sequence number is equal to or greater than the update transaction sequence number in the request packet, the wake-up and interrupted processing is continued.

[Reception packet reception]
When the control unit 5 of the client 3 receives the response packet from the server 4, the control unit 5 exits from the standby state of Step 109 (see FIG. 5). If the reference ID is unknown, the reference ID stored in the client session 9 is included in the response packet. (Steps 111 and 112).
Subsequently, the control unit 5 determines whether or not the response transaction sequence number in the response packet is the same as an expected value (step 113). This expected value is the same as the transaction sequence number. If the response transaction sequence number is the same as the expected value, it means that an appropriate response to the request has been received. When receiving a response to the update request, the control unit 5 replaces the update transaction sequence number with the transaction sequence number (step 114), and proceeds to step 101 for the next request or is controlled by the application. Return.

If the response transaction sequence number is not the same as the expected value in step 112, it means that an inappropriate response to the request has been received. Therefore, the control unit 5 discards the received response packet (step 115) and returns to step 107. Wait for an appropriate response.

When a plurality of response packets are derived from a single request, for example, in the case of multiple SQL statements in the database, the control unit 6 of the server 4 accumulates the response packets using a hold response flag, and holds a plurality (predetermined number) of hold packets. At this point, the master control unit 6 may transmit the response as a single response. That is, since the control unit 6 can know that a plurality (predetermined number) of response packets are created by analyzing the command of the request packet, the pending response flag is kept until the plurality (predetermined number) of response packets are created. Is set to “true”, the generated response packet is held so as not to be transmitted immediately, and sequentially stored in the pending response. When all of a plurality (predetermined number) of response packets are generated, the pending response flag is set to “false”, and the master control unit 6 adds a plurality of (predetermined number) of response packets with a header to the batch. Send.

[Event management]
As described above, the server 4 can manage events by the server session 10.

In order for the client 3 to request the server 4 to start event management, an event acquisition start request is transmitted. After the event acquisition start request is transmitted, the client 3 transmits an event acquisition request to request the server 4 to acquire the event. In addition, when the client 3 requests the server 4 to end the event management, an event acquisition end request is transmitted. Since these requests related to event management are update requests, specifically, the following processing may be performed in step 311.

That is, as shown in FIG. 12, when receiving the event acquisition start request, the control unit 6 of the server 4 associates the event resource with the server session 10 and stores it in the storage device 8 (step 601). This association is performed by setting a joint between the event resource and the server session 10. Here, the joint includes an event link that links the joint and the event resource, a session link that links the joint and the server session 10, And a post-processing function when the joint is discarded. The event resource to be associated is specified by the request packet of the event acquisition start request, and the server session 10 to be associated is specified at step 302 (or step 202).

When the association is completed, the control unit 6 generates a response packet to that effect (step 602) and transmits it to the control unit 5 of the client 3 (step 603). The method is the same as the response packet transmission process of FIG. It is.

When an event occurs after receiving an event acquisition start request, the processing of the server 4 differs depending on whether an event acquisition request is received from the client 3 or not.

Specifically, as shown in FIG. 13, when the control unit 6 of the server 4 knows that an event has occurred in the event resource (step 611), it checks the event waiting flag (as described above, and will be described in detail later). As described above, if an event acquisition request is received from the client 3, the event wait flag is “true”, and if no event acquisition request is received from the client 3, the event wait flag is “false”: Step 612). If the event waiting flag is “true”, a response packet is generated to transmit the generated event to the client 3 (step 613), and is transmitted to the control unit 5 of the client 3 in the same manner as in FIG. In step 614), the event waiting flag is set to "false" (step 615).

On the other hand, if the event waiting flag is “false”, the control unit 6 adds the generated event to the hold event and registers it (step 616), and enters a standby state such as an event acquisition request or the next event occurrence. .

As shown in FIG. 14, when there is an event acquisition request from the client 3 (step 621), the control unit 6 checks whether there is a pending event in the pending event list (step 622). The wait flag is set to “true” (step 623) so that an event generated until the next event acquisition request is held in the hold event list.

If there is a pending event in step 622, the control unit 6 cuts all the pending events registered in the pending event list in a lump and generates a response packet including these (step 624), and sends the response packet to the control unit 5 of the client 3 Transmission is performed in the same manner as in FIG. 8 (step 625).

As shown in FIG. 15, upon receiving the event acquisition end request, the control unit 6 of the server 4 releases the association between the event resource and the server session 10 (step 631), and generates a response packet to that effect (step 631). 632), and transmits to the control unit 5 of the client 3 by the same method as in FIG. 8 (step 633).

[Master selection array]
The servers 4a, 4b, and 4c are not necessarily always in operation (in order to form an agreement by Paxos, it is only necessary that two of the three are in operation), and the master itself may fail. A master change may occur due to a communication failure between the master and another server. When the server 4 recognizes the master change, if the server 4 is connected to the client 3, a master notification packet as shown in FIG. 4 can be transmitted to notify the client 3 of the master change.

However, since it takes a certain time for all of the servers 4a, 4b, and 4c to recognize the new master when the master is replaced, the entire cell 2 is in a state as if the old and new masters coexist. At this time, if the old master recognizes the new master as the master and the new master still recognizes the old master as the master, when the client 3 issues a request to the old master, a master notification packet notifying the new master is returned. When a request is issued to the new master, a circulation occurs in which a master notification packet for notifying the old master is returned. This circulation is resolved naturally as the recognition of the servers 4a, 4b, and 4c converges to one master over time, but during this time, packets flow continuously and cause a network load.

Therefore, in this embodiment, in the client session 9, for example, a master selection array for avoiding circulation as shown in FIG. 16 is prepared. In the master selection array, every time the master ID of the client session 9 is changed, the array element of the server 4 corresponding to the old master ID is marked with “1” (the array element not marked is “0”). .) Therefore, if the client 3 newly receives the master notification packet and the array element of the server 4 corresponding to the master ID in the master notification packet is marked, the master notification packet has notified the old master. Since there is a fear, the control unit 5 of the client 3 considers that the above-mentioned circulation has occurred, and waits for the next master notification packet without replacing the master ID of the client session 9 with the master ID notified by the master notification packet.

Specifically, if the replica ID and the master ID in the master notification packet match, the server 4 that has transmitted the master notification packet recognizes itself as the master, and no circulation occurs. Then, the control unit 5 of the client 3 sets the replica ID as the master ID of the client session 9 and clears all marks in the master selection array.

If the master ID in the master notification packet matches the master ID of the client session 9, the master 3 and the server 4 have the same master recognition. do nothing.

On the other hand, if the master ID in the master notification packet and the master ID of the client session 9 do not match, the control unit 5 marks “1” in the array element of the server 4 corresponding to the master ID in the master notification packet. It is judged whether it is attached. If it is marked, it is determined that the above-described circulation has occurred and that the master is being selected (Jeopard), and the master ID of the client session 9 is unknown, and the next master notification packet is awaited. If not marked, the array element of the server 4 corresponding to the master ID previously stored in the client session 9 is marked, and the master ID of the client session 9 is replaced with the master ID in the master notification packet. .

When the controller 5 detects the disconnection of any of the servers 4a, 4b, and 4c based on the connection information so that the old master whose connection has been disconnected is not recognized as the master, the control unit 5 If the master ID of the client session 9 matches, the array element of the server 4 corresponding to the master ID of the client session 9 is marked.

If the master notification packet is not sent even after a certain time during the master selection, the control unit 5 clears all the marks of the master selection array and sequentially sends request packets to the servers 4a, 4b, 4c. And wait for the master notification packet to be sent.

According to the distributed computing system 1 according to the present embodiment, each server 4 configuring the cell 2 receives a processing request from the client 3, and notifies that the processing request has been made and the client ID that identifies the client 3 to the other server 4. A processing request notification means for notifying the server 4 and a processing request from the client 3 or a notification to the effect that a processing request has been received from another server 4 that has received the processing request. Since the control unit 6 is provided that functions as a process execution means for establishing a session between them and performing a predetermined process, each server 4 establishes the same session with the client 3 based on the client ID for processing. By doing this, there are multiple connection windows with the cell client, which is caused by a failure at the connection window. It is possible to prevent the stopping of the service provider.

As mentioned above, although the form for implementing this invention was illustrated, embodiment of this invention is not restricted to what was mentioned above, You may change suitably etc. in the range which does not deviate from the meaning of this invention.

For example, the servers that make up the cell may synchronize without relying on Paxos, and the Paxos leader may not be the master. Also, the configuration of the request packet, response packet, master notification packet, and client ID may be different from those described above.

DESCRIPTION OF SYMBOLS 1 Distributed computing system 2 Cell 3 Client 4a, 4b, 4c Server (replica)
5 Control unit (processing request means, request transmission means, master ID changing means)
6a, 6b, 6c Control unit (processing request notification means, session establishment means, response generation means, response transmission means, master change notification means)
7 Storage device (first storage means)
8a, 8b, 8c Storage device (second storage means)
9 Client session 10a, 10b, 10c Server session

Claims

A cell including a plurality of replicas synchronized with each other and a client;
The client stores, as session information for managing a session between the cell and the client, connection information for the client to connect to each replica and a client ID for specifying the client. Storage means, and processing request means for making a request for a predetermined process and a notification of the client ID based on the connection information for any of the plurality of replicas,
Wherein each replica receives a processing request from the processing requesting means receives the processing request notifying means for notifying that there is a processing request and the client ID to the other replicas, the processing request from said processing request means, or receives a notification indicating that a processing request from a processing request notification means of the replica which has received the processing request, the processing executing means for performing the predetermined processing by opening the session between the client based on the client ID And a distributed computing system.
The distributed computing system according to claim 1, wherein the predetermined process is an update process.
2. The distributed computing system according to claim 1, wherein the predetermined processing request is a session establishment request for causing each replica to establish the session.
The first storage means stores a master ID for identifying which replica is the master when one of the plurality of replicas is selected as a master;
2. The processing requesting unit makes a request for the predetermined processing and a notification of the client ID to a replica specified by a master ID stored in the first storage unit. Item 4. The distributed computing system according to any one of items 3.
When the first storage unit is selected as a reference to be used for reference during a reference process when one of the plurality of replicas is stored, a reference ID for specifying which replica is a reference;
5. The distributed computing system according to claim 4, wherein the processing request unit requests a reference process for a replica specified by a reference ID stored in the first storage unit.
In each of the replicas, a response generation unit that generates a response to the request for the predetermined process, and the master ID are stored independently of the first storage unit, and the response generated by the response generation unit is stored Second storage means, and response transmission means capable of transmitting the response stored in the second storage means to the client,
When the response transmission unit determines that its replica is a master based on the master ID stored in the second storage unit, the response stored in the second storage unit is transmitted to the client. The distributed computing system according to claim 4, wherein:
In each of the replicas, a response generation unit that generates a response to the request for the predetermined process, and the master ID are stored independently of the first storage unit, and the response generated by the response generation unit is stored Second storage means, and response transmission means capable of transmitting the response stored in the second storage means to the client,
When the response transmission unit determines that its replica is a master based on the master ID stored in the second storage unit, the response stored in the second storage unit is transmitted to the client. The distributed computing system according to claim 5, wherein:
When the response transmission unit determines that its replica is not a master based on the master ID stored in the second storage unit, the response stored in the second storage unit is information related to the master ID. The distributed computing system according to claim 6, wherein the data is transmitted to the client instead.
When the response transmission unit determines that its replica is not a master based on the master ID stored in the second storage unit, the response stored in the second storage unit is information related to the master ID. 8. The distributed computing system according to claim 7, wherein the distributed computing system transmits to the client instead.
5. The master change notification means for notifying the client of information related to a master ID of a new master when each of the replicas has a master change among the plurality of replicas. The distributed computing system described in 1.
6. The master change notifying means for notifying the client of information related to a master ID of a new master when the master is changed between the plurality of replicas in each of the replicas. The distributed computing system described in 1.
7. The master change notifying means for notifying the client of information related to a master ID of a new master when each replica has a master change between the plurality of replicas. The distributed computing system described in 1.
8. The master change notifying means for notifying the client of information related to a master ID of a new master when each replica has a master change among the plurality of replicas. The distributed computing system described in 1.
9. The master change notifying means for notifying the client of information related to a master ID of a new master when each replica has a master change among the plurality of replicas. The distributed computing system described in 1.
10. The master change notifying means for notifying the client of information related to a master ID of a new master when each replica has a master change between the plurality of replicas. The distributed computing system described in 1.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the response transmission means, the master ID stored in the first storage means is used. Master ID changing means for replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. The distributed computing system of claim 8, wherein the distributed computing system is provided.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the response transmission means, the master ID stored in the first storage means is used. Master ID changing means for replacing the master ID stored in the first storage means with the master ID if the master ID is not the same as the master ID stored in the master table. The distributed computing system according to claim 9, wherein the distributed computing system is provided.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. The distributed computing system according to claim 10, wherein the distributed computing system is provided.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. The distributed computing system according to claim 11, further comprising:
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. The distributed computing system according to claim 12, wherein:
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
If the master ID is the same as the master ID stored in the master table when the client receives information on the master ID from the master change notification means, the master ID stored in the first storage means If the master ID is not the same as the master ID stored in the master table, the master ID changing unit replaces the master ID stored in the first storage unit with the master ID. The distributed computing system according to claim 13, wherein the distributed computing system is provided.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
When the master ID is the same as the master ID stored in the master table when the client receives information about the master ID from the response transmission means or the master change notification means, the first storage means The stored master ID is not replaced with the master ID, and if the master ID is not the same as the master ID stored in the master table, the master ID stored in the first storage means is replaced with the master ID. 15. The distributed computing system according to claim 14, further comprising: a master ID changing unit that performs the operation.
The first storage means stores a master table that stores a master ID before the change when the master ID is changed,
When the master ID is the same as the master ID stored in the master table when the client receives information about the master ID from the response transmission means or the master change notification means, the first storage means The stored master ID is not replaced with the master ID, and if the master ID is not the same as the master ID stored in the master table, the master ID stored in the first storage means is replaced with the master ID. 16. The distributed computing system according to claim 15, further comprising: a master ID changing unit that performs the operation.
When the master ID changing unit receives information on the master ID from the response transmission unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 17. The distributed computing system according to claim 16, wherein the master ID stored in the first storage unit is not replaced when the master ID is the same as the master ID stored in.
When the master ID changing unit receives information on the master ID from the response transmitting unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 18. The distributed computing system according to claim 17, wherein when the master ID is the same as the master ID stored in the first storage unit, the master ID stored in the first storage unit is not replaced.
When the master ID changing unit receives information on the master ID from the response transmitting unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 19. The distributed computing system according to claim 18, wherein the master ID stored in the first storage means is not replaced if the master ID is the same as the master ID stored in.
When the master ID changing unit receives information on the master ID from the response transmitting unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 20. The distributed computing system according to claim 19, wherein when the master ID is the same as the master ID stored in the first storage means, the master ID stored in the first storage means is not replaced.
When the master ID changing unit receives information on the master ID from the response transmitting unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 21. The distributed computing system according to claim 20, wherein the master ID stored in the first storage means is not replaced when the master ID is the same as the master ID stored in.
When the master ID changing unit receives information on the master ID from the response transmitting unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is the first storage unit 23. The distributed computing system according to claim 21, wherein the master ID stored in the first storage means is not replaced if the master ID is the same as the master ID stored in.
When the master ID changing unit receives information on the master ID from the response transmitting unit or the master change notifying unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is 23. The distributed computer according to claim 22, wherein when the master ID is the same as the master ID stored in the first storage unit, the master ID stored in the first storage unit is not replaced. System.
When the master ID changing unit receives information on the master ID from the response transmitting unit or the master change notifying unit, if the master ID is the same as the ID of the replica of the transmission source, or the master ID is 24. The distributed computer according to claim 23, wherein when the master ID is the same as the master ID stored in the first storage unit, the master ID stored in the first storage unit is not replaced. System.