JP2011210109A - Synchronous processing method and system for large-scale distributed processing system, primary server, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce, in a large-scale distributed processing system, the response time to a request from a client terminal while efficiently using a network zone by improving the throughput between a primary server and a backup server.SOLUTION: The request from the client terminal is executed, state information is updated, and a "real-time log" or "gradual log" is generated based on an identifier contained in the request and stored in the trail end of a log storage means. When a read trail end log is the "real-time log", all logs stored therein are transmitted to the backup server, and when the read log is the "gradual log", the accumulation is continued as it is. When the read log is the "gradual log" and either a condition in which the amount of accumulated log groups is a predetermined threshold or more or a condition in which a predetermined time has passed is satisfied, all log groups stored in the log storage means are transmitted to the backup server.

Description

  The present invention relates to a synchronous processing method and system, a primary server, and a program in a large-scale distributed processing system. In particular, as in data mining, Web content and log information with a scale of tens of TB (terabytes) to hundreds of TB Processing method in a large-scale distributed processing system for managing a log for replicating a server operating as a redundant system by software when a failure occurs in a primary server in a large-scale distributed system that efficiently distributes And a system, a primary server, and a program.

  Conventionally, even when a failure occurs in the primary server, as a first method for replicating a backup for maintaining the function of the system, as shown in FIG. Send update message and reply. When performing active replication, as shown in FIG. 9, a request from a client terminal is transmitted to all servers (primary server and all backup servers), and the server receiving the request updates the status. There is a method of returning to the client terminal (for example, see Non-Patent Document 1).

  As a second technique for managing logs in the primary server, as shown in FIG. 10, there is a technique for improving throughput by buffering operation logs by the primary server and sending them together to a backup server ( For example, refer nonpatent literature 2).

  As a third conventional technique, as shown in FIG. 11, there is a technique in which a client terminal transmits a request to a primary server, receives the result, and then transmits the result from the client terminal to the backup server (for example, , See Patent Document 1).

JP-T-2002-525748

Software-Based Replication for Fault Tolerance, IEEE Computer, 30 (4), pp. 68-74, April, 1997. The Google File System, Proceedings of the 19th ACM Symposium on Operating Systems Principles, pp. 29-43, October, 2003.

  However, the first conventional method transmits update messages in the order of “client terminal → primary server → backup server”, and the primary server backs up a large number of requests received from a plurality of client terminals one by one. Since the data is transmitted to the server, the throughput between the primary server and the backup server decreases. In addition, when performing active replication, it is necessary to provide communication means for maintaining the order and primitiveness of update messages in order to simultaneously transmit from a plurality of client terminals to the primary server and the backup server, and the implementation is complicated. Become.

  In addition, the above-described second conventional method buffers a predetermined number of operation logs in the primary server, and collectively transmits them to the backup server, and collectively returns responses from the backup server to the primary server to the client terminal. There is a problem that the response time after the client terminal transmits a request to the primary server increases.

  The third method transmits a request from the client terminal to the primary server, and after the response is returned from the primary server, the client terminal transmits a request to the backup server. Similar to active duplication, there is a problem that a communication means for maintaining the order and primitiveness of update messages is required, and the implementation is complicated.

  The present invention has been made in view of the above points. In a large-scale distributed processing system, the present invention improves throughput between a primary server and a backup server, efficiently uses a network bandwidth, and responds to a request from a client terminal. It is an object of the present invention to provide a synchronous processing method and system, a primary server, and a program in a large-scale distributed processing system capable of shortening the time.

The present invention (Claim 1) is a synchronous processing method between a primary server and a backup server in a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server,
The primary server having log storage means for storing logs is
A request including a first identifier indicating that information is stored only in the server, or a second identifier indicating that information can be obtained from a worker even if a failure occurs in the server When received from the client terminal, the request is executed, status information is updated, and if the first identifier is included in the request, an immediate log is generated and the second identifier is included. A log storage step for generating a gradual log and storing it at the end of the log storage means;
If the last log stored in the log storage means is an immediate log, a synchronous transmission step of transmitting all the logs stored in the log storage means up to that point to the backup server;
The log at the end of the log storage means is a gradual log, and
The amount of log sets stored in the log storage means is equal to or greater than a predetermined threshold (Th _A );
A predetermined time (Th _B ) has elapsed since the log was stored in the log storage means;
If any of the conditions is met, an asynchronous transmission step of transmitting all log sets stored in the log storage means to the backup server;
I do.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 2) is a synchronous processing system between a primary server and a backup server in a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server,
The primary server 100
Log storage means 130 for storing logs;
Upon receiving a request from the client terminal, the request execution means 111 that executes the request and updates the status information;
If the request includes the first identifier indicating that the information is stored only on the server, an “immediate log” is generated, and the information is acquired from the worker even if a failure occurs in the server Log generation means 112 that generates a “gradual log” and stores it at the end of the log storage means 130;
When the tail log stored in the log storage unit 130 is “immediate log”, the synchronous transmission unit 141 that transmits all the logs stored in the log storage unit to the backup server;
The log at the end of the log storage means 130 is “gradual log”, and
The amount of log sets stored in the log storage means 130 is greater than or equal to a predetermined threshold;
A predetermined time has elapsed since the log was stored in the log storage means 130;
If any of the conditions is met, an asynchronous transmission unit 142 that transmits all log sets stored in the log storage unit to the backup server,
Have

The present invention (Claim 3) is a primary server that performs synchronization processing with a backup server in a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server,
Log storage means for storing logs;
A request including a first identifier indicating that information is stored only in the server, or a second identifier indicating that information can be obtained from a worker even if a failure occurs in the server When received from the client terminal, the request is executed, status information is updated, and if the first identifier is included in the request, an immediate log is generated and the second identifier is included. Log generating means for generating a gradual log and storing it at the end of the log storage means,
If the tail log stored in the log storage means is an immediate log, synchronous transmission means for transmitting all the logs stored in the log storage means to the backup server,
The log at the end of the log storage means is a gradual log, and
The amount of log sets stored in the log storage means is greater than or equal to a predetermined threshold;
A predetermined time has elapsed since the log was stored in the log storage means;
If any of the above conditions is met, an asynchronous transmission unit that transmits all log sets stored in the log storage unit to the backup server is provided.

  The present invention (Claim 4) is a synchronization processing program for causing a computer to function as each means constituting the primary server according to Claim 3.

  As described above, according to the present invention, the log stored in the storage means is stored in the server only and cannot be recovered when the information is lost due to a failure of the server, By using “gradual log” that can acquire information from the worker even if a server failure occurs, it is stored in the storage means at the time of “immediate log” when replicating to the backup server. In the case of “gradual log”, it is stored in the storage means until a predetermined condition is satisfied, and if the predetermined condition is met, it is stored in the storage means at that time Since all logs are transmitted, the throughput between the primary server and the backup server is improved, and the network bandwidth can be used efficiently.

  In the case of “gradual log”, a response is immediately returned to the client terminal, so that the response time for the request from the client terminal can be shortened.

It is a principle block diagram of this invention. 1 is a system configuration diagram to which the present invention is applied. It is a primary server hardware block diagram in one embodiment of this invention. It is a primary server function block diagram in one embodiment of this invention. It is a flowchart of the process of the primary server in one embodiment of this invention. It is a flowchart of the log synchronous transmission process of the primary server in one embodiment of this invention. It is a flowchart of the log asynchronous transmission process of the primary server in one embodiment of this invention. It is the sequence chart of the message by the conventional 1st method (backup replication). It is the sequence chart of the message by the conventional 1st method (active duplication). It is the sequence chart of the message by the conventional 2nd method. It is the sequence chart of the message by the conventional 3rd method.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a system configuration diagram to which the present invention is applied.

  As shown in FIG. 2, the present invention includes a primary server 100 that performs data management and control of a distributed file system, a backup server 200, and a lock server that provides basic functions for enhancing the availability and consistency of the distributed system. Master 300, a plurality of client terminals 10 having client terminal libraries linked to applications using a distributed file system of the distributed system, a plurality of workers 20 that physically hold data, and a plurality of terminals connected to the lock server master 300 This is a technique applied to a large-scale distributed processing system composed of lock server replicas 30.

  The primary server 100 acquires a file meta-information acquisition / update request from the client terminal 10, copies the updated meta-information and transmits it to the backup server 200, and manages data for the worker 20 based on the meta-information. Take control. Further, the state of the distributed file master is stored in the lock server master 300.

  The client terminal 10 acquires the location information of the primary server 100 from the lock server 300, acquires the meta information of the file from the primary server 100, and the meta information of the worker 20 that manages the data passed from the primary server 100. Based on the above, data is read or data is written.

  The worker 20 registers and provides data by being accessed from the client terminal 10.

  The lock server master 300 manages the distributed file master status and the location information of the distributed master, receives lock acquisition requests from the primary server 100 and the backup server 200, performs exclusive control between the servers, and monitors the alive state. For example, when a failure occurs in the primary server 100, the alive monitoring file of the primary server 100 is deleted.

The lock server replica 30 holds the replicated information of the lock server 300. The present invention describes log management of the primary server 100 on the system as described above.

  FIG. 3 shows the hardware configuration of the primary server in one embodiment of the present invention.

  As shown in the figure, the primary server 100 includes a memory 102, a storage unit such as a hard disk 103, a CPU 101, a timer 104, and a network interface 105.

  FIG. 4 shows a functional configuration of the primary server in the embodiment of the present invention.

  The primary server 100 includes a request processing unit 110, a state management unit 120, a log storage unit 130, and a log transmission unit 140. Of these, the state management unit 120 and the log storage unit 130 are storage media such as a memory.

  When receiving a request from the client terminal 10, the request processing unit 110 executes the request, stores the resulting state in the state management unit 120, generates a log, and stores the log in the log storage unit 130. The request processing unit 110 determines whether the content of the request message is “immediate” or “gradual” based on a type identifier (hereinafter referred to as a type ID) included in the received request message. Or “gradual log”. For example, if the type ID included in the request message is “001”, it is stored in the log storage unit 130 as “immediate log”, and if it is “002”, it is stored as “gradual log”.

  Note that the above “immediate log” refers to a log in which information is stored only on the server, and cannot be recovered if the information is lost due to a server failure.

  The “gradual log” refers to a log that can be recovered by acquiring necessary information from the worker 20 even if information is lost due to a server failure.

  The state management unit 120 has a state management table that stores the state after the request passed from the request processing unit 110 is executed. When there is a discrepancy in the status information between the primary server 100 and the backup server 200 and the primary server 100 fails, the backup server 200 acquires necessary information from the worker 20.

  The log storage unit 130 stores the log passed from the request processing unit 110. The log storage unit 130 is a FIFO queue, and the stored log is a request command, target parameters, and status information.

  The log transmission unit 140 transmits a log set to the backup server 200 based on the type of log (immediate log / gradual log) stored in the log storage unit 130.

  Hereinafter, a series of processes of the primary server 100 will be described.

  FIG. 5 is a flowchart of the processing of the primary server in one embodiment of the present invention.

  Step 210) The request processing unit 110 of the primary server 100 receives and executes the request from the client terminal 10, and updates the state of the state management table of the state management unit 120.

  Step 220) The request processing unit 110 generates a log and stores the log at the end of the log set in the log storage unit 130. At this time, the type ID included in the request is referred to, it is determined whether the type ID is “immediate” or “gradual”, and an immediate log or a gradual log is generated. In the above example, the “immediate log” is generated when the type ID = 001, and the “gradual log” is generated when the type ID = 002.

  Step 230) The log transmission unit 140 determines whether the log generated in Step 210 is “immediate log” or “gradual log”. If it is “immediate log”, the process proceeds to step 240. Goes to step 250.

  Step 240) The log transmission unit 140 performs the primary server log synchronous transmission process shown in FIG. 6 and proceeds to Step 290.

  Step 250) The log transmission unit 140 determines whether the amount of the log set stored in the log storage unit 130 is equal to or greater than a predetermined threshold value. Shifts to Step 270.

  Step 260) The log transmission unit 140 performs a primary server log asynchronous transmission process shown in FIG. The process of this step can be executed asynchronously with the main process (the flowchart) of the primary server 100. For this reason, the log storage unit 130 can use a known method such as a double buffer or a ring buffer as a method for storing the log.

  Step 270) The log transmission unit 140 acquires time information from the timer, and proceeds to step 280 if the elapsed time calculated using the time information is equal to or greater than a predetermined threshold value. 290. Here, as the elapsed time, it is possible to use an elapsed time or the like since the log is accumulated when the log set stored in the log storage unit 130 is empty (= φ) at regular intervals. it can. Further, the present invention is not limited to this example, and the process may proceed to step 280 at an arbitrary time when a predetermined elapsed time has expired. When the determination in step 280 is Yes, or after execution of step 260 when the determination is No, the process returns to immediately after the transition to step 280.

  Step 280) If the amount of the log set stored in the log storage unit 130 is empty (= φ), the process proceeds to Step 290, and if it is not empty (≠ φ), the process proceeds to Step 260, and the primary server log Perform asynchronous transmission processing.

  Step 290) The request processing unit 110 notifies the processing result to the client terminal 10.

  Next, the primary server log synchronous transmission process in step 240 will be described.

  FIG. 6 is a flowchart of the primary server log synchronous transmission process according to the embodiment of the present invention.

  Step 310) The log transmission unit 140 transmits the log set stored in the log storage unit 130 to the backup server 200.

  Step 320) The log transmission unit 140 receives a response from the backup server 200.

  Step 330) The log transmission unit 140 deletes the log set stored in the log storage unit 130.

  Next, the primary server asynchronous transmission process in step 260 will be described.

  FIG. 7 is a flowchart of the primary server log asynchronous transmission process according to the embodiment of the present invention.

  Step 410) The log transmission unit 140 transmits the log set stored in the log storage unit 130 to the backup server 200.

  Step 420) The log transmission unit 140 receives a response from the backup server 200.

  Step 430) The log transmission unit 140 deletes the log set stored in the log storage unit 130.

As mentioned above, logs are classified into immediate logs and gradual logs at the time of log generation,
-Log is immediate log;
-A predetermined threshold value or more of the log amount stored in the log storage unit 130;
When the elapsed time is any one of the predetermined threshold value or more, the primary server 100 transmits the log set stored in the log storage unit 130 up to that time to the backup server 200, thereby the primary server By improving the throughput between the server 100 and the backup server 200, the network bandwidth can be used efficiently.

  Further, when the log is a gradual log, the response time for the request from the client terminal 10 can be shortened by storing the log in the log storage unit 130 and immediately returning a response from the primary server 100 to the client terminal 10. it can.

  Note that the operation of the constituent elements of the primary server shown in FIG. 4 can be constructed as a program and installed in a computer used as the primary server for execution, or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

10 Client terminal 20 Worker 30 Lock server replica 100 Primary server 101 CPU
102 memory 103 hard disk 104 timer 105 network interface 110 request processing unit 111 request execution unit 112 log generation unit 120 status management unit 130 log storage unit, log storage unit 140 log transmission unit 141 synchronous transmission unit 142 asynchronous transmission unit 200 backup server

Claims (4)

A synchronization processing method between a primary server and a backup server in a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server,
The primary server having log storage means for storing logs is
Upon receiving a request from the client terminal, a request execution step of executing the request and updating state information;
If the request includes a first identifier indicating that the information is stored only on the server, an "immediate log" is generated and information is obtained from the worker even if the server fails A log storage step of generating a “gradual log” and storing it at the end of the log storage means when a second identifier indicating that it is possible to do so is included;
When the tail log stored in the log storage means is an “immediate log”, a synchronous transmission step of transmitting all the logs stored in the log storage means to the backup server;
The log at the end of the log storage means is “gradual log”, and
The amount of log sets stored in the log storage means is greater than or equal to a predetermined threshold;
A predetermined time has elapsed since the log was stored in the log storage means;
If any of the conditions is met, an asynchronous transmission step of transmitting all log sets stored in the log storage means to the backup server;
A synchronization processing method characterized by: A synchronous processing system between a primary server and a backup server in a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server,
The primary server is
Log storage means for storing logs;
Upon receiving a request from the client terminal, request execution means for executing the request and updating state information;
If the request includes a first identifier indicating that the information is stored only on the server, an "immediate log" is generated and information is obtained from the worker even if the server fails Log generation means for generating a “gradual log” and storing it at the end of the log storage means when a second identifier indicating that it is possible to do so is included;
If the tail log stored in the log storage means is "immediate log", synchronous transmission means for transmitting all the logs stored in the log storage means to the backup server,
The log at the end of the log storage means is “gradual log”, and
The amount of log sets stored in the log storage means is greater than or equal to a predetermined threshold;
A predetermined time has elapsed since the log was stored in the log storage means;
If any of the conditions is met, asynchronous transmission means for transmitting all log sets stored in the log storage means to the backup server,
A synchronous processing system comprising: In a large-scale distributed processing system including a plurality of client terminals, a plurality of workers, a primary server, and a backup server, a primary server that performs synchronization processing with the backup server,
Log storage means for storing logs;
Upon receiving a request from the client terminal, request execution means for executing the request and updating state information;
If the request includes a first identifier indicating that the information is stored only on the server, an "immediate log" is generated and information is obtained from the worker even if the server fails Log generation means for generating a “gradual log” and storing it at the end of the log storage means when a second identifier indicating that it is possible to do so is included;
If the tail log stored in the log storage means is "immediate log", synchronous transmission means for transmitting all the logs stored in the log storage means to the backup server,
The log at the end of the log storage means is “gradual log”, and
The amount of log sets stored in the log storage means is greater than or equal to a predetermined threshold;
A predetermined time has elapsed since the log was stored in the log storage means;
If any of the conditions is met, asynchronous transmission means for transmitting all log sets stored in the log storage means to the backup server,
A primary server characterized by comprising:   A synchronization processing program for causing a computer to function as each means constituting the primary server according to claim 3.

Images