JP6197872B2 - Data processing system, data processing method, and data processing program - Google Patents

Description

  The present invention relates to a data processing system, a data processing method, and a program recording medium. In particular, the present invention relates to a data processing system, a data processing method, and a program recording medium for recording a data processing program used for a distributed key value store.

  As a technique for processing a huge amount of data at high speed, a system called distributed key value store (hereinafter referred to as distributed KVS) is used (KVS: KEY-VALUE Store). In distributed KVS, a key (KEY) serving as a unique indicator is set for arbitrary data (value: VALUE) to be stored, and the data is stored as a pair of KEY and VALUE (KV pair) (KV: KEY− VALUE). In distributed KVS, data can be acquired by specifying KEY and acquiring the corresponding VALUE.

  By the way, examples of data backup include full backup, incremental backup, and differential backup. Compared to full backup, incremental backup and differential backup have the advantages of small backup capacity and short processing time. Data to be backed up is stored in a storage means such as a database for each generation. Therefore, when backing up a huge amount of data, incremental backup and differential backup that require a smaller backup capacity are more suitable than full backup.

  Patent Document 1 discloses a technique for restoring data in a heterogeneous storage environment by providing a data management server that can easily access a plurality of data regardless of the position and type of the data.

  In addition, the distributed KVS is equipped with a function called replication that writes the same data to a plurality of servers in order to ensure fault tolerance as a system. As a method of performing replication, there are a server routing method in which a replication destination server is determined and executed on the server side, and a client routing method in which a replication destination server is determined and executed on the client side.

  FIG. 18 is a diagram for explaining a general client routing method. In the client routing method of FIG. 18, the clients 90 (90-1 to 4) replicate the acquired key value data (KV pairs), and the replicated KV pairs are server 91 (91-1 to 6) by routing processing. Send to. Regardless of which client 90 (90-1 to 4) inserts a KV pair, the servers 91 (91-1 to 6) serving as transmission destinations of all KV pairs are determined by keys.

  Patent Document 2 discloses a method for processing data stored in a database node constituting a distributed database system. In Patent Document 2, when processing key information and value pairs (KV pairs) distributed to distributed database nodes, a client node transmits program information to each database node. When data processing is executed in each database node, the client node acquires only the result of the processing in each database node. Therefore, the amount of data exchanged by one communication can be reduced.

  Moreover, in patent document 2, in order to make data processing efficient, possibility that the data required for one process will exist in a specific database node is raised. Therefore, according to Patent Document 2, the number of communications can be reduced when processing is performed on a large amount of distributed data.

JP-T-2005-503616 JP 2012-108889 A

  In general, data restoration targets a server configuration that has been backed up, and restoration for a different server configuration is not considered. The backup collected on each server must be restored to a system with the same configuration as when it was collected. Therefore, it is not always useful to apply differential backup or incremental backup as a backup of a scale-out system such as a distributed KVS that requires restoration for different server configurations.

  According to the technique of Patent Document 1, it is possible to restore backup data regardless of the server configuration before and after restoration. However, according to the technique of Patent Document 1, since a plurality of different data is processed through the data management server, there is a problem that processing time increases when a large amount of data is handled in a single process.

  According to the method of Patent Document 2, the number of communications in data processing can be reduced. However, processing of data distributed to a plurality of database nodes depends on the data amount and performance of each database node. For this reason, the processing time differs depending on the database node, and the time until one process is completed is not necessarily shortened. There is also a problem that it is difficult to reconstruct the server configuration because the data is biased to a specific database node.

  An object of the present invention is to provide a data processing system that restores backup data for multiple generations in a distributed KVS environment to a distributed KVS having an arbitrary server configuration in a single process without depending on the server configuration of the distributed KVS. .

  The data processing system of the present invention is a data processing system for restoring data having update information for determining an update order backed up in a distributed key-value store environment, and at least one of the first database server group. A backup means that backs up a key-value pair that contains the value of data stored in one server and a key that distinguishes the data in pairs, and at least one key-value pair backed up for each generation is read and read. A first key-value pair group is formed by aggregating at least one key-value pair issued with a key, and one key-value pair having the latest update information is selected from the first key-value pair group; Depending on the number of replications, the key-value pair with the latest update information selected Replicate Te, and an output means for outputting in accordance with the duplicated key-value pairs the predetermined node identifier.

  The data processing method of the present invention is a data processing method for restoring data having update information for determining an update order backed up in a distributed key-value store environment, and is at least one of the first database server group. A key-value pair containing a pair of a data value stored in one server and a key that distinguishes the data is backed up for each generation, and at least one key-value pair backed up for each generation is read and read The first key-value pair group is formed by aggregating at least one key-value pair with the key, and one key-value pair having the latest update information is selected from the first key-value pair group and selected. The key value pair with the latest update information is replicated according to the number of replications, and the replicated key And it outputs the Yupea according to a predetermined node identifier.

  The program recording medium of the present invention is a program recording medium for recording a data processing program for restoring data having update information for determining an update order backed up in a distributed key-value store environment, and the first database server A process for backing up each generation of a key-value pair including a pair of a data value stored in at least one server constituting the group and a key for distinguishing the data, and at least a key-value pair backed up for each generation The first key-value pair group is formed by aggregating at least one read-out key-value pair with a key, and a key-value pair having the latest update information from the first key-value pair group. Select one and replies the key-value pair with the latest update information selected Replicated according to the application number, and records the data processing program for executing a process of outputting to the computer a duplicated key-value pairs in accordance with a predetermined node identifier.

  According to the present invention, backup data over multiple generations in a distributed KVS environment can be restored to a distributed KVS having an arbitrary server configuration in a single process without depending on the server configuration of the distributed KVS.

1 is a functional block diagram showing a configuration of a distributed data processing system according to a first embodiment of the present invention. It is a conceptual diagram which shows the internal structure of the hardware which comprises the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows a part of structure of the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a conceptual diagram which shows an example of a structure of the KV pair processed in the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a conceptual diagram which shows an example of a structure of the KV pair processed in the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows a part of structure of the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the 1st processing means of the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the 2nd processing means of the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a conceptual diagram which shows the structure of the temporary KV pair produced in the distributed data processing system which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the flow of a process of the partial restore apparatus which concerns on the 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the distributed data processing system which concerns on the 2nd Embodiment of this invention. It is a conceptual diagram for demonstrating the distributed data processing system which concerns on the 2nd Embodiment of this invention. It is a conceptual diagram for demonstrating the process of the 1st processing means of the distributed data processing system which concerns on the 2nd Embodiment of this invention. It is a conceptual diagram for demonstrating the process of the 2nd processing means of the distributed data processing system which concerns on the 2nd Embodiment of this invention. It is a conceptual diagram for demonstrating the process of the 1st process means of the distributed data processing system which concerns on the 3rd Embodiment of this invention. It is a conceptual diagram for demonstrating the distributed data processing system which concerns on the 4th Embodiment of this invention. It is a conceptual diagram for demonstrating the distributed data processing system which concerns on the 4th Embodiment of this invention. It is a conceptual diagram for demonstrating a general client routing system.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated using drawing. However, the preferred embodiments described below are technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following.

(First embodiment)
FIG. 1 is a block diagram showing a functional configuration of a distributed data processing system 1 according to the first embodiment of the present invention. The distributed data processing system 1 according to the present embodiment is a data processing system that restores data having update information for determining an update order backed up in a distributed key-value store environment.

  The distributed data processing system 1 according to the first embodiment of the present invention includes a first database server group 10, a backup unit 20, a data restore device 30, and a second database server group 70.

  The data restore apparatus 30 according to the first embodiment of the present invention includes a partial restore means 40, a first processing means 50, and a second processing means 60. The data restoration device 30 is output means for restoring backup data of a plurality of servers constituting the first database server group 10 for each server, executing map reduction processing (hereinafter referred to as MapReduce processing) of a two-stage configuration, and outputting the same. is there. In the first embodiment of the present invention, it is assumed that backup data is acquired by differential backup or incremental backup. However, the backup method is not limited to the differential backup or the incremental backup, and may be a backup method other than the full backup.

  The distributed data processing system 1 according to the first exemplary embodiment of the present invention is configured by combining an information processing device such as a computer or a server machine and a storage device such as a memory or a storage.

  FIG. 2 is a conceptual diagram showing an example of the hardware configuration of the information processing apparatus used in the distributed data processing system 1 according to the first embodiment of the present invention. For example, as shown in FIG. 2, the information processing apparatus may have a (CPU: Central Processing Unit) configuration having a central processing unit 5 (described as CPU in FIG. 2). Further, as shown in FIG. 2, the information processing apparatus can be configured to have a ROM 6 and a RAM 7 (ROM: Read Only Memory, RAM: Random Access Memory). Further, as shown in FIG. 2, the information processing apparatus can be configured to include a storage device 8 (described as HDD in FIG. 2) and an input / output device 9 (described as I / O in FIG. 2) (HDD: Hard Disk Drive, I / O: Input / Output). That is, the information processing apparatus used in the distributed data processing system 1 according to the present embodiment has a configuration in which the above-described central processing unit 5, ROM 6, RAM 7, storage device 8, and input / output device 9 are connected by a data bus. It is realized by doing. Note that the information processing apparatus used in the distributed data processing system 1 according to the first embodiment of the present invention is not limited to the configuration in FIG. 2, and various devices and functions are added to or deleted from the configuration in FIG. It can be configured.

  Subsequently, components of the data processing system 1 according to the first embodiment of the present invention will be described in detail.

(First database server group)
The first database server group 10 shown in FIG. 3 includes a plurality of servers including a first server 11-1, a second server 11-2,..., An mth server 11-m (m is 2). Or an integer). Although FIG. 3 shows four servers as an example, the number of servers 11 is not limited to four. The number of servers 11 constituting the first database server group 10 may be at least two.

  The database server group 10 according to the present embodiment can be scaled out, and another server can be arbitrarily added to the plurality of servers 11 (11-1 to m) constituting the first database server group 10. That is, the database server group 10 according to the present embodiment may have a different configuration for each data backup timing.

  The distributed data processing system 1 forms a distributed KVS environment. In the distributed KVS environment, there are a plurality of servers 11 for storing data, and the data backup is performed in units of each server 11 (KVS: KEY-VALUE Store).

  In the distributed data processing system 1, data is stored in the form of a KV pair in which a numerical value called a key (KEY) is assigned to a stored value (VALUE). The data stored in the distributed data processing system 1 includes information (update information) related to the update order in which the KV pair is updated. As the update information, information in which the order of updating the KV pair such as update time (also referred to as update time) and vector clock (Vector Clock) is clarified is used. In this embodiment, a case where update time (Time) is used as update information will be described. Note that information regarding the timing (time / order) at which backup is executed may be used as the update information. The timing at which backup is executed may be included in the data instead of the update time (Time) of the KV pair, for example, or may be included in the data separately from the update time.

  As shown in FIG. 4, the KV pair handled in the data processing system 1 according to the first embodiment of the present invention includes a key (KEY) including key information (Key), update information (Time), and a value (Value). ) Including a value (VALUE). The KV pair handled in the data processing system 1 according to the first embodiment of the present invention may use information other than key information as a key, and may include information other than update information and data as a value. Good. When the backup timing is included in the data, for example, the update time (Time) in FIG. 4 may be replaced with the backup timing (Tn), or the backup timing (Tn) is added to the data separately from Time as shown in FIG. May be.

  Each server 11 included in the first database server group 10 is configured as an information processing apparatus such as a different server machine or storage. However, if a plurality of servers can be configured in a single device, a plurality of different servers can be configured in the same device.

  In FIG. 3 and the like, the first database server group 10 and the backup unit 20 are shown as different configurations, but the first database group 10 may include the backup unit 20.

  The data processing system 1 according to the present embodiment backs up data to the backup unit 20 at a predetermined timing. When backing up data, the backup unit 20 stores the backup data of each server 11 at each backup timing of each server 11.

  For example, in the data processing system 1 according to the present embodiment, data updated at a predetermined timing is backed up by differential backup or incremental backup. However, at the time of the first data backup, the data is backed up by a full backup.

  Note that the first database server group 10 and the second database server group 70 do not have to exist simultaneously as separate server groups, and the server 11 that constitutes the first database server group 10 includes the second database server group. 70 may constitute a server.

(Backup method)
As shown in FIG. 3, the backup means 20 according to the first embodiment of the present invention includes a first backup unit 21-1, a second backup unit 21-2, ..., an nth backup unit 21-n. It has a plurality of backup units 21 (n is an integer of 2 or more). Each backup unit 21 stores backup data whose backup timing is T1 to Tn for each generation. In FIG. 3, the data of the first to n servers 11-1 to 11-n included in the first database server group 10 are stored in the backup units 21 as backup data for each generation. In the distributed data processing system 1 according to the first embodiment of the present invention, the server configurations of the plurality of servers 11 that constitute the first database server group 10 may be different from each other in each generation. Data to be backed up in the backup unit 21 is stored so that it can be classified for each different server 11.

  When restoring the data backed up by the backup unit 20, the backup unit 20 transmits the backup data to the partial restore unit 40.

(Partial restore method)
The partial restore means 40 individually restores several generations of backup data with different backup timings stored in each backup unit 21 for each server 11. Such a restore is called a partial restore.

  The partial restore means 40 can be configured by, for example, a combination of partial servers that individually restore backup data for each server 11 for each generation. The partial server is a server that partially restores (loads) backup data of each server in each generation. In general, since the format of backup data is unique for each distributed KVS, it is assumed that it is difficult to directly read by the subsequent distributed processing framework. The partial server is arranged for the purpose of acquiring a KV pair of data backed up for each generation so that the distributed processing part does not depend on the format of the backup data.

  FIG. 6 shows an example in which a plurality of backup data stored in the first backup unit 21-1 and the second backup unit 21-2 are partially restored by the partial restore unit 40. In addition, although the example of two generations (T1 and T2) is given in the backup means 20 in FIG. 6, actually, backup data for many generations is stored. Further, each backup unit 21 in FIG. 6 shows only backup data for four servers, but each backup unit 21 includes as many backup data as the number of servers 11 that have been backed up for each generation. .

  In FIG. 6, the first backup unit 21-1 for storing backup data backed up at time T1 includes backup data of the first to m servers 11-1 to 11-m backed up at time T1. The backup data of the first to m servers 11-1 to 11-m backed up at time T1 is partially restored in the first to fourth partial servers 41-1 to 41-4, respectively.

  In FIG. 6, the second backup unit 21-2 that stores backup data backed up at time T2 includes backup data of the first to m servers 11-1 to 11-m backed up at time T2. The backup data of the first to m servers 11-1 to 11-m backed up at time T2 is partially restored in the fifth to eighth partial servers 41-5 to 8, respectively.

  FIG. 6 illustrates an example in which backup data of m servers 11 is partially restored to four partial servers 41, but in actuality, m pieces of servers 11 correspond to m servers 11. The partial server 41 is preferably arranged. Further, the backup data of the m servers 11 can be configured to be distributed to an arbitrary number of partial servers 41.

  The partial server 41 may have an internal configuration of each server 11 included in the first database server group 10 or a configuration included in an information processing apparatus different from each server 11. The partial server 41 may be configured as a program inside the information processing apparatus.

  Since the partial restore means 40 restores backup data for each different partial server 41, it is possible to restore backup data stored in different data structures in parallel.

  The data restored by the partial restoration means 40 is transmitted to the first processing means 50 and the second processing means 60 in the subsequent stage, and subjected to the two-stage MapReduce process. The first-stage MapReduce process by the first processing unit 50 is a process for selecting a KV pair with the latest update time (Time) in a certain key (KEY). The second RedReduce processing by the second processing means 60 is based on the server configuration of the second database server group 70 to be restored, and KV pairs are created for each server constituting the second database server group 70 in consideration of replication. This is a process for aggregation.

(First processing means)
The first processing means 50 for executing the first stage of the two-stage MapReduce process will be described with reference to FIG.

  The first processing means 50 includes a first Map processing unit 51 (hereinafter referred to as a first map processing unit 51), a first Shuffle processing unit 53 (hereinafter referred to as a first shuffle processing unit 53), and a first Reduce processing unit 55 (hereinafter referred to as a first processing unit). 1 reduction processing unit 55). The first processing means 50 executes parallel distributed processing using MapReduce as a framework. The first processing means 50 sorts the backup data acquired from the partial restore means 40 by the key (KEY), and outputs it as a KV pair group aggregated for each key.

  The first map processing unit 51 executes map processing (Map processing) for reading in parallel the KV pairs acquired from the partial servers 41. The first map processing unit 51 outputs the plurality of read KV pairs to the first shuffle processing unit 53.

  The first shuffle processing unit 53 sorts the plurality of KV pairs acquired from the first map processing unit 51 by the key information (Key) that is the key (KEY) of the KV pair, and collects the KV pairs for each key. A pair group (also referred to as a first key-value pair group) is used. The first shuffle processor 53 outputs the KV pair group (first key-value pair group) grouped for each key to the first reduce processor 55.

  The first reduction processing unit 55 has an update time (Time) from among the KV pairs constituting the KV pair group (first key-value pair group) grouped for each key acquired from the first shuffle processing unit 53. Select the latest KV pair. Normally, data is sorted and backed up by a plurality of servers, and there are a plurality of data having the same key and value. Therefore, the first reducing unit arbitrarily selects one of a plurality of KV pairs whose update times are the latest.

  Then, the first reduce processing unit 55 replicates (duplicates) the KV pair having the latest update time (Time) selected in each KV pair group (first key-value pair group) according to the number of replications. For example, if the number of replications is 3, three KV pairs having the same key (KEY) and value (VALUE) are replicated.

  Further, the first reduce processing unit 55 creates a temporary KV pair by assigning the device ID (also referred to as node ID or node identifier) of the server that is the restore destination of the replicated KV pair to all replicated KV pairs. To do. When the node ID (Node ID) of the server that is the storage destination of the KV pair is determined, a temporary KV pair shown in FIG. 9 is newly created. Normally, since distributed KVS performs replication, KV pairs corresponding to the number of replications with different storage destinations are created for one latest KV pair. Note that the replication and node ID assignment may be performed in the order in which replication is executed after the node ID is assigned.

  By the way, the server to be stored (restored) for each KV pair is appropriately determined according to the server configuration of the second database server group 70. It is assumed that a series of numbers such as device IDs are assigned to servers to be restored. For example, the storage destination server for the KV pair may be determined by a storage destination server determination algorithm for the KV pair unique to the distributed KVS. The correspondence between the KV pair and the storage destination server can be understood by the first reduce processing unit 55 of the first processing unit 50. For example, an algorithm or a correspondence table that determines which server the KV pair is allocated to may be implemented in the first reduce processing unit 55 as the reduction process.

  The first reduce processing unit 55 aggregates temporary KV pairs having the same node ID as a temporary KV pair group (first temporary key-value pair group), and outputs the aggregated temporary KV pairs to the second processing unit 60.

(Second processing means)
Next, the second processing means 60 that executes the second stage of the MapReduce process having a two-stage configuration will be described with reference to FIG.

  The second processing means 60 includes a second Map processing unit 61 (hereinafter referred to as a second map processing unit 61), a second Shuffle processing unit 63 (hereinafter referred to as a second shuffle processing unit 63), and a second Reduce processing unit 65 (hereinafter referred to as a second processing unit). 2 reduction processing unit 65). The second processing means 60 executes parallel distributed processing using MapReduce as a framework.

  First, the second processing means 60 acquires a temporary KV pair group aggregated for each key by the first processing means 50. Then, the second processing means 60 reconfigures the temporary KV pairs constituting the temporary KV pair group as KV pair groups for each server constituting the second database server group 70 in accordance with specific conditions and rules. The constructed KV pair group is output.

  The second map processing unit 61 executes map processing (Map processing) for reading a plurality of temporary KV pairs acquired from the first reduce processing unit 55 of the first processing unit 50 in parallel. The second map processing unit 61 outputs the plurality of read temporary KV pairs to the second shuffle processing unit 63.

  The second shuffle processing unit 63 sorts the plurality of temporary KV pairs acquired from the second map processing unit 61 according to the node ID (Node ID) that becomes the key (KEY) of the temporary KV pair. The second shuffle processing unit 63 sets a temporary KV pair group (second temporary key-value pair group) in which temporary KV pairs are grouped for each node ID. The second shuffle processor 63 outputs the temporary KV pair group (second temporary key-value pair group) grouped for each node ID to the second reduce processor 65.

  Since the node ID (NodeID) means the storage destination server number of the KV pair, the second reduction processing unit 65 aggregates the KV pairs having the same storage destination.

  The second reduce processing unit 65 is assigned to the temporary KV pair in the temporary KV pair group (second temporary key-value pair group) grouped for each node ID acquired from the second shuffle processing unit 63. The node ID is deleted and converted to a normal KV pair.

  Then, the second reduce processing unit 65 aggregates the KV pairs having a normal data structure as a KV pair group (second key-value pair group) grouped for each node ID, and sets the KV pair group (second key value group). Key-value pair group) is output to the server designated by the node ID.

  The second reduce processing unit 65 collects temporary KV pairs using the same node ID as a key (KEY), and extracts a KV pair that should be originally stored from the value (VALUE) of the temporary KV pair. Then, the second reduce processing unit 65 transmits the group of the extracted KV pairs to the storage destination server as a lump called a KV pair group (second key-value pair group). As a result, the use efficiency of the network can be improved by transferring a KV pair group (second key-value pair group) having a size larger than that of a single KV pair.

(Second database server group)
The second database server group 70 stores the KV pair acquired from the second reduce processing unit 65 of the second processing means 60. Similar to the first database server group 10, the second database server group 70 has a distributed KVS type server configuration. The second database server group 70 according to the first embodiment of the present invention can have a different server configuration from the first database server group 10. The second database server group 70 may have the same server configuration as the first database server group 10.

  This completes the description of the configuration of the distributed data processing system according to the first embodiment of the present invention. The configuration of the distributed data processing system according to the first embodiment of the present invention is not limited to the above-described configuration, and various additions / deletions and the like may be made as necessary.

(Operation)
Next, the operation of the distributed data processing system 1 according to the first embodiment of the present invention will be described. In particular, a partial restore process and a two-stage MapReduce process in the data restore apparatus 30 which is a characteristic part of the present invention will be described in order using the flowchart of FIG.

(Partial restore processing)
First, in the partial restore process of FIG. 10, the plurality of partial servers 41 included in the partial restore unit 40 acquires backup data of each server at each backup timing when the backup unit 20 backs up data. Then, each partial server 41 restores the acquired backup data (step S10).

  In the partial restore process, each partial server 41 holds only KV data of differential backup data (incremental backup data). In the partial restore process, all backup data stored in the backup unit 20 is individually restored (partial restore) to each partial server 41 regardless of the generation of the differential backup (incremental backup).

(MapReduce process in the first processing means)
Subsequently, in the first processing means 50, MapReduce processing, which is a distributed processing framework, is executed through the following steps S20 to S40.

  In the first map processing of FIG. 10, the first map processing unit 51 of the first processing unit 50 performs Map processing for reading in parallel a plurality of KV pairs restored by the plurality of partial servers 41 included in the partial restoration unit 40. It is executed (step S20).

  In the first map process, it is not necessary to consider the generation of data held by each partial server 41.

  In the first shuffle processing, the first shuffle processing unit 53 of the first processing means 50 selects a KV pair group (with the same key (KEY)) from among a plurality of KV pairs mapped by the first map processing unit 51 ( First key / value pair group) (step S30).

  In the first reduction process, first, the first reduction processing unit 55 of the first processing unit 50 updates the update time (Time) in the KV pair group (first key value pair group) grouped for each key. Is selected for each KV pair group. Second, the first reduce processing unit 55 replicates (replicas) the selected KV pair according to a predetermined number of replications. Third, the first reduce processing unit 55 assigns the node ID of the server that newly stores the KV pair to the replicated KV pair, and creates a temporary KV pair. Through the first reduce process including the above first to third procedures, the KV pair with the latest update time is selected for each key (KEY) from the partially restored backup data, and the replication of the KV pair and the node ID The assignment is executed (step S40). Note that the second and third procedures in the first reducing process may be executed in a reversed order. Temporary KV pairs having the same key (KEY) are aggregated as a temporary KV pair group (first temporary KV pair group) and output to the second processing means 60.

(MapReduce processing in the second processing means)
Subsequently, in the second processing means 60, MapReduce processing, which is a distributed processing framework, is executed through the following steps S50 to S70.

  In the second map processing of FIG. 10, the second map processing unit 61 of the second processing means 60 converts the temporary KV pair of each key (KEY) into a temporary KV pair group (first first) for each key (KEY). A Map process for reading in parallel as a key-value pair is executed (step S50).

  In the second shuffle process, the second shuffle processor 63 aggregates a plurality of KV pairs that have undergone the map process by the second map processor 61 for each temporary KV pair having the same node ID (NodeID). Then, the second shuffle processing unit 63 generates a temporary KV pair group (second temporary key-value pair group) using a plurality of KV pairs aggregated for each temporary KV pair having the same node ID (NodeID). Form (step S60).

  In the second reduction process, the second reduction processing unit 65 deletes the node ID (NodeID) of the temporary KV pair collected for each node ID (NodeID), and converts the node ID (NodeID) into a KV pair having a normal structure (step). S70). KV pairs having a normal structure are distributed to a designated server as a KV pair group (second key-value pair) aggregated for each same node ID.

  The above is a detailed description of the operation of the data restoration apparatus according to the present embodiment. Note that the operation of the data restoration apparatus according to the first embodiment is not limited to the above-described processing procedure, and various changes such as addition and deletion may be performed as necessary.

  In general, the distributed KVS has KV pair update time information in order to enable exclusive control during KV pair update. In the distributed data processing system according to the first embodiment of the present invention, the KV pair update time information is used to select the KV pair with the latest update time. As a result, differential / incremental backup data over multiple generations can be read and processed in parallel, and data restoration can be performed in a short time.

  In the distributed data processing system according to the first embodiment of the present invention, instead of directly restoring to each server of the distributed KVS at the time of data restoration, the data is restored once to the partial server, and the data from the partial server is restored. Is read directly. As a result, data restoration independent of the server configuration at the time of data backup becomes possible.

  Furthermore, in the MapReduce process of the distributed data processing system according to the first embodiment of the present invention, a primary KV pair is internally created by assigning a node ID indicating a storage destination server number to the original KV pair. . As a result, KV pairs can be aggregated in units of distributed KVS servers that restore data.

  In the Reduce process, the KV pairs having the same storage destination are collected, so that when the data is transmitted to the distributed KVS server, it can be transferred as a large size data, and the network can be used efficiently.

  That is, according to the distributed data processing system according to the first embodiment of the present invention, the differential backup data over multiple generations in the distributed KVS environment can be processed at any time in a single process without depending on the server configuration of the distributed KVS. It is possible to restore to the distributed KVS of the configuration.

  Further, according to the distributed data processing system according to the first embodiment of the present invention, it is possible to improve network transfer efficiency at the time of restoration by aggregating KV pairs in units of servers.

  By the way, the data processing method by the distributed data processing system according to the first embodiment of the present invention is included in the scope of the present invention even if it is realized with a different configuration from the distributed data processing system described so far. It is. Further, a data processing program for executing the data processing method of the present invention and a program recording medium for recording the data processing program are also included in the scope of the present invention.

  In the following, a more specific embodiment of the distributed data processing system according to the first embodiment of the present invention will be described.

(Second Embodiment)
11 and 12 are conceptual diagrams of a distributed data processing system according to the second embodiment. FIG. 13 and FIG. 14 are conceptual diagrams for explaining processing of each component in the second embodiment.

  As shown in FIGS. 11 and 12, the distributed data processing system according to the second embodiment includes a backup unit 200, a partial restore unit 400, a first processing unit 500, and a second processing unit 600. FIG. 12 shows an example in which the second processing unit 600 distributes the KV pairs that have been MapReduce processed by the distributed data processing system according to the second embodiment to each server 710 included in the second database server group 700. . The second database server group 700 may be configured to be included in the distributed data processing system according to the second embodiment, or may be configured not to be included.

  The backup means 200 includes a first backup unit 210-1 that stores a KV pair backed up at time T1, and a second backup unit 210-2 that stores a KV pair backed up at time T2. In the second embodiment, the number of replications is 3.

  As shown in FIG. 11, the first backup unit 210-1 is differentially backed up for KV pairs including Key1 and Key6 with respect to the first server, the third server, the fourth server, the fifth server, and the sixth server. . The second backup unit 210-2 is differentially backed up for KV pairs including Key1 with respect to the first server, the third server, and the fourth server.

  At the timing of restoring the backed up KV pairs, the partial restore unit 400 configures a plurality of partial servers 410 (410-1 to 8) corresponding to each KV pair stored in the backup unit 200. In the second embodiment, first to eighth partial servers 410-1 to 410-8 are configured, and a KV pair is acquired in each partial server 410. In each partial server 410, data is individually restored (partial restore). As a result, the data is restored so as to correspond to an arbitrary server configuration without depending on the server configuration when the differential backup is performed.

  The KV pair partially restored by each partial server 410 is output toward the first processing means 500.

  As shown in FIG. 13, in the first processing means 500, the first map processing unit 510 executes Map processing on the KV pair in parallel. Then, the first map processing unit 510 outputs the plurality of KV pairs to the first shuffle processing unit 530.

  The first shuffle processing unit 530 sorts a plurality of KV pairs by a key (KEY), aggregates a plurality of KV pairs having the same key (KEY), and a KV pair group (first key-value pair group). To do. Then, the first shuffle processing unit 530 outputs the aggregated KV pair group (first key-value pair group) toward the first reduce processing unit 550.

  The first reduce processing unit 550 selects one KV pair having the latest update time (Time) from each KV pair group (first key-value pair group). The first reduce processing unit 550 replicates (replicates) three selected KV pairs according to the number of replications (3 in the second embodiment). The first reduce processing unit 550 creates temporary KV pairs by assigning node IDs of storage destination servers to all replicated KV pairs. The first reduce processing unit 550 collects temporary KV pairs for each node ID and forms a temporary KV pair group (first temporary key-value pair group). Then, the first reduce processing unit 550 outputs the temporary KV pair group (first temporary key-value pair group) to the second processing unit 600.

  As shown in FIG. 14, in the second processing means 600, the second map processing unit 610 performs a plurality of temporary KV pairs collected as a temporary KV pair group (first temporary key-value pair group). The Map process is executed in parallel. Then, the second map processing unit 610 outputs a plurality of temporary KV pairs to the second shuffle processing unit 630.

  The second shuffle processing unit 630 rearranges a plurality of temporary KV pairs using the node ID (NodeID) as a key (KEY), and collects temporary KV pairs having the same node ID (NodeID) as a temporary KV pair. Group (second temporary key-value pair group). Then, the second shuffle processing unit 630 outputs the temporary KV pair group (second temporary key-value pair group) to the second reduce processing unit 650.

  The second reduce processing unit 650 deletes the node ID (NodeID) from the plurality of temporary KV pairs collected for each node ID. And the 2nd reduction process part 650 outputs the KV pair group (2nd key value pair group) containing the several KV pair put together for every node ID (NodeID) toward the server corresponding to node ID. .

  As shown in FIG. 14, in the second embodiment, a KV pair including Key1 and Key6 is stored in the first 'server 710-1 (node ID1). Also, KV pair including Key1 in the 3 ′ and 5 ′ servers 710-3 and 5 (node IDs 3 and 5), and Key6 in the 4 ′ and 6 ′ servers 710-4 and 6 (node IDs 4 and 6). The containing KV pair is stored. Each server 710 (710-1 to 710-1) included in the second database server group 700 stores the KV pair with the latest update time (Time).

(Third embodiment)
The third embodiment is an example when replication fails during backup.

  As shown in FIG. 15, the first processing unit 500 of the distributed data processing system according to the third embodiment includes a first map processing unit 512, a first shuffle processing unit 532, and a first reduce processing unit 552. Have. In addition, about the 1st processing means 500 of 3rd Embodiment, although the code | symbol of an internal structure differs from 2nd Embodiment, the same code | symbol as 2nd Embodiment is used. In addition, since the other components of the third embodiment are the same as those of the second embodiment, description thereof is omitted.

  The third embodiment shown in FIG. 15 is an example in which only two KV pairs are found even though the number of replications is three. Normally, if the number of replications is 3, three KV pairs should be found. However, in FIG. 15, there are only two KV pairs whose update time (Time) is 4 among the KV pairs whose Key is 1.

  Normally, the distributed KVS replicates one KV pair to different servers for the purpose of ensuring fault tolerance. Therefore, if the number of replications is 3 in the Reduce process, three same KV pairs should be stored. However, in the third embodiment, replication could not be performed due to some failure or the like, and only two identical KV pairs were found.

  Also in the case of the third embodiment, only one KV pair with the latest update time (Time) is selected. Therefore, it can be seen that the distributed data processing system according to the third embodiment operates without any problem even if the expected number of KV pairs does not exist.

  As described above, in the distributed data processing system according to the third embodiment of the present invention, it is possible to correct inconsistencies between remote data in replication by using data update information. Therefore, replication failure during backup does not become a problem.

(Fourth embodiment)
As shown in FIGS. 16 and 17, the distributed data processing system according to the fourth embodiment includes a backup unit 201, a partial restore unit 401, a first processing unit 501, a second processing unit 601, and a second database server group. 701.

  The fourth embodiment is an example when the database is scaled out. In the example of the first database server group 101 in FIG. 16, the 1-1 database server group 101-1 and the 1-2 database server group 101-2 are distinguished from each other before and after the scale-out. Actually, the configuration in which the fourth server 112-4 is added to the first to third servers 111-1 to 111-3 constituting the first 1-1 database server group 101-1, is the first-2 database server group 101-. 2 and the server configuration is changed by the scale-out.

  In the first stage, the database server group 101 storing the KV pair has a server configuration like the 1-1 database server group 101-1 in FIG. In the first stage, backup data of the 1-1 database server group 101-1 is stored in the first backup unit 211-1.

  Next, in a second stage different from the first stage, the database server group 101 storing the KV pairs is scaled out to a server configuration like the 1-2 database server group 101-2 in FIG. In the second stage, the backup data of the 1-2 database server group 101-2 is stored in the second backup unit 211-2.

  At the timing of restoring the backup data, the partial restore unit 401 obtains the backup data in parallel from the first and second backup units 211-1 and 211-2 storing the backup data of the database server groups 101 having different server configurations. . The partial restore unit 401 configures the first to seventh partial servers 411-1 to 7-7 in correspondence with the configurations of the servers 111 and 112 at each backup timing when the backup unit 201 backs up data. Then, the partial restore unit 401 individually restores (partial restore) backup data in each partial server 411.

  As shown in FIG. 17, the KV pair partially restored by the partial restore unit 401 is processed by the first processing unit 501 and the second processing unit 601 and restored to the second database server group 701. In the fourth embodiment, KV pairs are allocated to the first 'to eighth' servers 711-1 to 8.

  In the distributed data processing system according to the fourth embodiment of the present invention, the storage destination of the KV pair having a certain key (KEY) is associated with the first reduce process, so that the server configuration at the time of backup is It is possible to restore without depending on it.

  Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A data processing system for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
Backup means for backing up a key-value pair for each generation including the value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data;
At least one key-value pair backed up for each generation is read, and the first key-value pair group is formed by aggregating at least one read-out key-value pair with the key, and the first key-value pair group is formed. One key-value pair having the latest update information is selected from the key-value pair group, the selected key-value pair having the latest update information is duplicated according to the number of replications, and the duplicated key-value pair A data processing system comprising: output means for outputting the information according to a predetermined node identifier.
(Appendix 2)
The backup means includes
The data according to appendix 1, wherein when the data stored in the server constituting the first database server group is updated, the key-value pair including the updated data is differentially backed up. Processing system.
(Appendix 3)
The backup means includes
The supplementary note 1 or 2, wherein when the data stored in the server constituting the first database server group is updated, the key-value pair including the updated data is incrementally backed up. Data processing system.
(Appendix 4)
The output means includes
Receiving a key-value pair backed up for each generation from the backup means, and comprising a partial restore means for restoring the received key-value pair backed up for each generation for each generation;
The partial restoration means includes
4. The apparatus according to claim 1, further comprising at least one partial server that restores the key-value pair backed up for each generation for each of the servers constituting the first database server group. 5. Data processing system.
(Appendix 5)
The output means includes
The data processing system according to any one of appendices 1 to 4, wherein the duplicated key-value pairs are aggregated and output for each node.
(Appendix 6)
The output means includes
A first map processing unit for reading in parallel at least one restored key-value pair;
A first shuffle processing unit that forms a first key-value pair group in which the key-value pairs read out by the first map processing unit are aggregated by the key;
The key value pair having the latest update information is selected from the key value pairs constituting the first key value pair group formed by the first shuffle unit, and the selected key value pair is selected. Duplicating according to the number of replications, creating a temporary key-value pair by assigning the node identifier to all of the duplicated key-value pairs, and adding at least one temporary key-value pair having the same key A first reduction processing unit that aggregates and outputs the aggregated at least one temporary key-value pair as a first temporary key-value pair group;
A second map processing unit that reads in parallel at least one of the temporary key-value pairs constituting the first temporary key-value pair group output by the first reduce processing unit;
A second shuffle processor that aggregates the temporary key-value pairs read out by the second map processor by the node identifier to form a second temporary key-value pair group;
The node identifier is deleted from the temporary key-value pair that constitutes the second temporary key-value pair group formed by the second shuffle processing unit, and the second temporary key-value pair group is configured. The at least one key-value pair that has been collected is aggregated for each node identifier to form a second key-value pair group, and the second key-value pair group is assigned to the node that constitutes the second database server group. A data processing system according to any one of appendices 1 to 5, further comprising: a second reduction processing unit that outputs in accordance with the identifier.
(Appendix 7)
When the first database server group changes the server configuration,
The backup means includes
The data processing system according to any one of appendices 1 to 6, wherein the key-value pair is backed up corresponding to the update information in accordance with a change in a server configuration of the first database server group.
(Appendix 8)
The data processing system according to any one of appendices 1 to 7, wherein the output unit assigns the node identifier to the duplicated key-value pair according to a specific algorithm.
(Appendix 9)
The output means includes
The data processing system according to any one of appendices 1 to 8, wherein an update time included in the data is used as the update information.
(Appendix 10)
The backup means includes
Adding information about the timing of the backup to the data,
The output means includes
10. The data processing system according to any one of appendices 1 to 9, wherein information regarding a timing at which the backup is executed is used as the update information.
(Appendix 11)
A data processing method for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
Back up a key-value pair for each generation including a pair of a value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data,
At least one key-value pair backed up for each generation is read, and the first key-value pair group is formed by aggregating at least one read-out key-value pair with the key, and the first key-value pair group is formed. One key-value pair having the latest update information is selected from the key-value pair group, the selected key-value pair having the latest update information is duplicated according to the number of replications, and the duplicated key-value pair Is output in accordance with a predetermined node identifier.
(Appendix 12)
A program recording medium for recording a data processing program for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
A process of backing up a key-value pair for each generation including the value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data;
At least one key-value pair backed up for each generation is read, and the first key-value pair group is formed by aggregating at least one read-out key-value pair with the key, and the first key-value pair group is formed. One key-value pair having the latest update information is selected from the key-value pair group, the selected key-value pair having the latest update information is duplicated according to the number of replications, and the duplicated key-value pair A program recording medium for recording a data processing program that causes a computer to execute a process of outputting a program according to a predetermined node identifier.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2013-188460 for which it applied on September 11, 2013, and takes in those the indications of all here.

1 Distributed Data Processing System 5 Central Processing Unit 6 ROM
7 RAM
8 Storage Device 9 Input / Output Device 10 First Database Server Group 11 Server 20 Backup Unit 21 Backup Unit 30 Data Restore Device 40 Partial Restore Unit 41 Partial Server 50 First Processing Unit 51 First Map Processing Unit 53 First Shuffle Processing Unit 55 First reduction processing unit 60 Second processing means 61 Second map processing unit 63 Second shuffle processing unit 65 Second reduction processing unit 70 Second database server group

Claims (10)

A data processing system for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
Backup means for backing up a key-value pair for each generation including the value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data;
At least one key-value pair backed up for each generation is read, and the first key-value pair group is formed by aggregating at least one read-out key-value pair with the key, and the first key-value pair group is formed. One key-value pair having the latest update information is selected from the key-value pair group, the selected key-value pair having the latest update information is duplicated according to the number of replications, and the duplicated key-value pair Output means for outputting according to the node identifier of the server as the storage destination. The backup means includes
The differential backup of the key-value pair including the updated data is performed when the data stored in the server constituting the first database server group is updated. Data processing system. The backup means includes
3. The key-value pair including the updated data is incrementally backed up when the data stored in the server constituting the first database server group is updated. The data processing system described. The output means includes
Receiving a key-value pair backed up for each generation from the backup means, and comprising a partial restore means for restoring the received key-value pair backed up for each generation for each generation;
The partial restoration means includes
4. The apparatus according to claim 1, further comprising at least one partial server that restores the key-value pair backed up for each generation for each of the servers constituting the first database server group. 5. The data processing system described. The output means includes
5. The data processing system according to claim 1, wherein the duplicated key-value pairs are aggregated and output for each node identifier. The output means includes
A first map processing unit for reading in parallel at least one restored key-value pair;
A first shuffle processing unit that forms a first key-value pair group in which the key-value pairs read out by the first map processing unit are aggregated by the key;
The key value pair having the latest update information is selected from the key value pairs constituting the first key value pair group formed by the first shuffle processing unit, and the selected key value pair is selected. According to the number of replications, and assigning the node identifier to all of the duplicated key-value pairs to create a temporary key-value pair, and at least one temporary key-value pair having the same key A first reduction processing unit that outputs the aggregated at least one temporary key-value pair as a first temporary key-value pair group;
A second map processing unit that reads in parallel at least one of the temporary key-value pairs constituting the first temporary key-value pair group output by the first reduce processing unit;
A second shuffle processor that aggregates the temporary key-value pairs read out by the second map processor by the node identifier to form a second temporary key-value pair group;
The node identifier is deleted from the temporary key-value pair that constitutes the second temporary key-value pair group formed by the second shuffle processing unit, and the second temporary key-value pair group is configured. The at least one key-value pair that has been collected is aggregated for each node identifier to form a second key-value pair group, and the second key-value pair group is assigned to the node that constitutes the second database server group. The data processing system according to claim 1, further comprising: a second reduction processing unit that outputs according to the identifier. When the first database server group changes the server configuration,
The backup means includes
The data processing system according to any one of claims 1 to 6, wherein the key-value pair is backed up corresponding to the update information in accordance with a change in a server configuration of the first database server group.   8. The data processing system according to claim 1, wherein the output unit assigns the node identifier to the duplicated key-value pair according to a specific algorithm. A data processing method for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
A backup means backs up a key-value pair including a value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data in pairs for each generation,
The output means reads at least one key-value pair backed up for each generation, and aggregates at least one read-out key-value pair with the key to form a first key-value pair group, One key-value pair having the latest update information is selected from the first key-value pair group, and the selected key-value pair having the latest update information is replicated according to the number of replications. A data processing method comprising: outputting the key-value pair according to a node identifier of a server as a storage destination. A data processing program for restoring data having update information for determining an update order backed up in a distributed key-value store environment,
A process of backing up a key-value pair for each generation including the value of the data stored in at least one server constituting the first database server group and a key for distinguishing the data;
At least one key-value pair backed up for each generation is read, and the first key-value pair group is formed by aggregating at least one read-out key-value pair with the key, and the first key-value pair group is formed. One key-value pair having the latest update information is selected from the key-value pair group, the selected key-value pair having the latest update information is duplicated according to the number of replications, and the duplicated key-value pair A data processing program for causing a computer to execute a process of outputting a message according to a node identifier of a server as a storage destination.

Images