JP2016134116A - Information processing apparatus, information processing system, information processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for more sufficiently supporting result consistency between a record and an index in a distributed storage system.SOLUTION: An information processing apparatus includes: a processing request input part 11 for receiving a request for processing about a record comprising a key and a value; a message generation part 13 for generating an index update message representing an update instruction corresponding to the requested processing about index data comprising an index key and a key list; a record update part 12 for generating a record corresponding to the requested processing, adding the index update message to the generated record, and storing the record with the index update message added thereto in a distribution storage system 40; and an index data update part 14 for storing index data with the index update message reflected thereon in the distributed storage system 40 by executing the index update message included in the record stored in the distribution storage system 40.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for managing data and indexes stored in a distributed storage system.

  Distributed storage systems that store data in a distributed manner are known. As a distributed storage system, for example, there is a distributed key-value store (distributed KVS: Key-Value Store) that stores data consisting of keys and values in a distributed manner. The distributed KVS distributes the server that stores the key and value pairs (hereinafter referred to as KV pairs) based on the hash value of the key. In the distributed KVS, KV pairs are distributed almost uniformly on each server.

  In distributed KVS, atomicity is supported for one data operation on one KV pair. Examples of data operations include creation, update, deletion, or reference. On the other hand, in the distributed KVS, atomicity is not supported for a series of data operations for one KV pair. For example, when a series of data operations such as “update a GET KV pair and PUT” is requested from a plurality of applications, atomicity is not supported for each series of data operations. Therefore, in the distributed KVS, an exclusive control technique called a CAS (Compare And Swap) function is adopted. With the CAS function, a series of data operations on the same KV pair can be executed atomically. However, execution of exclusive control by CAS may fail due to optimistic exclusive control. An application that requested a series of data operations that failed to execute again requests execution of the series of data operations.

  There is a case where such data on the distributed KVS is allowed to be eventually consistent. For example, eventual consistency is allowed between data consisting of keys and values and an index for the data. A specific example will be described below.

For example, a “product master” and “order history” as shown in FIG. 19 are assumed. Here, information on each line included in the “product master” and the “order history” is referred to as a record. In FIG. 19, the primary key is underlined and the external key is underlined by a broken line. In this case, in the distributed KVS, the product master and the order history are stored as a KV pair shown in FIG. Hereinafter, the KV pair corresponding to each record of “product master” and “order history” is also referred to as “record KV pair”. In FIG. 20, “key → value” represents a KV pair. {Value 1, value 2,...} Represents the value of the record KV pair. These record KV pairs are created by the following processes 1 to 3.
Process 1: Records relating to product information (“product ID”, “stock quantity”, “price”) are registered in the “product master”.
Process 2: When there is an order for a product, a record (“order ID”, “product ID”, “order date”) related to order information is registered in “order history”.
・ Process 3: Reduce the inventory quantity of the product in the product master by 1 by allocating the product by order.

  Now, consider creating an index in the “Order Date / Time” column of “Order History”. In this case, in the distributed KVS, the index is stored as an index KV pair shown in FIG. In FIG. 21, [key 1, key 2,...] Represents the value of the index KV pair. The value of the index KV pair represents a list of keys of the record KV pair that includes the key of the index KV pair in the value. At this time, it is assumed that the update of the record KV pair and the update of the index KV pair are performed in separate transactions. If no failure or the like occurs, even if a mismatch occurs temporarily between the record and the index, the index KV pair is correctly generated as a result. That is, records and indexes satisfy eventual consistency.

  However, if a failure occurs between the update of the record KV pair and the update of the index KV pair, the data of the index KV pair is lost. As a result, the eventual consistency between records and indexes is not satisfied. As described above, there are cases where reference, update, and deletion using the values included in the record KV pair as indexes cannot be performed unless the consistency between the records and the indexes is satisfied. If the key of the KV pair to be accessed is unknown, data access to that KV pair is lost. In this case, it is necessary to rebuild the index by scanning all cases of all servers. However, index reconstruction by full scan is not preferable in distributed KVS that handles large amounts of data.

  For this reason, in many distributed storage systems, each server creates and holds an index for data stored in itself. Thereby, the system facilitates ensuring consistency. In addition, each server creating and maintaining an index for data stored in itself secures “consistency” stronger than “eventual consistency” between records and indexes.

  However, in a distributed storage system, when each server creates an index for records stored in itself, there are the following problems. In this case, in the search using the index, all the servers need to check whether the corresponding data exists using their own index. Therefore, when there are many search processes using indexes, the index confirmation process also increases as the number of servers increases. For this reason, there was a problem that performance could not be improved by increasing the number of scales.

  Even when there are many search processes using indexes, in order to scale out performance by increasing the number, it is necessary to distribute not only record KV pairs but also index KV pairs to a plurality of servers. In this case, each server only needs to perform processing when holding the index to be searched. An example of a distributed arrangement of record KV pairs and index KV pairs is shown in FIG. In FIG. 22, the distributed KVS is configured by four servers 900a to 900d. The servers 900a to 900d store record KV pairs and index KV pairs in a distributed manner. In this case, the application accesses, for example, the three servers 900a, 900b, and 900c to refer to the record of the order history received on “2014-01-01” (January 1, 2014). Just do it. That is, in this case, an index KV pair with “2014-01-01” as a key can be acquired from the server 900a. Then, the record KV pair using “order 1” and “order 2” obtained as a key can be acquired from the servers 900b and 900c. As described above, by distributing the index KV pairs in a distributed manner, as the number of servers constituting the distributed KVS increases, the ratio of the number of servers accessed in one search process can be reduced.

  In order to realize such a distributed arrangement of index KV pairs, it is necessary to prevent inconsistency between the update of the record KV pair and the update of the index KV pair. An example of a technique for updating a plurality of KV pairs without causing inconsistency is described in Patent Document 1.

  In the related technique described in Patent Document 1, any one of the keys of each KV pair is set as a representative key in a KV pair group that is a target for maintaining consistency. Then, this related technique writes update information for each KV pair in the representative log corresponding to the representative key. In this related technology, information indicating that processing is in progress is added to the subordinate log other than the representative log. The representative log has a WAL (Write Ahead Log) role of the transaction. This related technique reflects the update information written in the representative log in each KV pair. As a result, this related technology makes it possible to perform update without causing inconsistency between a plurality of KV pairs. For example, in the example of FIGS. 19 to 21 described above, it is necessary to execute the above-described processing 2 and processing 3 atomically. That is, in FIG. 20, it is necessary to maintain consistency between the KV pair of the product master using item1 as a key and the KV pair of the order history using order1 as a key. Therefore, this related technology sets, for example, item1 as a representative key of these KV pairs. Thereby, the process 2 and the process 3 are executed atomically, and the KV pair of the product master and the KV pair of the order history are kept consistent.

  It is conceivable to maintain consistency between records and indexes by setting a representative key for the record KV pair and the corresponding index KV pair using this related technique.

JP 2012-238061 A

  However, even if the related technique described in Patent Document 1 is used, there is a case where consistency between records and indexes cannot be maintained.

  The reason will be described using the examples shown in FIGS. When the related technique is applied to this specific example, it is necessary to set the same representative key for the record KV pair and the index KV pair that are targets for maintaining consistency. Here, it is assumed that the representative key “item1” is set to the index KV pair having “2014-01-01” as a key. In this case, this related technique can update the index KV pair in the same transaction as the creation of the record KV pair with “Order 1” as a key. This is because “order 1” is set with the same representative key “item1”. However, in this case, this related technology cannot update the index KV pair in the same transaction as the creation of the record KV pair with “Order 2” as a key. This is because “order 2” is set with a different representative key “item2”.

  Therefore, the representative key “2014-01-01” is set to the index KV pair having “2014-01-01” as a key and the record KV pairs having “order 1” and “order 2” as keys. And In this case, according to this related technique, the update of each record KV pair in the order history and the update of the index KV pair can be executed in the same transaction. However, in this case, in this related technique, since the representative key is different between the KV pair of the order history and the KV pair of the corresponding product master, these updates cannot be executed in the same transaction.

  If the same representative key can be set for the record KV pair and the index KV pair that need to maintain consistency, no inconsistency will occur if the related technique is used. However, as described above, this related technique cannot match the representative key of the target KV pair group depending on the process, and cannot process the same transaction. In this case, the update of the record KV pair and the update of the index KV pair are executed in different transactions. At this time, if a failure occurs between the update transaction of the record KV pair and the update transaction of the index KV pair, the index data is lost. Then, the inconsistency remains between the record KV pair and the index KV pair. That is, in such a case, the distributed KVS does not satisfy the eventual consistency.

  As described above, the related technique described in Patent Document 1 has a problem that it may not be possible to maintain the result consistency between records and indexes.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a technique that more fully supports eventual consistency between records and indexes in a distributed storage system.

  An information processing apparatus according to the present invention includes: a processing request input unit that receives a processing request related to a record including a key and a value; an index key that is an index creation target among values included in the record; and a key of the record A message generation unit that generates an index update message that represents an update instruction according to the requested process, and a record that corresponds to the requested process is generated with respect to the index data including the key list, and the generated record A record update unit for adding the index update message generated by the message generation unit and storing the record to which the index update message is added in a distributed storage system; and an index included in the record stored in the distributed storage system By executing the update message, the search reflecting the index update message is performed. Comprising the index data updating unit that stores data in the distributed storage system, the.

  An information processing system according to the present invention includes the information processing apparatus described above and the distributed storage system.

  Further, in the information processing method of the present invention, when a processing request relating to a record including a key and a value is input, an index key that is an index creation target among values included in the record, and a key including the key of the record An index update message representing an update instruction corresponding to the requested process is generated for the index data consisting of a list, a record corresponding to the requested process is generated, and the index update message is added to the generated record. And adding the index update message to the distributed storage system, and executing the index update message included in the record stored in the distributed storage system to reflect the index update message. Data is stored in the distributed storage system.

  The computer program of the present invention includes a processing request input step in which a processing request relating to a record including a key and a value is input, an index key to be indexed among values included in the record, A message generation step for generating an index update message indicating an update instruction corresponding to the requested process, and a record corresponding to the requested process are generated and generated with respect to the index data including a key list including keys. Included in the record stored in the distributed storage system and the record update step of adding the index update message generated in the message generation step to the record and storing the record with the index update message in the distributed storage system By executing the index update message The index update message to execute the index data updating step of storing the index data reflected in the distributed storage system, to the computer device.

  The present invention can provide a technology that more fully supports eventual consistency between records and indexes in a distributed storage system.

It is a functional block diagram of an information processing system as a 1st embodiment of the present invention. It is a figure which shows an example of the hardware constitutions of the information processing system as the 1st Embodiment of this invention. It is a flowchart which shows the record processing operation | movement of the information processing system as the 1st Embodiment of this invention. It is a flowchart which shows the index data update operation | movement of the information processing system as the 1st Embodiment of this invention. It is a functional block diagram of an information processing system as a 2nd embodiment of the present invention. It is a figure explaining an example of the correspondence of the processing request inputted in the 2nd embodiment of the present invention, the index update message generated, and the processing performed. It is a flowchart explaining the record reference operation | movement of the information processing system as the 2nd Embodiment of this invention. It is a flowchart explaining the record processing operation | movement of the information processing system as the 2nd Embodiment of this invention. It is a figure which shows the specific example of the record processed by the information processing system as the 2nd Embodiment of this invention. FIG. 10 is a diagram illustrating an example of a correspondence relationship between a process request input to the record illustrated in FIG. 9, a generated index update message, and a process to be executed in the second embodiment of the present invention. . It is a flowchart explaining the index data update operation | movement of the information processing system as the 2nd Embodiment of this invention. In the 2nd Embodiment of this invention, it is a figure which illustrates typically an example of the information stored in distribution KVS before index update. In the 2nd Embodiment of this invention, it is a figure which illustrates typically an example of the information stored in distribution KVS after index update. It is a functional block diagram of the information processing system as the 3rd Embodiment of this invention. It is a figure which shows an example of the merge rule in the 3rd Embodiment of this invention. It is a figure which shows another example of the merge rule in the 3rd Embodiment of this invention. It is a flowchart explaining the message extraction operation | movement of the information processing system as the 3rd Embodiment of this invention. It is a flowchart explaining the index data update operation | movement of the information processing system as the 3rd Embodiment of this invention. It is a figure which shows an example of the information used as the storing object to distributed KVS in related technology. It is a figure which shows an example of the KV pair showing the information shown in FIG. 19 in related technology. It is a figure which shows an example of the index with respect to the KV pair shown in FIG. 20 in related technology. In related technology, it is a figure which illustrates typically an example of the index KV pair arranged in a distributed manner.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 shows the configuration of an information processing system 1 as a first embodiment of the present invention. In FIG. 1, the information processing system 1 includes an information processing apparatus 10 and a distributed storage system 40. The information processing apparatus 10 is communicably connected to the distributed storage system 40. The information processing apparatus 10 includes a processing request input unit 11, a record update unit 12, a message generation unit 13, and an index data update unit 14. The distributed storage system 40 includes a plurality of servers 400. Each server 400 includes a record storage unit 401.

  Here, FIG. 2 shows an example of the hardware configuration of each device constituting the information processing system 1. The information processing apparatus 10 can be configured by a computer device including a CPU (Central Processing Unit) 1001, a RAM (Random Access Memory) 1002, a ROM (Read Only Memory) 1003, and a storage device 1004 such as a hard disk. Further, the computer device constituting the information processing apparatus 10 further includes a network interface 1005, an input device 1006, and an output device 1007. Each server 400 included in the distributed storage system 40 can be configured by a computer device including a CPU 4001, a RAM 4002, a ROM 4003, a storage device 4004 such as a hard disk, and a network interface 4005.

  In this case, the processing request input unit 11 includes an input device 1006 and a CPU 1001 that reads a computer program stored in the ROM 1003 or the storage device 1004 into the RAM 1002 and executes it. The record update unit 12 and the index data update unit 14 include a network interface 1005 and a CPU 1001 that reads a computer program stored in the ROM 1003 or the storage device 1004 into the RAM 1002 and executes it. The message generation unit 13 is configured by a CPU 1001 that reads a computer program stored in the ROM 1003 or the storage device 1004 into the RAM 1002 and executes it. The distributed storage system 40 realizes its function in each server 400 by the CPU 4001 reading the computer program stored in the ROM 4003 or the storage device 4004 into the RAM 4002 and executing it. The record storage unit 401 is configured by a storage device 4004.

  Note that the information processing apparatus 10 may be realized on the same computer apparatus as any of the servers 400. The hardware configuration of each device and each functional block of the information processing system 1 is not limited to the above-described configuration.

  The distributed storage system 40 stores records consisting of keys and values in a distributed manner in the record storage units 401 of the plurality of servers 400. In response to a request from the information processing apparatus 10, the distributed storage system 40 specifies the storage destination server 400 for the record to be processed. Then, the distributed storage system 40 executes the requested process in the identified server 400.

  Next, details of each functional block of the information processing apparatus 10 will be described.

  The processing request input unit 11 acquires a processing request related to a record including a key and a value as an input. Here, the process related to the record may be, for example, a process such as new generation, update, or deletion of the record. For example, the processing request input unit 11 may acquire information indicating the processing request content from the input device 1006 or the like.

  The record update unit 12 generates a record corresponding to the processing request input to the processing request input unit 11. Here, in the distributed storage system 40, there is a case where related index data needs to be updated in accordance with the record processing. Accordingly, when the index update unit 12 needs to update the index data, the record update unit 12 generates an index update message representing an index data update instruction according to the record processing, using a message generation unit 13 described later. Then, the record update unit 12 adds an index update message to the generated record and stores it in the distributed storage system 40.

  The message generation unit 13 generates an index update message corresponding to the processing content of the record. As described above, the index update message represents an index data update instruction. Here, the index data includes an index key and a key list. Of the values included in the record, the index creation target value is set in the index key. The key list is a list of keys of records that include the index key as a value.

  The index data update unit 14 executes an index update message included in a record stored in the distributed storage system 40. Then, the index data update unit 14 stores the index data in which the index update message is reflected in the distributed storage system 40. In this way, the index data update unit 14 updates the index data asynchronously with the record processing by the record update unit 12. The index data update unit 14 may operate at every predetermined timing. Alternatively, the index data update unit 14 may operate at the timing when the record update unit 12 processes the record.

  The operation of the information processing system 1 configured as described above will be described with reference to the drawings.

  First, an outline of the record processing operation of the information processing system 1 is shown in FIG.

  In FIG. 3, the process request input unit 11 first acquires a process request relating to a record as an input (step A1).

  Next, the record update unit 12 generates a record corresponding to the process requested in Step A1 (Step A2).

  Next, the record update unit 12 generates an index update message corresponding to the process requested in step A1 using the message generation unit 13 (step A3).

  Next, the record update unit 12 adds the index update message generated in step A3 to the record generated in step A2 (step A4).

  Next, the record update unit 12 stores the record added with the index update message in the distributed storage system 40 (step A5).

  Thus, the information processing system 1 ends the record processing operation.

  Next, the index data update operation of the information processing system 1 is shown in FIG.

  In FIG. 4, first, the index data update unit 14 acquires an index update message included in a record stored in the distributed storage system 40 (step B1).

  Next, the index data update unit 14 executes an index update message. Then, the index data update unit 14 stores the index data in which the index update message is reflected in the distributed storage system 40 (step B2).

  Thus, the information processing system 1 ends the index data update operation.

  Note that the information processing system 1 may execute the index data update operation illustrated in FIG. 4 at predetermined timings (for example, at predetermined intervals). Alternatively, the information processing system 1 may perform this index data update operation every time the record processing operation shown in FIG. 3 is performed.

  Next, effects of the first exemplary embodiment of the present invention will be described.

  The information processing system as the first exemplary embodiment of the present invention can more fully support eventual consistency between records and indexes in a distributed storage system.

  The reason for this is that when a processing request relating to a record consisting of a key and a value is input to the processing request section, the message generation section uses an index that indicates an index data update instruction that accompanies the generation of the record according to the requested processing. This is because an update message is generated. This is because the record update unit generates a record corresponding to the requested processing, adds an index update message to the generated record, and stores the record to which the index update message is added in the distributed storage system. Further, the index data update unit stores the index data reflecting the index update message in the distributed storage system by executing the index update message included in the record stored in the distributed storage system.

  As described above, according to the present embodiment, the record is stored in the distributed storage system including the index data update instruction accompanying the processing of the record. In this embodiment, index data update processing is executed in accordance with the contents of the update instruction included in the record asynchronously with the record processing. Thus, in the present embodiment, even if a failure occurs after the record processing and before the update of the index data is completed, the information indicating the index data update instruction is not lost, and the record and the index Eventual consistency between data can be satisfied.

(Second Embodiment)
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, an example in which distributed KVS (Key-Value Store) is applied to the distributed storage system of the present invention will be described.

  First, FIG. 5 shows a configuration of an information processing system 2 as a second embodiment of the present invention. In FIG. 5, the information processing system 2 includes an information processing device 20 and a distributed KVS 50. The information processing apparatus 20 is communicably connected to the distributed KVS 50. The information processing apparatus 20 includes a processing request input unit 21, a record update unit 22, a message generation unit 23, an index data update unit 24, and a record reference unit 25. The index data update unit 24 includes a message extraction unit 241. The distributed KVS 50 includes a plurality of servers 500. Each server 500 has a record storage unit 501. The distributed KVS 50 may include, in addition to the illustrated server 500, a transaction processing device (not shown) that processes transactions for records, a configuration management device (not shown) that manages the configuration of the distributed KVS 50, and the like. .

  Here, each device and each functional block of the information processing system 2 can be configured by the same hardware elements as each device and each functional block of the first embodiment of the present invention described with reference to FIG. . Note that the hardware configuration of each device and each functional block of the information processing system 2 is not limited to the above-described configuration.

  Further, the transaction processing device and the configuration management device described above may be configured by a computer device that includes the same hardware elements as the server 500. Alternatively, some or all of these devices may be realized on the same computer device as any server 500. Further, the information processing apparatus 20 may be realized on the same computer apparatus as any one of the apparatuses constituting the distributed KVS 50. In each server 500, the distributed KVS 50 realizes its function when the CPU 4001 reads the computer program stored in the ROM 4003 or the storage device 4004 into the RAM 4002 and executes it.

  The distributed KVS 50 distributes and stores KV (Key-Value) pairs composed of keys and values in the record storage units 501 of the plurality of servers 500. The KV pairs stored in the distributed KVS 50 include a record KV pair and an index KV pair. The record KV pair represents a record to be stored. The index KV pair represents index data for the record KV pair. Hereinafter, the record KV pair is also simply referred to as a record. The index KV pair is also simply referred to as an index or index data. The distributed KVS 50 specifies the storage destination server 500 for the KV pair to be processed in response to a request from the information processing apparatus 20. For example, the distributed KVS 50 may specify the storage destination server 500 according to the hash value of the key of the KV pair to be processed. Then, the distributed KVS 50 executes the processing of the record in the identified server 500.

The distributed KVS 50 has the following functions with respect to the KV pair.
PUT process: A process of storing a KV pair in the distributed KVS 50. Hereinafter, the process of storing the KV pair of the key K and the value V in the distributed KVS 50 is also referred to as “PUT (K, V)”.
GET process: A process of acquiring a KV pair from the distributed KVS 50 by specifying a key. Hereinafter, obtaining a value V by obtaining a KV pair having the key K from the distributed KVS 50 is also referred to as “GET (K) → V”.
DELETE processing: processing for deleting a KV pair from the distributed KVS 50 by specifying a key. Hereinafter, deleting the KV pair having the key K from the distributed KVS 50 is also referred to as “Delete (K)”.
Extraction processing: processing for extracting the KV pairs stored in each server 500 of the distributed KVS 50 by scanning all cases.

  Next, details of each functional block of the information processing apparatus 20 will be described.

  The process request input unit 21 acquires a process request related to the record KV pair as an input. Here, the requested process may be, for example, a new record generation process, an existing record update process, or an existing record deletion process. For example, if it is a request | requirement of a new production | generation process, the process request input part 21 will acquire as input the key and value of the record KV pair which are newly produced | generated. If the request is for an update process, the process request input unit 21 acquires the key of the record KV pair to be updated and the value to be updated as inputs. If the request is for a deletion process, the process request input unit 21 acquires the key of the record KV pair to be deleted as an input.

  The record update unit 22 generates a record KV pair according to the processing request input to the processing request input unit 21. Here, in the distributed KVS 50, there is a case where an associated index KV pair needs to be updated in accordance with the process (new creation / update / deletion) of the record KV pair. Therefore, when the index update unit 22 needs to update the index KV pair, the record update unit 22 uses the message generation unit 23 described later to generate an index update message indicating an index data update instruction in accordance with the record KV pair processing. Then, the record update unit 22 adds an index update message to the generated record KV pair and stores (PUT) it in the distributed KVS 50.

  For example, if the requested process is new creation or update, the record update unit 22 generates a record KV pair including the corresponding key and value. Further, if the requested process is deletion, the record update unit 22 may generate a record indicating that the corresponding record KV pair is deleted. Such a record may be, for example, a record including a key of a record KV pair to be deleted and an empty value.

  For example, when the requested process is newly created, the record update unit 22 passes the corresponding key and the value to be indexed among the corresponding values to the message generation unit 23 described later, Generate index update messages. For example, if the requested process is update or delete, the record update unit 22 acquires (GET) the record KV pair of the corresponding key from the distributed KVS 50. In this case, the record update unit 22 sends the corresponding key, the index creation target value included in the acquired record KV pair, and the updated value in the case of update to the message generation unit 23 described later. By passing, an index update message is generated.

  The message generator 23 generates an index KV pair update instruction corresponding to the processing request as an index update message. Here, the index KV pair is a KV pair having an index key as a key and a key list as a value. Of the values included in the record KV pair, the index creation target value is set in the index key. The key list is a list of keys of a record KV pair that includes an index key as a value. The message generation unit 23 generates an index KV pair update instruction corresponding to the record KV pair processing as an index update message.

  For example, if the requested process is newly created, the message generator 23 creates a newly created record KV in an index KV pair with the index creation target value included in the newly created record KV pair as an index key. Generate an index update message that adds a pair of keys. Also, for example, if the requested process is to be deleted, the message generator 23 deletes from the key list of the index KV pair whose index key is the index creation target value included in the record KV pair to be deleted. Generate an index update message that deletes the key of the record. Further, for example, if the requested process is update, the message generator 23 generates an index update message including two update instructions. The first update instruction is a process for deleting the key of the record to be updated from the key list of the index KV pair using the index creation target value included in the record KV pair to be updated as the index key. The second update instruction is a process of adding the key of the record to be updated to the key list of the index KV pair having the index creation target value included in the updated record KV pair as an index key.

The index update message includes information indicating the type of update processing, information specifying the destination KV pair, and the key to be added or deleted. Here, the type of update processing is information indicating whether a key is added to or deleted from the key list. Here, it is assumed that the addition of a key is represented as “Add” and the deletion of the key is represented as “Remove”. The destination KV pair is information for specifying an index KV pair to be operated. Here, the index key of the index KV pair is applied as information for specifying the destination KV pair. The key to be added or deleted represents the key of the record KV pair to be added to or deleted from the key list of the destination KV pair. A specific example of such an index update message is shown below. Here, it is assumed that there is the following record KV pair.
Order 1 → {item 1,204-14-01-01}
Order 2 → {item2, 2014-01-01}
Order 3 → {item3, 2014-01-01}
Note that “key → value” represents a KV pair. {Value 1, value 2,...} Represents the value of the record KV pair. At this time, it is assumed that an index is set for the second element of the value of the record KV pair with “Order 1” as a key. In this case, the index update message accompanying the new generation of this record KV pair is:
"Add order 1 to 2014-01-01"
It is expressed.

Further, it is assumed that an index is set for the second element of the value of the record KV pair having “Order 2” as a key. In this case, the index update message accompanying deletion of this record KV pair is
"Remove order 2 from 2014-01-01"
It is expressed.

Further, it is assumed that an index is set for the second element of the value of the record KV pair with “Order 3” as a key. Then, it is assumed that the second element of the value of this record KV pair is updated from “2014-01-01” to “2014-01-02”. In this case, the index update message accompanying the update of this record KV pair is:
"Remove order 3 from 2014-01-01"
"Add orders 3 to 2014-01-02"
It is expressed.

  A case will be described in which the requested process is a process (for example, update or deletion) for a record KV pair already stored in the distributed KVS 50. In this case, the record update unit 22 first acquires an index update message that has already been added to the record stored in the distributed KVS 50. Then, the record update unit 22 adds the index update message already added to the record KV pair generated according to the process requested this time and the index update message generated according to the current process.

  Here, FIG. 6 shows an example of a correspondence relationship between the process requested for the record KV pair, the generated index update message, and the process executed by the record update unit 22. In FIG. 6, the “&” symbol represents a delimiter between the value of the record KV pair and the index update message, or a delimiter between a plurality of index update messages. "..." represents omission of an element included in the value of the record KV pair. “K” represents a key of the record KV pair. “IK” represents an index creation target element among the values of the record KV pair. “IK0” represents an index creation target element in the record KV pair before update. “PreviousMessages” represents an index update message that is already included in the record KV pair GET from the distributed KVS 50.

  In FIG. 6, for example, the second line is a processing request “PUT (K, {..., IK,...})” For updating IK 0 included in the value to IK in the record KV pair with the key K. It explains about. In this case, one of the index update messages corresponding to the update processing request is an instruction “Remove K from IK0” for deleting the key K from the index KV pair having the index key IK0 before the update. Another index update message corresponding to this update processing request is an instruction “Add K to IK” for adding a key K to an index KV pair having an updated index key IK. In response to this update process request, the process actually executed by the record update unit 22 first obtains the previous update instruction message “PreviousMessages” included in the record KV pair having the key K (GET). Processing to include. Further, the process actually executed includes a process of adding (PUT) the “PreviousMessages” and the current two update instruction messages to the updated record KV pair.

  The third line describes the processing request “Delete (K)” for deleting the record KV pair with the key K. In this case, the index update message corresponding to the deletion processing request includes an instruction “Remove K from IK” for deleting the key K from the index KV pair having the index key IK. Further, the processing actually executed by the record update unit 22 in response to the deletion processing request is to first acquire the update instruction message “PreviousMessages” included in the record KV pair having the key K. Including. Further, the process actually executed includes a process of adding “PreviousMessages” and the current update instruction message to the record KV pair representing the deletion. Here, the record KV pair representing deletion is represented by a key K and an empty value pair.

  The first line describes the processing request “PUT (K, {..., IK,...})” For creating a new record KV pair with the key K and the value V. In this case, the index update message corresponding to the new creation processing request includes an instruction “Add K to IK” for adding the key K to the index KV pair having the index key IK. In this case, the record KV pair having this key K is not stored in the distributed KVS 50. Therefore, the record update unit 22 does not need to perform processing for acquiring an index update message that has already been added. As a result, the processing actually performed is processing for adding the current index update message to the newly created record KV pair and storing it.

  In FIG. 6, the record update unit 22 needs to atomically execute a GET operation and a PUT operation for the target record KV pair in the update process and the delete process. For this reason, the record update unit 22 may use a CAS function.

  The message extraction unit 241 extracts an index update message from the records stored in the record storage unit 501 of each server 500. For example, the message extraction unit 241 may extract the index update message using the all-case scanning function of the distributed KVS 50.

  The index data update unit 24 stores the index data that reflects the index update message in the distributed KVS 50 by executing the index update message extracted by the message extraction unit 241. The “reflecting” of the index update message means that the index KV pair indicated by the index update message is acquired (GET) from the distributed KVS 50, the value (key list) of the obtained index KV pair is updated, and the updated index KV The pair is stored (PUT) in the distributed KVS 50.

  Further, the index data update unit 24 may delete the reflected index update message from the record KV pair. At this time, the index data update unit 24 desirably performs exclusive control on the update of the record KV pair. Therefore, the index data update unit 24 may use the CAS function. Thereby, for example, the index data update unit 24 atomically executes “reading of record KV pair before reflecting index update message” and “writing of record KV pair from which reflected index update message is deleted”. Is possible.

  The record reference unit 25 is configured to operate as follows when a record KV pair reference request is input to the processing request input unit 21. Specifically, the record reference unit 25 acquires the record KV pair to be requested from the distributed KVS 50. When the acquired record KV pair includes an index update message, the record reference unit 25 outputs a portion that does not include the index update message. That is, the record reference unit 25 filters the index update message and outputs the contents of the record.

  The operation of the information processing system 2 configured as described above will be described with reference to the drawings.

  First, the record reference operation of the information processing system 2 is shown in FIG.

  In FIG. 7, the process request input unit 21 first acquires a record KV pair reference request as an input (step C1).

  Next, the record reference unit 25 acquires the record KV pair to be requested from the distributed KVS 50 (step C2).

  Then, when the index update message is not included in the acquired value of the record KV pair (No in Step C3), the record reference unit 25 outputs a record (Step C5).

  On the other hand, when the index update message is included in the value of the acquired record KV pair (Yes in step C3), the record reference unit 25 filters the index update message included in the value (step C4).

  And the record reference part 25 outputs the record after filtering (step C5).

  Thus, the information processing system 2 ends the record reference operation.

  For example, the user can input a new creation, update, or deletion processing request for a record by referring to the record output by the record reference operation.

  Next, the record processing operation of the information processing system 2 is shown in FIG.

  In FIG. 8, the process request input unit 21 first acquires a process request related to the record KV pair as an input (step A11).

  Next, the information processing system 2 branches the process according to the content of the process requested in step A11.

  Here, the case where the process requested in step A11 is a record KV pair new creation process ("new creation" in step A12) will be described.

  In this case, the record update unit 22 generates a newly created record KV pair using the key and value input to the processing request input unit 21 (step A13).

  Next, the record update unit 22 uses the message generation unit 23 to add the input key to the index data having the index creation target value among the values input to the processing request input unit 21 as an index key. An index update message is generated (step A14).

  Next, the record update unit 22 adds the generated index update message to the generated record KV pair (step A15) and stores it in the distributed KVS 50 (step A16).

  On the other hand, the case where the process requested in step A11 is a record KV pair update process ("update" in step A12) will be described.

  In this case, the record update unit 22 acquires a record KV pair having the update target key input to the processing request input unit 21 from the distributed KVS 50 (step A17).

  Next, the record update unit 22 acquires an index update message that has already been added to the record KV pair acquired in step A17 (step A18).

  Next, the message generation unit 23 generates an index update message for deleting the update target key from the index KV pair having the index creation target value included in the record KV pair acquired in Step A17 as an index key (Step S17). A19).

  Next, the information processing system 2 executes steps A13 to A16 as in the case of the new creation process. However, in the case of update, in step A15, the record update unit 22 adds the index update message so far acquired in step A18 and the index update message generated in step A19 and step A14 to the generated record KV pair. Append.

  Further, a case where the process requested in step A11 is a record KV pair deletion process (“delete” in step A12) will be described.

  In this case, the record update unit 22 acquires a record KV pair having the deletion target key input to the processing request input unit 21 from the distributed KVS 50 (step A20).

  Next, the record update unit 22 acquires an index update message that has already been added to the record KV pair acquired in step A20 (step A21).

  Next, the message generation unit 23 generates an index update message for deleting the deletion target key from the index KV pair having the index creation target value included in the record KV pair acquired in step A20 as an index key (step S20). A22).

  Next, the record update unit 22 generates a record KV pair representing deletion of the corresponding record KV pair using the key to be deleted (step A23). For example, as described above, the record update unit 22 may generate a record KV pair including a corresponding key and an empty value.

  Next, the information processing system 2 executes the processes of steps A15 to A16. However, in the case of deletion, in step A15, the record update unit 22 adds the index update message so far acquired in step A21 and the index update message generated in step A22 to the record KV pair generated in step A23. To do.

  Thus, the information processing system 2 ends the record processing operation.

  A specific example of such a record processing operation will be described. Here, a record KV pair corresponding to the order history table of FIG. 9 is assumed. In this order history table, the “order ID” column is a key, and the “product ID” and “order date” columns are values. Further, it is assumed that the “order date and time” column is an index creation target.

<Specific example of new creation processing>
Assume that a new creation request for a record KV pair representing the order history of the first line in FIG. 9 is input to the processing request input unit 21. This new creation request is expressed as “PUT (order 1, {item 1,014-14-01})” as indicated by the “processing request” in the first line of FIG. 10 (“new” in steps A11 and A12) Create ").

  In this case, the record update unit 22 first generates a newly created record KV pair (order 1, {item 1, 2414-01-01}) (step A13).

  In this case, it is necessary to add the key “order 1” of this record KV pair to the key list of the index KV pair whose index key is 2014-01-01. Therefore, the record update unit 22 uses the message generation unit 23 to generate an index update message representing an instruction to add to the index KV pair. This index update message is represented as “Add order 1 to 2014-01-01” as shown in the “index update message” on the first line in FIG. 10 (step A14).

  Then, the record update unit 22 executes a process of adding the index update message generated in step A14 to the record KV pair generated in step A13 and storing it in the distributed KVS 50. Specifically, the record updater 22 displays “PUT (Order 1, {item 1, 2414-01-01} & Add Order 1 to 2014-01-01)” shown in “Process to be executed” on the first line in FIG. ”Is executed.

<Specific examples of update processing>
Further, it is assumed that an update processing request for a record KV pair representing the order history of the first row in FIG. This update processing request is represented as “PUT (order 1, {item 1,014-14-01})” as shown in the second line “processing request” in FIG. 10 (“update” in steps A11 and A12). ").

  Here, for the record KV pair with “Order 1” as a key, processing for updating “Order Date / Time” from “2014-01-01” to “2014-01-02” has been executed in the past. And Here, it is assumed that a processing request for updating the “order date / time” from “2014-01-02” to “2014-01-01” is input again for the same record KV pair.

  In this case, the record update unit 22 first acquires a record KV pair with “Order 1” as a key from the distributed KVS 50. Here, the process to be executed is “GET (Order 1)” (Step A17). Then, by this GET process, “{item 1, 1414-01-02} & Remove order 1 from 2014-01-01 & Add order 1 to 2014-01-02” is obtained. The two index update messages after “&” added to the obtained record KV pair are those added with the previous update processing of the record KV pair (step A18).

  With this update process, it is necessary to delete the key “order 1” of this record KV pair from the key list of the index KV pair having “2014-01-02” as an index key. Therefore, the record update unit 22 uses the message generation unit 23 to generate the first index update message “Remove order 1 from 2014-01-02” shown in the second row “Index update message” in FIG. (Step A19).

  Next, the record update unit 22 generates an updated record KV pair (order 1, {item 1, 1414-01-01}) (step A13).

  Furthermore, it is necessary to add the key “order 1” of this record KV pair to the key list of the index KV pair whose index key is 2014-01-01. Therefore, the record update unit 22 uses the message generation unit 23 to generate the second index update message “Add order 1 to 2014-01-01” shown in the second line “index update message” in FIG. (Step A14).

  Then, the record update unit 22 adds the previous two index update messages acquired in step A18 and the current two index update messages respectively generated in steps A19 and A14 to the record KV pair generated in step A13. Add (step A15).

  Then, the record update unit 22 stores the record KV pair to which these index update messages are added in the distributed KVS 50 (steps A15 and A16).

  As a result, the process executed by the record update unit 22 in accordance with this update process is as shown in the “executed process” on the second line in FIG.

<Specific example of deletion processing>
Further, it is assumed that a record KV pair deletion processing request shown in the first line of FIG. 9 is input to the processing request input unit 21. This deletion processing request is represented as “DELETE (order 1)” as shown in the “processing request” on the third line in FIG. 10 (“deletion” in steps A11 and A12).

  In this case, the record update unit 22 first acquires a record KV pair with “Order 1” as a key from the distributed KVS 50. Here, the process to be executed is “GET (order 1)” (step A20). It is assumed that {item 1,204-14-01} is obtained by this GET process. Here, it is assumed that the record KV pair with “Order 1” as a key does not include the index update message so far. This is because the index update message previously added to this record KV pair is already reflected.

With this deletion process, it is necessary to delete the key “order 1” of this record KV pair from the key list of the index KV pair having “2014-01-01” as an index key. Therefore, the record update unit 22 uses the message generation unit 23 to generate the index update message “Remove order 1 from 2014-01-01” shown in the third line “Index update message” in FIG. 10 (step A22).
Next, the record updater 22 generates a key “order 1” and an empty value (order 1,) as a record KV pair indicating that this record KV pair is to be deleted (step A23).

  Next, the record update unit 22 adds the index update message generated in step A22 to the record KV pair generated in step A23 (step A15), and executes the process of storing in the distributed KVS 50 (step A16).

  As a result, the processing executed by the record update unit 22 is as shown in the third row “executed processing” in FIG.

  This is the end of the description of the specific example of the record processing operation.

  Next, the index data update operation of the information processing system 2 is shown in FIG.

  In FIG. 11, the message extraction unit 241 first extracts an index update message from the records stored in the distributed KVS 50 (step B11).

  For example, as described above, the message extraction unit 241 extracts the index update message included by confirming all the records stored in the record storage unit 501 of each server 500 included in the distributed KVS 50 (scan all cases). May be.

  Next, the index data update unit 24 executes the following steps B12 to B19 for each index update message extracted in step B11.

  Here, a case where this index update message is addition of a key to the index KV pair (“add” in step B12) will be described.

  In this case, the index data update unit 24 determines whether or not an index KV pair having the corresponding index key is stored in the distributed KVS 50 (step B13). Specifically, the index data update unit 24 may try to acquire such an index KV pair from the distributed KVS 50.

  Here, if there is no KV pair having the corresponding index key (No in Step B13), it is necessary to create a new index KV pair. In this case, the index data update unit 24 generates an index KV pair based on the index update message (step B14) and stores it in the distributed KVS 50 (step B15).

  On the other hand, when there is already a KV pair having the corresponding index key (Yes in Step B13), the index data update unit 24 acquires the index KV pair (Step B16).

  Then, the index data update unit 24 adds the corresponding key to the key list that is the value of the index KV pair (step B17) and stores it in the distributed KVS 50 (step B15).

  A case where the index update message is deletion of a key from the index KV pair (“deletion” in step B12) will be described.

  In this case, the index data update unit 24 acquires an index KV pair having the corresponding index key from the distributed KVS 50 (step B18).

  Next, the index data update unit 24 deletes the corresponding key from the key list that is the value of the index KV pair obtained in step B18 (step B19), and stores it in the distributed KVS 50 (step B15).

  When the above-described processing is completed for each index update message, the information processing system 2 ends the index data update operation.

  Next, a specific example of the index data update operation will be described with reference to FIGS.

  As shown in FIG. 12, in this specific example, the distributed KVS 50 includes four servers 500 (500a, 500b, 500c, and 500d). The servers 500b and 500c store a record KV pair including an index update message before reflection. In the state of FIG. 12, it is assumed that no index KV pair has been stored in the distributed KVS 50 yet.

  At this time, the message extraction unit 241 scans the four servers 500 and extracts index update messages from the servers 500b and 500c (step B11).

  First, the index update message “Add order 1 to 2014-01-01” extracted from the server 500b will be described.

  In this case, the index update message is “add” (“add” in step B12). Therefore, the index data update unit 24 tries to acquire the update target index KV pair based on the key “2014-01-01” of the destination KV pair in the index update message. However, the index KV pair is not yet stored and is not acquired (No in step B13).

  Therefore, the index data update unit 24 newly creates an index KV pair “2014-01-01 → [order 1]” and stores it in the distributed KVS 50 (steps B14 and B15). Thus, the value “2014-01-01” indicated by the destination KV pair of the index update message is applied to the index key of the newly created index pair. In addition, an additional key “order 1” included in the index update message is applied to the key list. Note that [key 1, key 2,...] Represents an index KV pair value (key list).

  At this time, the index data update unit 24 may delete the index update message that has been reflected from the record KV pair of the server 500b that includes the index update message.

  Next, the index update message “Add order 2 to 2014-01-01” extracted from the server 500c will be described.

  Also in this case, this index update message is “add” (“add” in step B12). Therefore, the index data update unit 24 tries to acquire the update target index KV pair based on the key “2014-01-01” of the destination KV pair in the index update message. Here, since the index KV pair “2014-01-01 → [Order 1]” with “2014-01-01” as the index key has already been stored, the acquisition is successful (Yes in Step B13, Step B16). ).

  Next, the index data update unit 24 adds an additional key “order 2” included in the index update message to the key list [order 1] which is the value of the acquired index KV pair [order 1, order 2 ]. Then, the index data update unit 24 stores the index KV pair “2014-01-01 → [order 1, order 2]” in the distributed KVS 50 (steps B17, B15).

  At this time, the index data update unit 24 may delete the index update message that has been reflected from the record KV pair of the server 500c that includes the index update message.

  Further, it is assumed that the index KV pair having “2014-01-01” as a key is stored in the server 500a. Then, the KV pairs stored in the distributed KVS 50 after reflecting these index update messages are as shown in FIG.

  In the present embodiment, the information processing apparatus 20 may include a plurality of index data update units 24. In this case, the plurality of index data update units 24 may operate in parallel. In this case, the plurality of index data update units 24 may perform exclusive control of KV pair update by the CAS function. For example, in the above specific example, it is assumed that the index data update unit 24_1 that processes the index update message of the server 500b and the index data update unit 24_2 that processes the index update message of the server 500c operate in parallel. Then, it is assumed that both index data update units 24 determine that there is no index KV pair having the index key “2014-01-01” in the distributed KVS 50 (No in Step B13). Then, it is assumed that the index data update unit 24_1 has completed the processes of subsequent steps B14 and B15 (a process of newly generating this index KV pair and performing PUT) before the index data update unit 24_2. In this case, the subsequent steps B14 and B15 of the index data update unit 24_2 will fail. This is because the KV pair of the same version already exists by the CAS function. In this way, when the update process fails due to the occurrence of a conflict, the index data update unit 24_2 may execute the operation from step B12 in FIG. 11 again.

  This is the end of the description of the specific example of the index data update operation.

  Next, the effect of the second exemplary embodiment of the present invention will be described.

<Ensuring eventual consistency and data persistence>
The information processing system as the second exemplary embodiment of the present invention can more fully support eventual consistency and data persistence between record KV pairs and index KV pairs in distributed KVS.

  The reason is that, in addition to the configuration similar to that of the first embodiment of the present invention, when the processing required for the record is processing for a record already stored in the distributed KVS, the record update unit This is because it is configured to operate as follows. That is, the record update unit acquires a record from the distributed KVS and adds an index update message corresponding to the current process in addition to another index update message already added to the acquired record.

  Specifically, when the processing requested for a record is a new record creation process, the message generator updates the index to add the key of the record to be newly created to the index data key list including the index key. Generate a message. This is because the record update unit adds the index update message to the newly created record and stores it in the distributed KVS.

  In addition, when the processing required for the record is a deletion processing for a record already stored in the distributed KVS, the message generation unit deletes the key of the record to be deleted from the index data key list including the index key. Generate index update messages. Then, the record update unit adds another index update message that has already been added to the record indicating that the record to be deleted and the index update message generated this time are added, and stores them in the distributed KVS. is there.

  Further, when the process requested for the record is an update process for a record already stored in the distributed KVS, the message generation unit generates the following two index update messages. The first index update message represents an instruction to delete the key of the record to be updated from the key list of the index data that includes the pre-update value of the index creation target as the index key. The second index update message represents an instruction to add the key of the record to be updated to the index data key list including the updated value as the index key. This is because the record update unit adds another index update message that has already been added to the updated record and the two index update messages generated this time, and stores them in the distributed KVS.

  As described above, in this embodiment, for a certain record KV pair, an index data update instruction that is performed asynchronously with processing such as new creation / update / deletion is changed to an update instruction already included in the corresponding record KV pair. Add and include. As a result, even if a failure occurs between the update of the record KV pair and the completion of the update of the index KV pair, the eventual consistency of the record and the index can be maintained. That is, this embodiment improves the fault tolerance of the index data. And this embodiment can also ensure data permanence as a result of ensuring consistency.

  Further, the present embodiment provides fault tolerance for updating an index KV pair without storing which index update message has been reflected among the index update messages added to the record KV pair. And can maintain eventual consistency of records and indexes.

  The reason will be described. Suppose a failure occurs while executing an index update message attached to a record. After the failure recovery, it is difficult for the index data update unit to determine which index update message has already been reflected. However, in such a case, the index data update unit may reflect all the index update messages added to the record. For this reason, there may be an index update message to be re-executed. This is because, in this embodiment, the index update message added to each record KV pair has idempotency.

For example, assume that the following three index update messages are added to a certain record KV pair.
a) Add orders 1 to 2014-01-01
b) Remove order 1 from 2014-01-01
c) Add orders 1 to 2014-01-01
At this time, it is assumed that a failure has occurred when the index data update unit executes the processing from a) to c) in the order (1) to (3) below.
(1) Execute a): Add order 1 to 2014-01-01
(2) Execute b): Remove order 1 from 2014-01-01
(3) Execute c): Add order 1 to 2014-01-01
After the failure recovery, since the index update messages a) to c) are still added to the record KV pair, the index data update unit again starts again from a) as in the following (4) to (6). Suppose that c) is executed.
(4) Execute a): Add order 1 to 2014-01-01
(5) Execute b): Remove order 1 from 2014-01-01
(6) Execute c): Add order received 1 to 2014-01-01
Here, in the index update messages (3) and (4) above, an instruction to add (Add) the same value to the same destination KV pair is described. However, even if (4) is executed after (3), there is no problem because the existing value is only overwritten with the same value. Similarly, even if the same index update message including Remove as shown in b) is executed twice or more in succession, no problem arises because only the already deleted value is deleted. In other words, in the present embodiment, the index update message is executed in the order of “execute index update add after removing added index”. Therefore, since this embodiment re-reflects all remove messages that may not be reflected after a failure occurs, inconsistency occurs from the key list of the index KV pair where inconsistency has occurred. Will delete all existing keys. Then, by executing the Add message after such a Remove message, the present embodiment can execute the latest index update message that is consistent with the record KV pair.

  As described above, according to the present embodiment, even if a failure occurs at an arbitrary timing, the eventual consistency and data persistence of the record KV pair and the index KV pair can be reliably ensured.

<Performance improvement by scale-out>
Furthermore, this embodiment enables scale-out at the time of search processing and update processing for distributed KVS.

  The reason is described. In the present embodiment, it is possible to distribute the index KV pairs in the distributed storage system while satisfying the eventual consistency between the record KV pair and the index KV pair. On the other hand, consider a case where index KV pairs are not distributed and each server holds index data for data stored in its own device. In this case, if there is a search request using an index, all servers need to search for data using the index held by the server. Therefore, when there are many search processes, there is a problem that the process does not scale out even if a server is added. In this embodiment, since index KV pairs are distributedly arranged, a server holding an index KV pair and a server holding a record KV pair may perform processing when executing a search using an index. For this reason, if the load of the search process with respect to distributed KVS becomes high if this Embodiment is used, the performance improvement by a scale-out will be attained by adding a server.

  Similarly, for the update of the record KV pair, the present embodiment enables performance improvement by scale-out. Here, the distributed KVS can scale out the processing by updating and referring to a KV pair having an appropriate data granularity. Also in the method of the present invention, the granularity of data update / reference is KV pairs. Therefore, this embodiment has a scale-out characteristic equivalent to the distributed KVS for updating / referencing the record KV pair.

  Also, suppose that each server holds index data regarding records stored in its own device. In this case, every time a server is added, in addition to the rearrangement of records, an index rebuilding process is required. On the other hand, in this embodiment, since index data is distributedly arranged, it is only necessary to re-arrange the record KV pair and the index KV pair without requiring the reconstruction of the index data when adding a server.

<Efficient processing related to data update control>
Furthermore, the present embodiment increases the efficiency of processing related to data update control.

  This is because the present embodiment does not update a plurality of KV pairs with the same transaction when updating data. When a plurality of KV pairs are executed in the same transaction, processing becomes complicated (TAT: Turn Around Time) due to complicated control. However, in the present embodiment, there is one KV pair to be updated, both when processing a record KV pair and when processing an associated index KV pair. Therefore, this embodiment can reduce the processing delay associated with transaction control.

  In the present embodiment, the index data update unit may delete the index KV pair whose key list is empty from the distributed KVS. Further, a record KV pair whose value is emptied by deleting the index update message may be deleted from the distributed KVS.

  In this embodiment, the index update message is included in the value of the record KV pair. Therefore, in the present embodiment, if a record KV pair including an index update message is regarded as a normal KV pair, it is implemented in combination with a technique for atomically updating a plurality of KV pair groups described in Patent Document 1. It is also possible.

  In the present embodiment, when the index KV pair is enlarged, each index KV pair may be divided into a plurality of KV pairs using the technique described in Japanese Patent Application No. 2013-203790. . In this case, the divided index KV pairs may be distributed on the distributed KVS.

(Third embodiment)
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, an example in which a distributed KVS is applied as a distributed storage system of the present invention will be described as in the second embodiment of the present invention.

  First, FIG. 14 shows the configuration of an information processing system 3 as a third embodiment of the present invention. In FIG. 14, the information processing system 3 includes an information processing device 30 and a distributed KVS 60. The information processing apparatus 30 is communicably connected to the distributed KVS 60. Further, the information processing device 30 includes an index data update unit 34 instead of the index data update unit 24 with respect to the information processing device 20 as the second embodiment of the present invention, and further includes a message storage unit 36 and The message merging unit 37 is different. The message storage unit 36 constitutes one embodiment of a part of the message merge unit of the present invention. The distributed KVS 60 is composed of a plurality of servers 600. Each server 600 includes a message extraction unit 602 in addition to the same record storage unit 501 as the server 500 according to the second embodiment of the present invention.

  Here, each device and each functional block of the information processing system 3 can be configured by the same hardware elements as each device and each functional block of the first embodiment of the present invention described with reference to FIG. . Note that the hardware configuration of each device and each functional block of the information processing system 3 is not limited to the above-described configuration.

  Similar to the distributed KVS 50 in the second embodiment of the present invention, the distributed KVS 60 distributes and stores KV pairs composed of keys and values in the record storage units 501 of the plurality of servers 600.

  The message extraction unit 602 of each server 600 extracts the index update message from the record KV pair stored in its own device. Then, the message extraction unit 602 transmits the extracted index update message to the message storage unit 36 of the information processing device 30.

  Specifically, the message extraction unit 602 extracts an index update message from a record KV pair stored in the own device at a predetermined timing. The predetermined timing may be, for example, every predetermined interval. The predetermined timing may be a timing at which the record KV pair is stored in the own device.

  For example, a case where the message extraction unit 602 operates at predetermined intervals will be described. In this case, the message extraction unit 602 starts from the record KV pair (unextracted record KV pair) from which the index update message has not been extracted yet among the record KV pairs already stored in the record storage unit 501 of the own device. Extract index update messages. For example, the message extraction unit 602 identifies an unextracted record KV pair stored (PUT) in the own device after a certain time (for example, the time of previous extraction) by referring to the transaction log of the own device. Is possible. Alternatively, the message extraction unit 602 has a local index related to the storage (PUT) time of the record KV pair for the own device, thereby identifying the unextracted record KV pair stored (PUT) in the own device after a certain time point. Is possible. The message extraction unit 602 may extract an index update message from the unextracted record KV pair specified in this way. In this case, the message extraction unit 602 may reconstruct the record KV pair obtained by removing the index update message from the record KV pair after extracting the index update message, and store it in the record storage unit 501.

  Or the case where the message extraction part 602 operate | moves at the timing when the record KV pair is stored with respect to an own apparatus is demonstrated. In this case, the message extraction unit 602 may extract the index update message from the record KV pair at the timing when the record KV pair is stored (PUT) in the own device. In this case, the message extraction unit 602 may extract the index update message from the record KV pair to be stored, and then reconstruct the record KV pair excluding the index update message and store it in the record storage unit 501. In this case, the record KV pairs stored in the record storage unit 501 are all after the index update message is extracted and removed. Therefore, in this case, the index update message is not included in the record KV pair that responds from the distributed KVS 60 in response to the reference request from the record reference unit 25. As a result, the network load between the information processing apparatus 30 and the distributed KVS 60 is reduced.

  The message storage unit 36 of the information processing apparatus 30 stores the index update message transmitted from each server 600 of the distributed KVS 60. Note that the message storage unit 36 stores and stores the index update messages added to the same record KV pair in the order of addition. Further, the message storage unit 36 may store the extracted index update message together with information (such as a key) that identifies the record KV pair to which the index update message has been added.

  The message merge unit 37 merges the index update messages stored in the message storage unit 36. Merging refers to simplifying multiple index update messages into a smaller number of index update messages. The message merging unit 37 performs merging so that the index update message after merging yields the same result as the index update message before merging.

  For example, the message merge unit 37 may merge a plurality of index update messages added to one record KV pair. Further, the message merge unit 37 may merge the index update messages added to the plurality of records, respectively. Specifically, the message merge unit 37 may merge index update messages related to the same index KV pair among index update messages added to the plurality of record KV pairs. Here, the message storage unit 36 stores index update messages extracted from a plurality of record KV pairs. Therefore, the message merge unit 37 may extract index update messages related to the same index KV pair from among the index update messages stored in the message storage unit 36 and merge those that can be merged.

  For example, the message merge unit 37 merges the index update messages according to the following merge rule.

<Merge rule of multiple index update messages included in one record KV pair>
FIG. 15 shows a merge rule for a plurality of index update messages included in the same record KV pair. In FIG. 15, each row represents a merge rule. Further, “... 1”, “... 2”, and “... 3” indicate that zero or more index update messages are arranged at the positions.

Here, there are the following types of index update messages.
Registration message: Index update message for adding (Add) a key of a record to a certain index KV pair Deletion message: Index update message for deleting (Delete) a key of a record from a certain index KV pair In FIG. The merge rule in the first line represents a rule for merging them into one registration message when registration messages with the same key for the same index KV pair appear multiple times. This is because no matter how many times the registration message of the same key is reflected for a certain index KV pair, only one key is registered.

  The merge rule in the second row represents a rule for merging them into one deletion message when a deletion message of the same key for the same index KV pair appears multiple times. This is because no matter how many times the deletion message of the same key is reflected for a certain index KV pair, the key is only deleted.

  Further, the merge rule in the third row represents a rule that omits both index update messages when a registration message and a deletion message of the same key for the same index KV pair appear in this order. That is, this merge rule represents a rule that replaces the process of “withdrawing index data once registered” with the process of “no processing at all”.

  The merge rule on the fourth line represents a rule for merging into one registration message when a deletion message and registration message of the same key for the same index KV pair appear in this order.

  As can be seen from these merge rules, in the reflection of the index update message for the same key for the same index KV pair, the key is finally “registered” regardless of the number of appearances and the order of appearance. It can only be in a state or "deleted" state. The state is created by the index update message executed last among such index update messages. Therefore, these merge rules in the first to fourth lines simplify a registration message and a deletion message that appear multiple times for the same key for the same index KV pair into one index update message that appears last. It represents what should be done.

<Merge rule for index update messages included in multiple record KV pairs>
In addition, FIG. 16 shows merge rules for index update messages respectively added to a plurality of record KV pairs. In FIG. 16, each row represents a merge rule. Further, “... 1”, “... 2”, and “... 3” indicate that zero or more index update messages are arranged at the positions. In FIG. 16, each merge rule in the first and second lines indicates that a plurality of registration messages related to different keys for the same index KV pair are merged into one registration message for registering a plurality of keys at a time. Yes. Each merge rule in the third and fourth lines represents that a plurality of deletion messages related to different keys for the same index KV pair are merged into one deletion message for deleting a plurality of keys at a time.

  The index data update unit 34 executes the index update message merged by the message merge unit 37. Thereby, the index data update unit 34 stores the index data in which the merged index update message is reflected in the distributed KVS 60.

  The operation of the information processing system 3 configured as described above will be described with reference to the drawings. Note that the record reference operation and the record processing operation of the information processing system 3 are the same as those in the second embodiment of the present invention described with reference to FIGS. To do.

  First, the message extraction operation of the information processing system 3 is shown in FIG.

  In FIG. 17, first, in the server 600, the message extraction unit 602 extracts an index update message from unrecorded record KV pairs for which message extraction processing has not yet been completed, among record KV pairs stored in its own device. (Step B31).

  Next, the message extraction unit 602 transmits the extracted index update message to the message storage unit 36 of the information processing apparatus 30 (step B32). As a result, the index update message extracted from each server 600 is stored in the message storage unit 36 of the information processing apparatus 30.

  The message extraction unit 602 of each server 600 executes the above operation at a predetermined timing. For example, as described above, each message extraction unit 602 may perform the above-described operation at a predetermined interval or at a timing when the record KV pair is stored in the own device. Further, as described above, each message extraction unit 602 may reconstruct the record KV pair excluding the extracted index update message after extracting the index update message, and store it in the distributed KVS 60.

  Next, the index data update operation of the information processing system 3 is shown in FIG.

  In FIG. 18, first, the message merge unit 37 merges the index update messages stored in the message storage unit 36 (step B41).

  For example, the message merge unit 37 merges a plurality of index update messages extracted from the same record KV pair using the merge rule shown in FIG. In this case, multiple index update messages having idempotency for the same index KV pair are simplified. For example, the message merge unit 37 merges index update messages extracted from a plurality of record KV pairs using the merge rule shown in FIG. In this case, a plurality of index update messages related to different keys for the same index KV pair are merged into one index update message for updating the plurality of keys at a time.

  Next, the index data update unit 34 executes the index update message merged in step B41 (step B42). In this step, the index data update unit 34 may operate for each index update message after merging in the same manner as B12 to B19 in the second embodiment of the present invention described with reference to FIG. .

  Thus, the information processing system 3 ends the index data update operation.

  Note that the information processing apparatus 30 executes such an index data update operation at every predetermined timing. For example, the information processing apparatus 30 may start the operation from step B41 each time an index update message is newly added to the message storage unit 36. Alternatively, the information processing apparatus 30 may start the operation from step B41 at predetermined intervals. Further, the message merge unit 37 may delete the index update message before merging from the message storage unit 36 after executing Step B41. The message merging unit 37 may store the merged index update message in the message storage unit 36. Further, the index data update unit 34 may execute the operation of step B42 on the merged index update message stored in the message storage unit 36 at a timing different from the operation of the message merge unit 37. In addition, the index data update unit 34 may delete the reflected index update message from the message storage unit 36.

  The above operation will be described using a specific example while comparing with the second embodiment of the present invention.

For example, assume that the following two index update messages A and B are stored in the message storage unit 36.
A) & Add orders 1 to 2014-01-01
B) & Add orders 2 to 2014-01-01
The index update message A is added to the record KV pair with “Order 1” as a key. Further, the index update message B is added to the record KV pair with “Order 2” as a key.

At this time, in the second embodiment of the present invention, the index data update unit 24 performs the following four data operations (GET / PUT twice) for one index KV pair.
GET (2014-01-01) → []
PUT (2014-01-01, [Order 1])
GET (2014-01-01)-> [Order 1]
PUT (2014-01-01, [Order 1, Order 2])
Note that [key 1, key 2,...] Represents an index KV pair value (key list).

On the other hand, in the present embodiment, the message merge unit 37 merges the above-described two index update messages A and B into the following one index update message C. That is, a plurality of index update messages for the same destination KV pair are merged into one index update message.
C) & Add Order 1, Order 2 to 2014-01-01
Then, in the present embodiment, the index data update unit 34 may perform the following two data operations (one GET / PUT each) for one index KV pair.
GET (2014-01-01) → []
PUT (2014-01-01, [Order 1, Order 2])
In this manner, by providing the message merge unit 37, the present embodiment reduces the number of data operations for the index KV pair.

  Next, effects of the third exemplary embodiment of the present invention will be described.

  The information processing system according to the third exemplary embodiment of the present invention can further reduce the load of index data update while more fully supporting eventual consistency between record KV pairs and index KV pairs in distributed KVS. it can.

  The reason is that, in addition to the configuration similar to the second embodiment of the present invention, the message merge unit of the information processing apparatus merges the index update messages stored in the message storage unit, and the index data update unit This is because the merged index update message is executed.

  Here, when maintaining eventual consistency for records and indexes, some delay is allowed in updating the index KV pair. Therefore, the index data update unit executes a plurality of index update messages asynchronously (for example, with a certain interval from the record update) without executing the index update message every time the record is updated. At this time, the present embodiment can reduce the number of index KV pair data operations and reduce the load of index update by executing merged ones of a plurality of index update messages. it can.

  Another reason is that the message extraction unit arranged in each server of the distributed KVS extracts the index update message added to the record KV pair and stores it in the message storage unit of the information processing apparatus.

  As described above, by arranging the message extraction unit in each server, in the present embodiment, when the index update message is extracted, the unextracted record KV pair is changed as in the second embodiment of the present invention. There is no need to specify all cases by scanning. In the present embodiment, the message extraction unit arranged in each server may extract the index update message from the record KV pair stored in its own device. For this reason, the present embodiment does not need to scan all cases to extract the index update message, and can improve the efficiency of the index data update process.

  However, if there is a function that realizes an efficient process for specifying an unextracted record KV pair as a message extraction target as a function of the distributed KVS itself, the message extraction unit need not be arranged in each server. Also good. Such a distributed KVS function may be, for example, a function of extracting only record KV pairs updated after a certain point in time. In this case, the present embodiment can be implemented even if a message extraction unit realized by such a function is arranged in the information processing apparatus.

  Each embodiment of the present invention described above may be configured to include a plurality of message extraction units, a plurality of message storage units, a plurality of message merge units, or a plurality of index data update units. In this case, the plurality of functional blocks may be distributed to a plurality of computer devices according to the number of index data handled by the distributed KVS, the number of servers constituting the distributed KVS, and the like.

  In each embodiment of the present invention described above, the message extraction unit may be configured to refer to a record KV pair list (unextracted list) from which no index update message has been extracted. In this case, the record update unit may be configured to update the unextracted list in accordance with the storage process of the record KV pair. Further, the record update unit may create / update the unextracted list from the transaction log used in the distributed KVS, its time stamp, or the version of MVCC (MultiVersion Concurrency Control).

  Further, in each of the above-described embodiments of the present invention, an example in which distributed KVS is applied to the distributed storage system of the present invention has been mainly described. In addition, the distributed storage system in each embodiment is not limited to the distributed KVS as long as it is a system that guarantees atomicity for one data operation on one data, and may be another system.

  Further, in each of the above-described embodiments of the present invention, the description has been focused on an example in which one index is created for one record. In addition, each embodiment can be applied to a case where a plurality of indexes are created for one record. In that case, the record update unit may add an index update message related to each of the plurality of index data to one record.

  Further, in each of the above-described embodiments of the present invention, the index data has been mainly described as an example representing a hash index, but may be information representing another index. For example, the index data may be information representing a B-tree index.

  In each of the above-described embodiments of the present invention, each functional block of each device constituting the information processing system is mainly implemented by a CPU that executes a computer program stored in a storage device or ROM. explained. In addition, a part, all, or a combination of each functional block may be realized by dedicated hardware.

  Moreover, in each embodiment of the present invention described above, the functional blocks of the information processing apparatus may be distributed and realized in a plurality of apparatuses.

  In each embodiment of the present invention described above, the operation of the information processing apparatus described with reference to each flowchart may be stored in a computer storage device (storage medium) as a computer program of the present invention. Good. Then, the computer program may be read and executed by the CPU. In such a case, the present invention is constituted by the code of the computer program or a storage medium.

  Moreover, each embodiment mentioned above can be implemented in combination as appropriate.

  The present invention is not limited to the above-described embodiments, and can be implemented in various modes.

A part or all of each of the above-described embodiments can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A processing request input unit for inputting a processing request related to a record including a key and a value;
An index update message representing an update instruction corresponding to the requested process is generated for index data including an index key to be indexed among values included in the record and a key list including the key of the record. A message generator;
A record that generates a record corresponding to the requested processing, adds the index update message generated by the message generation unit to the generated record, and stores the record with the index update message in the distributed storage system An update section;
An index data update unit for storing the index data in which the index update message is reflected in the distributed storage system by executing an index update message included in the record stored in the distributed storage system;
An information processing apparatus comprising:
(Appendix 2)
When processing required for the record is processing for a record already stored in the distributed storage system, the record update unit acquires the record from the distributed storage system and is already added to the acquired record. The information processing apparatus according to appendix 1, wherein an index update message corresponding to the current process is added in addition to the other index update message.
(Appendix 3)
A message merge unit that merges a plurality of the index update messages;
The information processing apparatus according to appendix 1 or appendix 2, wherein the index data update unit executes the index update message merged by the message merge unit.
(Appendix 4)
The information processing apparatus according to appendix 3, wherein the message merging unit merges a plurality of index update messages added to one record.
(Appendix 5)
The information processing apparatus according to appendix 3 or appendix 4, wherein the message merge unit merges index update messages respectively added to a plurality of records.
(Appendix 6)
When the process requested for the record is a new record creation process,
The message generation unit generates an index update message for adding a key of a record to be newly created to a key list of index data including the index key, according to any one of appendix 1 to appendix 5 The information processing apparatus described.
(Appendix 7)
When the process requested for the record is a process for deleting a record already stored in the distributed storage system,
The message generation unit generates an index update message for deleting a key of a record to be deleted from a key list of index data including the index key, according to any one of appendix 1 to appendix 6, Information processing device.
(Appendix 8)
When the process requested for the record is an update process for a record already stored in the distributed storage system,
The message generation unit deletes the key of the record to be updated from the key list of the index data including the value before update of the index creation target as an index key, and sets the updated value of the index creation target to the index key. The information processing apparatus according to any one of appendix 1 to appendix 7, wherein an index update message for adding a key of a record to be updated is added to a key list of index data included as
(Appendix 9)
When a reference request for the record KV pair is input to the processing request input unit, a record KV pair to be requested is acquired from the distributed storage system, and a portion of the acquired record KV pair that does not include the index update message The information processing apparatus according to any one of appendix 1 to appendix 8, further comprising a record reference unit for outputting.
(Appendix 10)
The information processing apparatus according to any one of appendix 1 to appendix 9,
The distributed storage system;
Information processing system with
(Appendix 11)
Each server constituting the distributed storage system includes a message extraction unit that extracts the index update message from a record stored in the server.
The information processing system according to appendix 10, wherein the index data update unit executes the index update message extracted by the message extraction unit.
(Appendix 12)
When a processing request is entered for a key and value record,
An index update message representing an update instruction corresponding to the requested processing is generated with respect to index data including an index key to be indexed among values included in the record and a key list including the key of the record. ,
Generate a record according to the requested process,
Add the index update message to the generated record,
Storing the record to which the index update message is added in a distributed storage system;
An information processing method for storing index data reflecting the index update message in the distributed storage system by executing an index update message included in a record stored in the distributed storage system.
(Appendix 13)
A process request input step in which a process request for a record consisting of a key and a value is input;
An index update message representing an update instruction corresponding to the requested process is generated for index data including an index key to be indexed among values included in the record and a key list including the key of the record. Message generation step;
A record that generates a record corresponding to the requested processing, adds the index update message generated in the message generation step to the generated record, and stores the record with the index update message in a distributed storage system An update step;
An index data update step of storing index data reflecting the index update message in the distributed storage system by executing an index update message included in a record stored in the distributed storage system;
Is a computer program that causes a computer device to execute.

1, 2, 3 Information processing system 10, 20, 30 Information processing device 11, 21 Processing request input unit 12, 22 Record update unit 13, 23 Message generation unit 14, 24, 34 Index data update unit 25 Record reference unit 36 Message Storage unit 37 Message merge unit 40 Distributed storage system 50, 60 Distributed KVS
241, 602 Message extraction unit 400, 500, 600, 900 Server 401, 501 Record storage unit 1001, 4001 CPU
1002, 4002 RAM
1003, 4003 ROM
1004, 4004 Storage device 1005, 4005 Network interface 1006 Input device 1007 Output device

Claims (10)

A processing request input unit for inputting a processing request related to a record including a key and a value;
An index update message representing an update instruction corresponding to the requested process is generated for index data including an index key to be indexed among values included in the record and a key list including the key of the record. A message generator;
A record that generates a record corresponding to the requested processing, adds the index update message generated by the message generation unit to the generated record, and stores the record with the index update message in the distributed storage system Update section,
An index data update unit for storing the index data in which the index update message is reflected in the distributed storage system by executing an index update message included in the record stored in the distributed storage system;
An information processing apparatus comprising:   When processing required for the record is processing for a record already stored in the distributed storage system, the record update unit acquires the record from the distributed storage system and is already added to the acquired record. The information processing apparatus according to claim 1, wherein an index update message corresponding to the current process is added in addition to the other index update message. A message merge unit that merges a plurality of the index update messages;
The information processing apparatus according to claim 1, wherein the index data update unit executes the index update message merged by the message merge unit. When the process requested for the record is a new record creation process,
The said message generation part produces | generates the index update message which adds the key of the record of newly created object to the key list | wrist of the index data containing the said index key, The any one of Claim 1 to 3 characterized by the above-mentioned. The information processing apparatus according to item. When the process requested for the record is a process for deleting a record already stored in the distributed storage system,
The said message production | generation part produces | generates the index update message which deletes the key of the record of deletion object from the key list | wrist of the index data containing the said index key, The any one of Claims 1-4 characterized by the above-mentioned. The information processing apparatus described in 1. When the process requested for the record is an update process for a record already stored in the distributed storage system,
The message generation unit deletes the key of the record to be updated from the key list of the index data including the value before update of the index creation target as an index key, and sets the updated value of the index creation target to the index key. 6. The information processing apparatus according to claim 1, wherein an index update message for adding a key of a record to be updated is generated to a key list of index data included. The information processing apparatus according to any one of claims 1 to 6,
The distributed storage system;
Information processing system with Each server constituting the distributed storage system includes a message extraction unit that extracts the index update message from a record stored in the server.
The information processing system according to claim 7, wherein the index data update unit executes the index update message extracted by the message extraction unit. When a processing request is entered for a key and value record,
An index update message representing an update instruction corresponding to the requested processing is generated with respect to index data including an index key to be indexed among values included in the record and a key list including the key of the record. ,
Generate a record according to the requested process,
Add the index update message to the generated record,
Storing the record to which the index update message is added in a distributed storage system;
An information processing method for storing index data reflecting the index update message in the distributed storage system by executing an index update message included in a record stored in the distributed storage system. A process request input step in which a process request for a record consisting of a key and a value is input;
An index update message representing an update instruction corresponding to the requested process is generated for index data including an index key to be indexed among values included in the record and a key list including the key of the record. Message generation step;
A record that generates a record corresponding to the requested processing, adds the index update message generated in the message generation step to the generated record, and stores the record with the index update message in a distributed storage system An update step;
An index data update step of storing index data reflecting the index update message in the distributed storage system by executing an index update message included in a record stored in the distributed storage system;
Is a computer program that causes a computer device to execute.

Images