JP2014229036A - Information processing device, information processing method and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce retrieval time while reducing recovery time when a fault occurs in a server used for KeyValue storage.SOLUTION: A server writes a (key, value) group wrote in a buffer area to a file when a data amount of the (key, value) group which is a set of (key, value) wrote in the buffer area exceeds a prescribed threshold. At this time, the server moves the (key, value) group wrote in the buffer area to a cache area secured on a main memory and releases the buffer area. When a retrieval request including a key is received, the server retrieves the (key, value) including the key from the cache area, and retrieves the (key, value) including the key from the file when the (key, value) including the key does not exist in the cache area.

Description

  The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

  In recent years, various KeyValue store technologies such as BigTable (see Non-Patent Document 1) have been proposed. For example, BigTable manages a (key, value) table on a file system (see Non-Patent Document 2) and has the following characteristics.

(1) The (key, value) table is divided into several ranges and managed by a server group on the cluster.
(2) Each server has one log write log file (hereinafter referred to as WAL log). The WAL log ensures persistence for write requests.
(3) When each server successfully writes to the WAL log for operations such as deletion, update, and addition of (key, value), the main memory on the main memory for storing (key, value) secured on the main memory Write to the buffer area. At this point, the written (key, value) can be referred to.
(4) When the amount of data in the buffer area secured on the main memory becomes large, each server writes to the file for each divided range and releases the buffer area. In this way, each server prevents the enlargement of the WAL log by releasing the buffer area when the buffer area on the main memory is written to the file.

  This WAL log exists in order to enable recovery even when a failure occurs in the server while (key, value) exists in the buffer area. For example, the server requested for recovery uses the WAL log to restore the updated (key, value) by reading the updated (key, value) into the buffer data in the main memory on the server. I do.

  In general, the larger the buffer area on the server's main memory, the larger the WAL log. If the WAL log is enlarged, the amount of data read to be expanded in the main memory buffer at the time of recovery after the failure of the server increases, so that the recovery time increases. On the other hand, if the buffer area on the main memory is large, the buffer on the main memory can be searched when the search request is received, and the result can be returned.

  Here, in order to shorten the recovery time, it is conceivable to reduce the buffer area on the main memory of the server in advance. However, if the buffer area is reduced, the server may not be able to return results from the buffer on the main memory when receiving a search request, which may increase the search time.

  In order to solve such a problem, there is a technique called block cache. This block cache prepares a cache area separately from the buffer on the server's main memory, and when the server reads a block (a group of (key, value)) from the hard disk, etc., this block is read. Store in the main memory as a cache. Then, the server uses a block existing on the cache when a read request to the same block is made in the next search. By doing so, the search time can be reduced while the recovery time in the server is reduced to some extent.

F. Chang et al., “Bigtable: A Distributed Storage System for Structured Data”, OSDI, 2006 S. Ghemawat et al., "The Google File System", SOSP, 2003

  However, in the block cache described above, a block existing on the cache is a block in which disk I / O has occurred at least once. For this reason, when the server receives a search request for data that has never been read from the hard disk, it is necessary to read the data from the hard disk or the like, and the search time is not necessarily shortened.

  Therefore, an object of the present invention is to solve the above-described problems and reduce the search time while reducing the recovery time when a failure occurs in the server used for the KeyValue store.

  In order to solve the above-described problem, the information processing apparatus of the present invention includes a WAL (Write Ahead Logging) log on the main memory, and deletes and changes (key, value) used for the KeyValue store in the WAL log. Or an information processing device that writes the (key, value) to the buffer area on the main memory when the operation indicating the addition is successful, the (key, value) written to the buffer area When the amount of data in the (key, value) group that is a set of data exceeds a predetermined threshold, the file creation unit that writes the (key, value) group written in the buffer area to a file, and the data that is written in the buffer area A cache creation unit for moving the (key, value) group to a cache area secured on the main memory, writing to the file of the (key, value) group, and After completion of the movement to the cache area, when a search request including the key and a buffer release unit that releases the buffer area is received, the cache area is searched for (key, value) including the key, and the cache area A search unit for searching for (key, value) including the key from the file when there is no (key, value) including the key.

  According to the present invention, it is possible to reduce the search time while reducing the recovery time when a failure occurs in the server used for the KeyValue store.

FIG. 1 is a diagram illustrating an overview of a server according to the first embodiment. FIG. 2 is a functional configuration diagram of the server. FIG. 3 is a diagram for explaining an example of a file creation procedure when information related to deletion of (key, ts, value) is included in the buffer area. FIG. 4 is a diagram for explaining the processing procedure of the server. FIG. 5 is a diagram for explaining the processing of S16 of FIG. 4 in detail. FIG. 6 is a diagram for explaining S164 of FIG. 5 in detail. FIG. 7 is a diagram illustrating a processing procedure when the server receives a search request for a table of (key, ts, value) from the outside. FIG. 8 is a diagram illustrating a processing procedure of the cache creation unit according to the second embodiment. FIG. 9 is a diagram illustrating an example of a cache in the server according to the second embodiment. FIG. 10 is a diagram illustrating the processing procedure of the search unit. FIG. 11 is a diagram illustrating a server in the distributed KeyValue store. FIG. 12 is a diagram illustrating a computer that executes an information processing program.

[First Embodiment]
[Overview]
Hereinafter, embodiments of the present invention will be described by dividing them into a first embodiment and a second embodiment. First, the outline of the information processing apparatus (server) of the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an overview of a server according to the first embodiment.

  This server 10 is a server used for the KeyValue store, and stores a table of (key, ts, value). In the present embodiment, the contents of the (key, ts, value) table are temporarily stored in the buffer area on the main memory, and then written to a file. In this example, the data constituting the table is described as an example of (key, ts, value), but at least (key, value) may be included, and ts (time stamp value, (key, value) Data that does not include (information indicating the creation time). Further, the file written by the server 10 is described as an example of a sorted KeyValue file, but is not limited thereto.

  The server 10 adds, deletes, and updates (key, ts, value) to the (key, ts, value) table based on an instruction from the external device. In addition, when receiving a search request including a key from the outside, the server 10 searches for a value having the key in the table of (key, ts, value), and returns the search result.

  Further, the server 10 has a log advance writing log file (hereinafter referred to as a WAL log). The WAL log is information indicating changes such as addition, deletion, update, etc. to the (key, ts, value) table, and is written before the actual change of the (key, ts, value) table. When a failure occurs in the server 10, the server 10 reads the updated (key, value) in the buffer area of the main memory and restores the updated data using the WAL log. The server 10 includes a buffer area and a cache area on the main memory.

  When such a server 10 receives (1) an instruction to add (newkey, tsN, newvalue) from a user terminal device or the like, the server 10 (2) writes the contents of the addition instruction to the WAL log. Do. After that, (3) (newkey, tsN, newvalue) is written to the buffer area on the main memory. (4) When the buffer on the main memory becomes large (that is, when the data amount in the buffer area exceeds a predetermined threshold), the server 10 stores the data in the buffer area ((key, ts, value) group). Export to file. For example, the server 10 sorts the (key, ts, value) group in the buffer area by the key value, and writes it to the sorted KeyValue file 1.

  (5) The server 10 moves the data in the buffer area to the cache area secured in the main memory. (6) The server 10 clears the data in the buffer area on the main memory.

  Thereafter, (7) when the server 10 receives a search request including a key, the server 10 first searches the cache area on the main memory, and (8) next searches for a file on the auxiliary storage unit. That is, when the server 10 receives a search request including a key, the server 10 first searches the cache area on the main memory for (key, ts, value) including the key, and includes the key in the cache area (key, ts , value), (key, ts, value) including the key is searched from the sorted KeyValue file on the auxiliary storage unit.

  In this way, when the data in the buffer area on the main memory becomes large, the server 10 writes to the file and clears the data in the buffer area. Thereby, the enlargement of the WAL log can be suppressed, and the recovery time when a failure occurs in the server 10 can be shortened. Further, before the server 10 clears the data in the buffer area, the (key, ts, value) group in the buffer area is moved to the cache area. When the server 10 receives a search request, it first searches the cache area on the main memory. Thereby, the search time in the server 10 can be shortened.

[Constitution]
Next, the configuration of the server 10 will be described with reference to FIG. FIG. 2 is a functional configuration diagram of the server. As shown in FIG. 2, the server 10 includes an auxiliary storage unit 11, a main memory 12, a control unit 13, and an input / output interface 14.

  The auxiliary storage unit 11 stores the WAL log and the sorted KeyValue file. As described above, the WAL log is information indicating changes such as addition, deletion, update, etc. to the (key, ts, value) table of the server 10, and the actual change of the (key, ts, value) table. Written before. The WAL log is referred to when the recovery unit 137 (described later) restores update data when a failure occurs in the server 10.

  The sorted KeyValue file is a file that stores a table of (key, ts, value). This sorted KeyValue file is, for example, a file in which (key, ts, value) groups in the buffer area are sorted by key values.

  The main memory 12 includes a buffer area and a cache area. The buffer area temporarily stores information (additionally, data indicated by (key, ts, value) group) indicating addition, deletion, and update contents to the (key, ts, value) table received by the server 10. This is the area to be stored.

  The cache area is an area for temporarily storing (key, ts, value) groups in the buffer area. The (key, ts, value) group in this cache area is searched when the search unit 134 (described later) receives a search request including Key.

  The control unit 13 controls the entire server 10, and includes a data writing unit 130, a file creation unit 131, a cache creation unit 132, a buffer release unit 133, a search unit 134, a cache deletion unit 135, and a maximum cache. A size registration unit 136 and a recovery unit 137 are provided. The cache creation unit 132a and the cache retention period registration unit 138 may or may not be equipped, and the case where they are equipped will be described in the second embodiment.

  When the data writing unit 130 receives input of information indicating update contents such as deletion, change, or addition to the (key, ts, value) table via the input / output interface 14, the data writing unit 130 stores the information in the WAL log. Write. When the data writing unit 130 succeeds in writing to the WAL log, the data writing unit 130 writes (key, ts, value) indicating the updated contents in the buffer area on the main memory 12. That is, the data writing unit 130 writes an operation indicating deletion, change, or addition of (key, ts, value) to the WAL log, and when this writing is successful, the contents of this operation are stored in the buffer area on the main memory 12. Write to.

  When the data amount of the (key, ts, value) group written in the buffer area on the main memory 12 exceeds a predetermined threshold, the file creation unit 131 writes the (key, ts, value) group to the file. . For example, when the data amount of the (key, ts, value) group written in the buffer area on the main memory 12 exceeds a predetermined threshold, the file creation unit 131 uses the (key, ts, value) group as the Key. The sorted KeyValue file is created and stored in the auxiliary storage unit 11.

  If the (key, ts, value) group written in this buffer area contains information related to the deletion of (key, ts, value), the (key, ts, value) written in this buffer area ) Group and all sorted KeyValue files are read, (key, ts, value) to be deleted are deleted, and a new sorted KeyValue file is created.

  FIG. 3 is a diagram for explaining an example of a file creation procedure when information related to deletion of (key, ts, value) is included in the buffer area. For example, as shown in FIG. 3, when D (key303, ts13, value303), which is information related to deletion of (key, ts, value), is included in the (key, ts, value) group of the buffer area, The file creation unit 131 (1) reads all (key, ts, value) pairs, deletes (key, ts, value) to be deleted, and creates a new file. That is, the file creation unit 131 reads (key, ts, value) in the buffer area of the main memory 12 and the sorted KeyValue files 1, 2,... In the auxiliary storage unit 11, and deletes (key303, ts13, value303). And create a sorted KeyValue file new. Thereafter, the file creation unit 131 (2) deletes the existing file and (key, ts, value) on the main memory 12. That is, the file creation unit 131 deletes the sorted KeyValue files 1, 2,... And the (key, ts, value) group in the buffer area.

  In this way, the file creation unit 131 can reflect the deletion of (key, ts, value) in the table of (key, ts, value).

  The cache creation unit 132 in FIG. 2 has a data amount of the (key, ts, value) group written in the buffer area on the main memory 12 exceeding a predetermined threshold value, and the file creation unit 131 causes the (key, ts, , value) group is written to the sorted KeyValue file, the (key, ts, value) group in the buffer area is moved to the cache area of the main memory 12 as a cache. When the file creation unit 131 deletes (key, ts, value) and writes to a new sorted KeyValue file, the deletion target (key, ts, value) is deleted ( key, ts, value) group is moved to the cache area of the main memory 12 as a cache.

  The buffer release unit 133 releases the buffer area after the writing of the (key, ts, value) group of the buffer area to the file and the movement to the cache area are completed. That is, the data in the buffer area is cleared so that other data can be written.

  When receiving a search request including a key, the search unit 134 first searches the cache area of the main memory 12 for a cache including the key. If there is a cache containing (key, ts, value) having the key, the search result from the (key, ts, value) group of this cache is returned to the transmission source of the search request. On the other hand, if there is no cache containing (key, ts, value) with the key in the cache area, search the sorted KeyValue file for (key, ts, value) with the key, and search the search result Return to the sender. When the search unit 134 searches the cache in the cache area, the search unit 134 assigns information indicating the search time at this time to the cache hit in the search. Although details will be described later, the information indicating the search time given to the cache is referred to when the cache deletion unit 135 selects the cache to be deleted.

  When the amount of data in the cache area exceeds a predetermined amount of data (maximum cache size A), the cache deletion unit 135 preferentially deletes the cache in the cache area from the cache having the oldest search time.

  The maximum cache size registration unit 136 accepts registration of the maximum cache size A used by the cache deletion unit 135 via the input / output interface 14. The accepted maximum cache size A is stored in the main memory 12.

  When a failure or the like occurs in the server 10, the recovery unit 137 reads (key, ts, value) in the buffer area of the main memory 12, and recovers data using the WAL log. For example, when the server 10 accepts input of update data, but the update data in the buffer area is not reflected in the sorted KeyValue file due to a failure or the like, the recovery unit 137 receives the data in the buffer area of the main memory 12 and The update data is restored using the WAL log and reflected in the sorted KeyValue file as necessary.

  The input / output interface 14 is an interface that manages data input and data output to the server 10. For example, the input / output interface 14 sends information indicating addition, deletion, and update contents to the (key, ts, value) table of the server 10 from the external device, a search request for the (key, ts, value) table, When input of information such as the maximum cache size A of the main memory 12 and setting information to the server 10 is received, the information is transferred to the control unit 13. Further, the input / output interface 14 outputs the search result output from the control unit 13 to an external device or the like.

[Processing procedure]
Next, the processing procedure of the server 10 will be described with reference to FIG. It is assumed that the maximum cache size registration unit 136 of the server 10 accepts registration of the maximum cache size A and stores it in the main memory 12 in advance. FIG. 4 is a diagram for explaining the processing procedure of the server.

  First, when the data writing unit 130 of the server 10 receives input of update information such as deletion, change, or addition to the (key, ts, value) table via the input / output interface 14 (S11), Information is written to the WAL log (S12). The data writing unit 130 also writes the information in the buffer area on the main memory 12 (S13).

  After S13, when the amount of data in the buffer area on the main memory 12 exceeds a predetermined threshold (Yes in S14), the file creation unit 131 writes the (key, ts, value) group in the buffer area to the file ( S15). For example, the file creation unit 131 creates a sorted KeyValue file by sorting the (key, ts, value) group written in the buffer area on the main memory 12 by the Key value, and stores it in the auxiliary storage unit 11. .

  On the other hand, when the amount of data in the buffer area on the main memory 12 is equal to or smaller than the predetermined threshold (No in S14), the process returns to S11.

  Further, after S15, the cache creation unit 132 of the server 10 moves the (key, ts, value) group of the buffer area to the cache area of the main memory 12 as a cache (S16). Details of the processing of S16 will be described later.

  After S16, the buffer release unit 133 releases the buffer (S17). That is, the buffer release unit 133 clears the (key, ts, value) group in the buffer area, makes the data writable, and ends the process.

  Next, the process of S16 of FIG. 4 will be described in detail with reference to FIG. FIG. 5 is a diagram for explaining the processing of S16 of FIG. 4 in detail.

  After S15 in FIG. 4, the cache creation unit 132 of the server 10 moves the buffer on the main memory 12 to the cache area (S161). That is, the cache creation unit 132 moves the (key, ts, value) group in the buffer area to the cache area in the main memory 12 as a cache.

  After S161, the cache creation unit 132 adds the moved buffer size to the cache size B currently used (S162). That is, the cache creation unit 132 adds the data amount of the (key, ts, value) group moved from the buffer area to the cache area to the cache size B.

  When the cache creation unit 132 determines whether the cache size B added in S162 exceeds the maximum cache size A, and determines that the cache size B added in S162 exceeds the maximum cache size A, (Yes in S163), the cache deletion unit 135 performs cache deletion (S164). And the process of S16 is complete | finished. Details of the process of S164 will be described later.

  On the other hand, when the cache creation unit 132 determines that the cache size B added in S162 does not exceed the maximum cache size A (No in S163), the process of S16 ends.

  Next, S164 in FIG. 5 will be described in detail with reference to FIG. FIG. 6 is a diagram for explaining S164 of FIG. 5 in detail.

  When the result in S163 in FIG. 5 is Yes, the cache deletion unit 135 selects the cache with the oldest search time from all the caches in the cache area and releases it from the main memory 12 (S1641). After S1641, the cache deletion unit 135 reduces the size of the released cache data from the cache size B (S1642), and determines whether the cache size B exceeds the maximum cache size A (S1643). If the cache size B does not exceed the maximum cache size A (No in S1643), the process of S164 is terminated. On the other hand, even after the cache is deleted, if the cache size B exceeds the maximum cache size A (Yes in S1643), the process returns to S1641, and the cache deletion unit 135 further deletes the cache in the cache area.

  As described above, the cache deletion unit 135 deletes the caches in order from the oldest search time so that the amount of data occupied by the cache in the cache area is equal to or smaller than the maximum cache size A. As a result, (key, ts, value) retrieved relatively recently is placed on a limited cache area on the main memory 12. Note that the search request accepted by the server 10 has a relatively recent search (key, ts, value) in the cache area when there are many search requests related to (key, ts, value) created relatively recently. The possibility of shortening the search time can be increased by placing.

  Next, a processing procedure when the server 10 receives a search request for a (key, ts, value) table from the outside will be described with reference to FIG. FIG. 7 is a diagram illustrating a processing procedure when the server receives a search request for a table of (key, ts, value) from the outside.

  First, upon receiving a search request including a key (S21), the search unit 134 of the server 10 first determines whether or not a cache including the key exists in the cache area (S22). Here, when there is a cache including the key in the cache area (Yes in S22), the search unit 134 searches the cache instead of the sorted KeyValue file (S23). Then, the search unit 134 returns (key, ts, value) including the key obtained from the cache as a search result. On the other hand, when there is no cache including the key in the cache area (No in S22), the search unit 134 searches for the sorted KeyValue file (S24). Then, the search unit 134 returns (key, ts, value) including the key obtained from the sorted KeyValue file as a search result.

  In other words, if (key, ts, value) to be searched exists in the cache area, the server 10 returns (key, ts, value) in this cache area. The possibility of shortening is increased. In particular, when the server 10 often receives a search request regarding (key, ts, value) that has been added or searched to the (key, ts, value) table relatively recently (key, ts, value). , value) can be further shortened.

  In the first embodiment, when the cache deleting unit 135 deletes the cache, the cache is deleted preferentially from the oldest search time, but the present invention is not limited to this. For example, the cache deleting unit 135 may delete preferentially the oldest (key, ts, value) value of ts from the (key, ts, value) group included in the cache.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. The description of the same configuration and the same processing as in the first embodiment is omitted. When the server 10a of the second embodiment moves the (key, ts, value) group of the buffer area to the cache area, the server 10a in the buffer area (key, ts, value) group of the buffer area (key, ts, value) group is past a predetermined time from the current time. The (key, ts, value) group of minutes is moved to the cache area.

[Constitution]
The server 10a according to the second embodiment further includes a cache retention period registration unit 138 indicated by a broken line in FIG. The server 10a includes a cache creation unit 132a instead of the cache creation unit 132.

  The cache retention period registration unit 138 receives an input of the cache retention period via the input / output interface 14. This cache retention period T is stored on the main memory 12.

  When the cache creation unit 132a moves the (key, ts, value) group in the buffer area on the main memory 12 to the cache area, the value of ts in the (key, ts, value) group in the buffer area is (currently A (key, ts, value) group of time-cache retention period = time T) is selected and moved.

[Processing procedure]
The processing procedure of the cache creation unit 132a will be described with reference to FIG. FIG. 8 is a diagram illustrating a processing procedure of the cache creation unit according to the second embodiment. It is assumed that the server 10a accepts an input of the cache holding period via the input / output interface 14 by the cache holding period registration unit 138 in advance.

  First, the cache creation unit 132a causes the data amount of the (key, ts, value) group written in the buffer area on the main memory 12 to exceed a predetermined threshold, and the file creation unit 131 causes the (key, ts, When the (value) group is written to the sorted KeyValue file, the ts of the (key, ts, value) group in the buffer area are acquired. Further, the cache creation unit 132a sets (current time−cache retention period) as T (S31).

  After S31, the cache creation unit 132a determines whether or not T <ts with respect to ts acquired in S31 (S32), and when T <ts (Yes in S32), that is, the time indicated by ts Is after the time T, the group (key, ts, value) having the ts is moved as a cache of the cache area (S33), and the process proceeds to S34. On the other hand, when T <ts is not satisfied (No in S32), the process proceeds to S34.

  In S34, if there is an unprocessed (key, ts, value) in the (key, ts, value) group in the buffer area (No in S34), the cache creation unit 132a returns to S31 and returns (key in the buffer area , ts, value) group has been processed (Yes in S34), the process proceeds to S35. Since the processing after S35 is the same as the processing after S163 in FIG. 5 described above, a detailed description thereof will be omitted. However, the cache deletion unit 135 determines that the cache size B currently used exceeds the maximum cache size A. When (Yes in S36), in S37, the cache including the oldest (key, ts, value) of ts is deleted from the cache in the cache area.

  By doing so, a group (key, ts, value) having relatively new values of ts is placed in the cache area. Therefore, when there are many search requests regarding (key, ts, value) created relatively recently, the cache area is loaded with a group (key, ts, value) having a relatively new value of ts. This increases the possibility that the search time in the server 10a can be shortened.

  The effect of the server 10a of the second embodiment will be described in detail with reference to FIG. FIG. 9 is a diagram illustrating an example of a cache in the server according to the second embodiment. Here, a case where four caches of cache 1 to cache 4 are on the cache area will be described as an example.

  The horizontal axis in FIG. 9 indicates the value of ts. A solid line box indicates a group (key, ts, value) constituting each cache in the server 10a of the second embodiment, and a broken line box indicates the (key) of the buffer area that is the origin of the caches 1 to 4 , ts, value) group. That is, the rightmost value in each box in FIG. 9 indicates the newest ts value in the (key, ts, value) group, and the leftmost value in each box is the (key, ts, value) group. The oldest ts value is shown. For example, after all the (key, ts, value) groups indicated by reference numerals 91 to 94 are loaded on the cache area as a cache, the cache deletion unit 135 indicates the oldest ts value indicated by reference numeral 93 (key, ts, value). If the (value) group is deleted, a relatively new (key, ts, value) value of ts is also deleted, and there is a possibility that no hit will occur in the cache area during the search.

  However, the cache creation unit 132a of the server 10a selects the (key, ts, value) group in which (current time−cache retention period) <ts from the (key, ts, value) group in the buffer area, Put it in the cache area. As a result, the memory usage of the cache in the cache area can be shortened, and the proportion of the (key, ts, value) group with relatively new values of ts can be increased. That is, data with a high possibility of hitting can be placed on a limited cache area.

[Other Embodiments]
In addition, the search request received by the search unit 134 of the servers 10 and 10a may include not only the key but also the range of ts (time stamp interval) that is the search target. In this case, the search unit 134 searches the cache area and the file for (key, ts, value) having a value of ts in the range indicated in the received search request.

  The processing procedure of the search unit 134 will be described with reference to FIG. FIG. 10 is a diagram illustrating the processing procedure of the search unit. When the search unit 134 receives a search request including a key and a time stamp interval (S41), a cache ((key, ts, value) group including the key exists in the cache area (Yes in S42), and When the ts of the cache is included in the time stamp section (Yes in S43), the process proceeds to S44. Since S44 is the same as S23 of FIG. 7, description thereof is omitted.

  On the other hand, if there is no cache including the key in the cache area (No in S42) or ts of the cache is not included in the time stamp interval (No in S43), the process proceeds to S45. Since S45 is the same as S24 of FIG. 7, description thereof is omitted.

  In this way, the servers 10 and 10a can return a search result for a search request in which a time stamp interval is specified.

  The servers 10 and 10a described above may be used in a distributed KeyValue store system. In this case, the (key, ts, value) table held by the servers 10 and 10a is a table obtained by dividing the original table in the distributed KeyValue store. FIG. 11 is a diagram illustrating a server in the distributed KeyValue store.

  For example, as shown in FIG. 11, a case is considered in which the system includes a master server (server 10D) and a plurality of slave servers (servers 10A, 10B, 10C). In such a case, the (key, ts, value) table held by each of the servers 10A, 10B, and 10C is a table obtained by dividing the original table A into a plurality of areas. For example, the server 10A holds a table of (key, ts, value) in areas 0 and 3, and the server 10B holds a table of (key, ts, value) in areas 1 and 4. When the server 10D receives a request for deletion, change, addition, search, etc. of (key, ts, value), the server 10D creates a table of (key, ts, value) for the area including the (key, ts, value). The request is transferred to the slave servers (servers 10A, 10B, 10C) to be held. And server 10A, 10B, 10C performs the process similar to the process described in each above-mentioned embodiment, and returns the execution result to server 10D.

[program]
The processing executed by the servers 10 and 10a can be realized by a program (information processing program) described in a language that can be executed by a computer. In this case, the same effect as the above-described embodiment can be obtained by the computer executing the program. Further, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer and executed to execute the same processing as in the above-described embodiment. Below, an example of the computer which performs the information processing program which implement | achieves the function similar to the servers 10 and 10a is demonstrated.

  FIG. 12 is a diagram illustrating a computer that executes an information processing program. As illustrated in FIG. 12, the computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012 as illustrated in FIG. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031 as illustrated in FIG. The disk drive interface 1040 is connected to the disk drive 1041 as illustrated in FIG. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052 as illustrated in FIG. The video adapter 1060 is connected to a display 1061, for example, as illustrated in FIG.

  Here, as illustrated in FIG. 12, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the above program is stored in, for example, the hard disk drive 1031 as a program module in which a command to be executed by the computer 1000 is described.

  The various data described in the above embodiment is stored as program data, for example, in the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1031 to the RAM 1012 as necessary, and realizes the functions of the respective units of the control unit 13.

  Note that the program module 1093 and program data 1094 related to the program are not limited to being stored in the hard disk drive 1031, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1041 or the like. Good. Alternatively, the program module 1093 and the program data 1094 related to the program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and via the network interface 1070. It may be read by the CPU 1020.

10, 10a server (information processing device)
DESCRIPTION OF SYMBOLS 11 Auxiliary memory part 12 Main memory 13 Control part 14 Input / output interface 130 Data writing part 131 File creation part 132,132a Cache creation part 133 Buffer release part 134 Search part 135 Cache deletion part 136 Maximum cache size registration part 137 Recovery part 138 Cache Retention period registration section

Claims (7)

A WAL (Write Ahead Logging) log is provided on the main memory, and if the operation indicating deletion, change, or addition of (key, value) used in the KeyValue store is successfully written to the WAL log, the (key , value) to the buffer area on the main memory,
When the amount of data in the (key, value) group that is a set of (key, value) written in the buffer area exceeds a predetermined threshold,
A file creation unit for writing (key, value) group written in the buffer area to a file;
A cache creation unit that moves (key, value) group written in the buffer area to a cache area secured on the main memory;
A buffer release unit for releasing the buffer area after completion of writing to the file of the (key, value) group and moving to the cache area;
When a search request including the key is accepted,
Search for (key, value) including the key from the cache area, and search for (key, value) including the key from the file when the key does not include (key, value) in the cache area An information processing apparatus comprising a unit.   When the data amount of the (key, value) group in the cache area exceeds a predetermined data amount, the search time or creation time (key) of the (key, value) group in the cache area is the oldest (key , value) group, further comprising a cache deleting unit that deletes preferentially from the group. When the amount of data of the (key, value) group written in the buffer area exceeds a predetermined threshold, the (key, value) group of (key, value) written in the buffer area When information indicating deletion was included,
The file creation unit
When writing the (key, value) group written in the buffer area to a file, create a new file with the (key, value) to be deleted deleted from the file,
The cache creation unit
2. The (key, value) group in which (key, value) to be deleted is deleted from the (key, value) group written in the buffer area to the cache area. Alternatively, the information processing apparatus according to claim 2. The cache creation unit
When (key, value) group written in the buffer area is moved to the cache area,
2. The (key, value) group that is updated or added is moved from a current time to a past predetermined time in the (key, value) group. Item 4. The information processing device according to any one of items 3 to 4. The search unit
When the search request further includes a creation time range of (key, value) to be searched, the search request includes the key from the cache area and the file, and is created in the creation time range. The information processing apparatus according to claim 1, wherein (key, value) is searched. A WAL (Write Ahead Logging) log is provided on the main memory, and if the operation indicating deletion, change, or addition of (key, value) used in the KeyValue store is successfully written to the WAL log, the (key , value) to the buffer area on the main memory, an information processing method using an information processing apparatus,
The information processing apparatus is
When the amount of data in the (key, value) group that is a set of (key, value) written in the buffer area exceeds a predetermined threshold,
A file creation step of writing a (key, value) group written in the buffer area to a file;
A cache creation step of moving the (key, value) group written in the buffer area to a cache area secured on the main memory;
A buffer release step of releasing the buffer area after completion of writing to the file of the (key, value) group and moving to the cache area;
When a search request including the key is accepted,
A search step of searching for (key, value) including a key from the cache area, and searching for (key, value) including the key from the file when there is no (key, value) including the key in the cache area And an information processing method. A WAL (Write Ahead Logging) log is provided on the main memory, and if the operation indicating deletion, change, or addition of (key, value) used in the KeyValue store is successfully written to the WAL log, the (key , value) is an information processing program for writing to the buffer area on the main memory,
When the amount of data in the (key, ts, value) group that is a set of (key, value) written in the buffer area exceeds a predetermined threshold,
A file creation step of writing a (key, value) group written in the buffer area to a file;
A cache creation step of moving the (key, value) group written in the buffer area to a cache area secured on the main memory;
A buffer release step of releasing the buffer area after completion of writing to the file of the (key, value) group and moving to the cache area;
When a search request including the key is accepted,
A search step of searching for (key, value) including a key from the cache area, and searching for (key, value) including the key from the file when there is no (key, value) including the key in the cache area An information processing program for causing a computer to execute.

Images