JP2012168781A - Distributed data-store system, and record management method in distributed data-store system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid wasting computing resources and improve response performance in a distributed data-store system which performs distributed management of information using a plurality of computers connected by a network.SOLUTION: A distributed data-store system performs distribute management of records identified by Keys in a plurality of nodes 2-1 to 2-3 connected by a network. Each of the plurality of nodes 2-1 to 2-3 has: a storage unit 50 for storing a plurality of records managed by the own node as aggregations grouped by optional key ranges; an index assignment unit 10 for assigning an index which uses keys included in a range of an aggregation to the aggregation; and a record acquisition unit 40 for, in response to a record acquisition request, acquiring a record requested in the record acquisition request from the storage unit 50 by referring to indexes of aggregations.

Description

  The present invention relates to a distributed data store system that realizes scale-out by distributing and managing information using a plurality of computers connected via a network, and in particular, avoids waste of computing resources and improves response performance. Regarding technology.

  Data storage in a general database secures an area according to a certain size defined by the table schema, so the offset position of the stored data can be easily identified, and any record or column can be accessed efficiently be able to.

  On the other hand, in a record-added distributed data store system, multiple variable-length records are written to a file at the same time, and the existing file is not overwritten at the time of update, but is appended to the end of the file or another file. Therefore, in order to obtain a desired record, it is necessary to read these files into the memory area one by one and check the existence of the record, thereby obtaining one record and extracting only a specific column. Even if it is desired to do so, an access to a file that does not need to be read occurs, and inefficient processing such as useless computing resource consumption and a decrease in response is performed.

  In such a distributed data store system, in particular, there is a record in which a combination of Key and Value is used as a record. In that case, an arbitrary Key is assigned to Value that is data to be stored, and a continuous Key and corresponding to it. By dividing a pair with Value to be an arbitrary size as a minimum unit and saving it in a disk area that becomes a file system, the Key space is automatically divided into an arbitrary range, and the divided Key range is assigned to different nodes. To achieve scale-out. When a record corresponding to a certain key is read, the index node is inquired about the location of the node that is in charge of the key range including the key, and the location information such as the IP address of the node that is in charge of the key range is acquired. Then, based on this, the record corresponding to the key is read by making a request for reading the record corresponding to the key to the responsible node.

  In the node, the recently accessed key and value pair is placed on the memory area, and other data is read from the disk area. In addition, processing such as update and insertion is performed only on the memory area. When the size of the key and value on the memory area or the size of the commit log exceeds the threshold, a process called minor compaction occurs. A record group including a plurality of pairs of Key and Value above is written as a file on the disk area and stored permanently (for example, see Non-Patent Document 1).

OSDI'06: Seventh Symposium on Operating System Design and Implementation, Seattle, WA, November, 2006.

  As described above, in the distributed data store system, unless major compaction is executed, it is necessary to read all the files on the disk area corresponding to the specific key range and check which file contains the target data. In other words, the distributed file system has a problem that the disk I / O load on the disk area and the network I / O load between nodes constituting the distributed file system are high.

  The present invention has been made in view of the problems of the prior art as described above, and is applied to a distributed data store system in which information is distributed and managed using a plurality of computers connected by a network. It is an object of the present invention to provide a distributed data store system that can avoid waste of resources and improve response performance, and a record management method in the distributed data store system.

In order to achieve the above object, the present invention provides:
In a distributed data store system that distributes and manages records specified by identifiers at multiple nodes connected by a network,
The plurality of nodes are:
Record storage means for storing a plurality of records managed by the node as an aggregate for each arbitrary range of the identifier;
Index giving means for giving an index using an identifier included in the range of the aggregate to the aggregate;
Record acquisition means for acquiring the record requested by the record acquisition request from the record storage means by referring to the index in response to the record acquisition request.

Also, a record management method in a distributed data store system that distributes and manages records specified by identifiers at a plurality of nodes connected by a network,
When the plurality of nodes store a plurality of records managed by the node as an aggregate for each arbitrary range of the identifier, an index using an identifier included in the aggregate range for the aggregate The record requested in the record acquisition request is acquired by referring to the index in response to the record acquisition request.

  As described above, in the present invention, when a plurality of nodes connected by a network perform distributed management of a record specified by an identifier, in each of the plurality of nodes, a plurality of records managed by the node are stored. When storing as an aggregate for each arbitrary range of identifiers, an index using an identifier included in the aggregate range is assigned to the aggregate, and the index is referred to the record acquisition request. Thus, since the record requested by the record acquisition request is acquired, even in the situation where there are a plurality of files corresponding to the predetermined range of the identifier, the file including the target identifier can be accessed. This eliminates the need to access multiple files, shortens the response time when reading, and It is possible to reduce the load on the system or network.

It is a figure which shows one Embodiment of the distributed data store system of this invention. It is a figure for demonstrating the process at the time of writing a record in a memory | storage part in the distributed data store system shown in FIG. It is a figure which shows the structure of the index produced in the index provision part shown in FIG. It is a figure for demonstrating the process at the time of reading the record memorize | stored in the memory | storage part in the distributed data store system shown in FIG. It is a figure for demonstrating the method to manage a file per generation in the distributed data store system shown in FIG. It is a figure which shows the structure of the file produced by a generation unit, (a) is a figure which shows the whole structure, (b) is a figure which shows the structure of Key, (c) is a figure which shows a specific structural example. is there. 3 is a flowchart for explaining record reading processing in the distributed data store system shown in FIG. 1.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing an embodiment of a distributed data store system of the present invention.

  As shown in FIG. 1, the present embodiment is configured by connecting an index node 1 and three nodes 2-1 to 2-3 via a network. The records distributedly managed by the nodes 2-1 to 2-3 are data in the Key-Value Store format (KVS format) composed of a set of Key and Value that is an identifier for uniquely identifying each record. It is a structure. The key components are RowKey, column name, and time stamp, which is a model that can represent a table structure having the concept of time stamp. In addition, a node that handles a range of an arbitrary key is determined by one index node 1 having a hierarchical structure, and all processing for a read / write request to a record corresponding to a certain key is in charge of that key. Node does. In this embodiment, as shown in FIG. 1, the node 2-1 is responsible for two key ranges Range1, Range2, the node 2-2 is responsible for one key range Range3, and the node 2-3 is one key. I am in charge of the range Range4.

  Each of the nodes 2-1 to 2-3 has a storage unit 50 that is a record storage unit that stores a record of the key range in charge, an index assigning unit 10, a storage control unit 20, and a file division Section 30 and record acquisition section 40.

  The storage unit 50 includes a memory area 51 having a cache function, and a disk area 52 in which files created by performing minor compaction on records accumulated in the memory area 51 are permanently stored. A plurality of records managed by the nodes 2-1 to 2-3 are stored as an aggregate for each arbitrary key range.

  The index assigning unit 10 creates and assigns an index for accessing a record corresponding to the key range for each arbitrary key range managed by the nodes 2-1 to 2-3.

  The storage control unit 20 accumulates records in an arbitrary key range managed by the nodes 2-1 to 2-3 in the memory area 51 in ascending or descending order of the keys, and the size of the record accumulated in the memory area 51 is a predetermined size. When the threshold value is exceeded, minor compaction is executed to store the record in the disk area 52 as one file unit.

  The file dividing unit 30 extracts a plurality of records according to a predetermined condition from a plurality of records stored in the disk area 52, and rearranges the plurality of records in the ascending order or descending order of the key to generate a new file. .

  In response to the record acquisition request, the record acquisition unit 40 acquires the record requested by the record acquisition request from the storage unit 50 and returns it to the record acquisition request source.

  Below, the record management method in the distributed data store system comprised as mentioned above is demonstrated.

  First, a process for writing a record in the storage unit 50 will be described. In the following, the process for the node 2-1 will be described as an example, but the same process is performed for the nodes 2-2 and 2-3.

  FIG. 2 is a diagram for explaining processing when a record is written in the storage unit 50 in the distributed data store system shown in FIG.

  A record write request for an arbitrary key from a client is transmitted to a node 2-1 that manages records in the key range, as shown in FIG.

  Then, in the node 2-1 that has received the record write request, first, the record is accumulated in the memory area 51 under the control of the storage control unit 20 (a in FIG. 2). At this time, records are stored in the memory area 51 in ascending or descending order of the keys.

  When the total record size stored in the memory area 51 exceeds a predetermined threshold, minor compaction is executed (a in FIG. 2), and a record is created as one file unit and is permanently stored in the disk area 52. It is memorized (c in FIG. 2). Note that the threshold value is generally specified based on the mounted physical memory size.

  At the same time, the index assigning unit 10 creates indexes corresponding to all records and assigns them to the records stored in the disk area 52. Thereby, the index information (index 1 in FIG. 2) corresponding to the Key range is updated.

  FIG. 3 is a diagram showing the configuration of the index created by the index assigning unit 10 shown in FIG.

  As shown in FIG. 3, the index created by the index assigning unit 10 stores the file name including the key, the offset in the file, and the record size, and the key in charge at the node 2-1 is in any range. Is given to each of the aggregates.

  Thereby, in each of the nodes 2-1 to 2-3, the records belonging to the range are managed as a logical aggregate for each key range (Range) in charge, and at that time, the key is assigned to the aggregate. An index for accessing the value corresponding to is assigned.

  Next, a process when reading a record stored in the storage unit 50 will be described.

  FIG. 4 is a diagram for explaining processing when reading a record stored in the storage unit 50 in the distributed data store system shown in FIG.

  As shown in FIG. 4, files are stored in the disk area 52 as described above (a in FIG. 4), and these files can be accessed by the indexes created as described above. (A in FIG. 4).

  When a record acquisition request for reading a record corresponding to an arbitrary key is transmitted from the client, the node 2-1 first refers to the index corresponding to the key range including the target key, and to which file It is checked whether or not the key is included (C in FIG. 4). In the index, as described above, since the file name including the key, the offset in the file, and the record size are stored, the file including the key can be checked by specifying the key.

  Next, the record acquisition unit 40 accesses a file including the target key, reads the record (D in FIG. 4), and returns the acquired record to the client.

  Here, when executing a query that acquires only the latest generation with the latest time stamp, the key of the distributed data store system is sorted by priority such as RowKey, column name, and time stamp. The records you have are sparsely present on the file. For this reason, even with sequential access, it is necessary to read the entire file contents from the file system, including records with old time stamps that do not need to be read, and the number of records to be read from the file system increases.

  Therefore, it is conceivable to recognize records having different time stamps as the same record having different generations, and generate records having the same generation as one file.

  FIG. 5 is a diagram for explaining a method of managing files in units of generations in the distributed data store system shown in FIG.

  When managing files in generation units in the distributed data store system shown in FIG. 1, first, major compaction is executed in the file dividing unit 30, and generations (G3: third generation, G2: second generation, G1). : First generation) file is created (in FIG. 5).

  6A and 6B are diagrams showing the configuration of a file created for each generation. FIG. 6A is a diagram showing the overall configuration, FIG. 6B is a diagram showing the configuration of a key, and FIG. 6C is a specific configuration example. FIG.

  As shown in FIG. 6, a file created in units of generations is composed of a key and a value. In the key, a row key, a column name, and a time stamp are connected by a colon (:).

  In the file dividing unit 30, records having only different time stamps among the key components are recognized as the same record having different generations, and records having the same generation are generated as one file. Specifically, when executing major compaction, the Key element dependent file division function is used, and by specifying “generation” as a predetermined condition (extraction condition) in the function, both the RowKey and the column name are the same. All records are collected and the time stamps are examined to identify the generation of each record and create a file for each generation. At this time, the file of the generation unit is sorted by the key as in the case of the file created by general minor compaction or major compaction.

  Thereby, the access to the third generation which is the latest generation is completed only by reading one file (FILE (G3)) (in FIG. 5). At that time, if it is necessary to read only the necessary columns of the latest generation, the record corresponding to the target key is located at which position in the third generation file by combining with the method of querying the index once (indicated in FIG. 5). In the case of a distributed file system, the network I / O can be reduced. In addition, after creating a generation unit file by major compaction, minor compaction occurs, and in addition to the generation unit file, when the file is created, only the latest generation record is read. While reading the file, search the index for a newer record in the file created by minor compaction and return the newer record as the result.

  In this way, records with different time stamps are recognized as the same record with different generations, and records with the same generation are generated as one file, so that a specific generation is recorded continuously in one file. As a result, by accessing a specific generation of files, an access pattern that was previously random access can be changed to sequential access, improving response time and throughput. Since it is not necessary to read unnecessary information, the load on the file system and the network can be reduced.

  In this embodiment, the file dividing unit 30 recognizes records having only different time stamps among the key components as the same record having different generations, and generates records having the same generation as one file. However, if a plurality of records conforming to a predetermined condition are extracted from a plurality of records stored in the disk area 52, and the plurality of records are rearranged in the ascending or descending order of the keys to generate a new file. As described above, it is not limited to generation management.

  Hereinafter, a record reading process in the above-described distributed data store system will be described.

  FIG. 7 is a flowchart for explaining record reading processing in the distributed data store system shown in FIG.

  First, in the record acquisition unit 40, whether the access is to a specific generation or an access to a specific column (multiple selection is possible) according to a record acquisition conditional expression included in a query that is a record acquisition request from a client. According to the determination, whether to read all files of a specific generation or to read using an index is selected (step 1).

  If no generation is specified and access is not to a specific generation, the index is referred to, and a record is read and acquired from the disk area 52 based on the record acquisition request (step 2).

  On the other hand, if a generation is specified and access is to a specific generation, it is determined whether or not access to a specific column is specified in the record acquisition request (step 3), and access to the specific column is specified. In this case, a record is acquired for a specific generation file based on the record acquisition request by the record acquisition function using the index described above (step 4).

  If access to the specific column is not specified, the requested record is directly acquired from the specific generation file based on the record acquisition request (step 5).

  Next, if there is a new file created by the minor compaction function (step 6), the index is referred to for the new file created by the minor compaction function, and the record is recorded on the disc based on the record acquisition request. It is read and acquired from the area 52 (step 7).

  The record acquired from the disk area 52 in this way is returned to the request source of the record acquisition request.

  In the index assigning unit 10, the index is updated and reassigned at the time of execution of minor compaction or major compaction, or when a new specific generation file is generated. That is, when reading a key included in the key range managed by each of the nodes 2-1 to 2-3, there is one or more files that have no key corresponding to the index and have not built an index. If there is, the index is updated at the same time as searching for the target record by the conventional method based on the record acquisition request from the client. Specifically, the index is updated either when a new file is generated by the minor compaction function or when a new file for each generation is generated.

DESCRIPTION OF SYMBOLS 1 Index node 2-1 to 2-3 node 10 Index assignment part 20 Storage control part 30 File division part 40 Record acquisition part 50 Storage part 51 Memory area 52 Disk area

Claims (7)

In a distributed data store system that distributes and manages records specified by identifiers at multiple nodes connected by a network,
The plurality of nodes are:
Record storage means for storing a plurality of records managed by the node as an aggregate for each arbitrary range of the identifier;
Index giving means for giving an index using an identifier included in the range of the aggregate to the aggregate;
A distributed data store system comprising: record acquisition means for acquiring the record requested by the record acquisition request from the record storage means by referring to the index in response to the record acquisition request. The distributed data store system according to claim 1,
The record storage means includes
A memory area having a cache function;
A disk area where records are permanently stored;
Records are accumulated in the memory area in ascending or descending order of the identifiers, and when the size of the record accumulated in the memory area exceeds a predetermined threshold, the record is stored in the disk area as one file unit A distributed data store system having storage control means. The distributed data store system according to claim 2,
A file dividing means for extracting a plurality of records according to a predetermined condition from a plurality of records stored in the disk area, and rearranging the plurality of records in the ascending or descending order of the identifiers to generate a new file; Distributed data store system. The distributed data store system according to claim 3,
The identifier comprises a time stamp as a component,
The distributed file store system, wherein the file dividing unit recognizes records having only different time stamps among the constituent elements of the identifier as the same records having different generations, and generates records having the same generation as one file. The distributed data store system according to claim 4,
The record acquisition unit acquires a record from the record storage unit based on the record acquisition request with reference to the index when a generation is not specified in the record acquisition condition in response to the record acquisition request, and the record acquisition A distributed data store system that acquires a record from the record storage unit based on the record acquisition request from a file of a specified generation when a generation is specified in the condition. The distributed data store system according to claim 5,
The index assigning means is a distributed type that reassigns the index to the aggregate when the record accumulated in the memory area is stored in the disk area or when the new file is generated. Data store system. A record management method in a distributed data store system that distributes and manages a record specified by an identifier in a plurality of nodes connected by a network,
When the plurality of nodes store a plurality of records managed by the node as an aggregate for each arbitrary range of the identifier, an index using an identifier included in the aggregate range for the aggregate A record management method in a distributed data store system that acquires the record requested by the record acquisition request by referring to the index in response to the record acquisition request.

Images