KR20190100537A - Apparatus for Accessing Data Using Internal Parallelism of Flash Storage based on Key-Value and Method thereof - Google Patents

Abstract

The present invention discloses a key value-based data access device, in particular, a data access device in which data input / output performance is improved by fully utilizing the parallelism of a flash memory-based storage device. The key value-based data access device of the present invention comprises: a server for receiving a key list including at least one key value based on a user′s input, and outputting data accessed according to the key list; a storage unit including a storage area divided into a plurality of nonvolatile memory cells for storing the data; and a control unit for requesting and receiving location information of the data stored in the memory cells in parallel, generating a stored data list from the received location information, and accessing the data stored in the memory cells by comparing the generated stored data list with the key list.

Description

Apparatus for Accessing Data Using Internal Parallelism of Flash Storage based on Key-Value and Method

The present invention relates to data access apparatus and methods. More specifically, the present invention relates to a device for accessing data in a flash memory based storage device.

Technologies for processing large amounts of big data stored and processed by cloud computing and multiple data centers are being actively researched. In dealing with big data, fast data retrieval, fast web services, NoSQL databases and other key-value repositories are at the center of the research field.

For example, Facebook and other SNS providers use big data to efficiently store and manage data stored in their databases and technologies for quick access to data stored in their databases that operate to effectively deliver services. Use techniques to

Key-Value Store is a kind of NoSQL database, including Dynamo, Level DB, Rocks DB, Cassandra, and HB BASE. The key value store can manage data more flexibly and efficiently when dealing with big data than a relational database. The key value store uses a solid state driver (SSD) as the main storage for improved data management. Often used. For example, RocksDB on Facebook, an open source key-value based database, provides a multi-get method to eliminate the overhead of repetitive calls to Get methods. This allows Rocks DB to speed up data access.

However, the conventional data access method merely repeats the get method internally, and has not limited the improvement of data I / O performance because the internal parallelism of the NAND flash memory-based SSD cannot be fully utilized.

The flash memory based solid state drive (SSD) has a characteristic of processing data in parallel due to the parallelism of a plurality of flash memory chips, and it is required to develop a data access technology in a key value store. .

Korean Unexamined Patent No. 10-2010-0121389 (Public)

The present invention has been made to solve the above problems, and discloses a key value based data access apparatus. In particular, an apparatus for accessing data stored in a flash memory is disclosed.

The present invention has been made to achieve the above object, the key value-based data access apparatus of the present invention receives a key list including at least one key value based on a user input, according to the key list A server for outputting the accessed data; A storage unit including a storage area divided into a plurality of nonvolatile memory cells for storing the data; And requesting and receiving position information of data stored in the memory cells in parallel, generating a stored data list from the received position information, comparing the generated stored data list with the key list, and storing the stored data in the memory cells. A control unit for accessing data; It includes.

In the present invention, the server may receive the key list using at least one Key Value Operations Interface operating based on the key values.

In the present invention, the memory cells include a nand type flash memory provided with a multi-channel for the data transmission, and the flash memory is pre-assigned a key range according to the size of the key values. It may include a plurality of blocks for storing separately.

In the present invention, the storage unit stores the data in the memory cells in a log structure merged tree (Log Structure Merged Tree) method for processing the data hierarchically, and the data stored in the memory cells are divided into at least one layer and the hierarchical layer. It can be managed through liver compression.

According to an embodiment of the present invention, the control unit may further include a storage data list generation unit configured to sequentially generate the storage data list from a lower layer of data stored in the memory cells in the log structure merging tree method using the position information; The apparatus may further include accessing data stored in the memory cells by using the generated stored data list.

In the present invention, the control unit sequentially compares the generated stored data list and the key list from a lower layer of the data, and returns the data stored at the position indicated by the position information corresponding to the key list in a return data list. A data list storage unit; The apparatus may further include and transmit the returned data list to the server after comparing to the highest layer of the layers.

In the present invention, the control unit collectively receives the position information of the data stored in the memory cells using a Parallel Synchronous I / O Interface operating based on an operation variable, wherein the operation variable is a buffer, It may include at least one of an I / O parameter and a set of pointers associated with the key values.

In the present invention, the location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data, and the I / O request set is located at a location of data stored in the memory cells. Can be ordered accordingly.

In the present invention, the block includes a bloom filter file that determines whether the data is included in the block by using a preset hash function having a hash value as an output, and when the data is included in the bloom filter file. It may include an index file for outputting offset information about which position of the block.

In the present invention, the compression is performed in parallel in the at least one layer, the compression is periodically performed in consideration of the number of threads, and the storage unit uses a plurality of key-value pairs. The data may be stored in a log structure merged tree method.

In addition, in order to achieve the above object, a key value-based data access method of the present invention comprises: receiving a key list including at least one key value based on a user input; Requesting location information of the data stored in a plurality of nonvolatile memory cells for storing data in parallel; Generating a stored data list representing a data list stored in the memory cells from the location information; Comparing the key list with the generated stored data list to access data stored in the memory cells; It includes.

In the present invention, the step of receiving the key list may be provided to receive the key list using at least one Key Value Operations Interface operating based on the key values.

In the present invention, the memory cells include a nand type flash memory provided with a multi-channel for the data transmission, and the flash memory is pre-assigned a key range according to the size of the key values. It may include a plurality of blocks for storing separately.

In the present invention, the memory cells are stored in a log structure merged tree method for processing the data hierarchically, and the stored data is divided into at least one layer to perform the inter-layer compression. Can be managed through

In the generating of the present invention, the stored data list may be sequentially generated from a lower layer of data stored in the memory cells by the log structure merging tree method using the location information.

In the present invention, the retrieving may include sequentially comparing the generated stored data list and the key list from a lower layer of the data; And storing data stored at a location indicated by the location information corresponding to the key list in a return data list according to the comparison result. The apparatus may further include and return to the user a list of return data in which the data is stored after comparing to the uppermost layer of the data.

In the generating of the present invention, the location information of the data stored in the memory cells is collectively received using a parallel synchronous I / O interface operating based on an operation variable, and the operation variable is It may include at least one of a buffer, an I / O parameter and a set of pointers associated with the key values.

In the present invention, the location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data, and the I / O request set is located at a location of data stored in the memory cells. Can be ordered accordingly.

In the present invention, the block includes a bloom filter file that determines whether the data is included in the block by using a preset hash function having a hash value as an output, and when the data is included in the bloom filter file. And an index file for outputting offset information regarding a location of the block, wherein the memory cells use a plurality of key-value pairs to log the data into a log structure merged tree. Can be stored in the

The present invention also discloses a computer program stored in a computer readable recording medium for executing the above key value based data access method on a computer.

According to the present invention, the input / output performance of the data access device can be improved.

In particular, there is an advantage that the data stored in the memory cell can be accessed effectively.

1 is a block diagram of an apparatus for accessing data based on a key value according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram illustrating data stored in a log merge tree structure in a storage unit in the embodiment of FIG. 1.
3 is an enlarged block diagram of a controller in the embodiment of FIG. 1.
4 is a conceptual diagram illustrating a structure of data stored in a flash memory based storage device.
5 shows a data access procedure performed in a conventional key value based data access apparatus.
6 illustrates a data access process performed in a key value based data access apparatus according to an embodiment of the present invention.
7 illustrates the performance of a key value based data retrieval apparatus measured according to a change in the number of multi-gets.
8 shows the performance of the data retrieval device based on the key value measured according to the change of the multi-get ratio.
9 shows the performance of a key value based data retrieval apparatus measured according to the size of a data block.
10 is a flowchart illustrating a key value based data retrieval method according to an embodiment of the present invention.
FIG. 11 is an enlarged flowchart of accessing in the embodiment of FIG. 10; FIG.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In the description with reference to the accompanying drawings, the same or corresponding components will be given the same reference numerals and redundant description thereof will be omitted.

In describing the present invention, when it is determined that detailed descriptions of related well-known structures or functions may obscure the gist of the present invention, the detailed description may be omitted.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of terms. Singular expressions include plural expressions unless the context clearly indicates otherwise. Each step described below may be provided by one or several software modules or may be implemented by hardware that is responsible for each function, or may be a combination of software and hardware. Specific meanings and examples of each term will be described below in the order of the drawings. Hereinafter, a configuration of a key value based data access device 10 according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an apparatus 10 for accessing data based on a key value according to an embodiment of the present invention.

The key value-based data access device 10 includes a server 100, a storage unit 200, and a controller 300. For example, the key value based data access device 10 may be used to access data stored in a Key-Valute Store. The key-value store referred to in the present invention is a kind of NoSQL database. Value, which is data corresponding to a corresponding key value by using a key value or a pointer set, in a nonrelational database management system for processing big data. ) Is a system for managing. In general, the key-value store optimizes key values for simple retrieval and additional operations in a distributed environment without schemas such as table-column, so the response latency (LATENCY) and data throughput (throughput) according to input request are excellent. There is this.

The key value-based data access device 10 of the present invention is a representative key-value store and can be operated on a RocksDB platform that is suitable for big data processing such as a server workload and performs high performance in a fast flash storage device. RocksDB can run in a variety of environments, including memory, flash, hard disk, and HDFS. It is based on the open source project LevelDB. Key-values referred to in the present invention are pure bit streams, and in the present invention, all key values called and returned through a multi get application programming interface (API) maintain consistency.

RocksDB in which the key value based data access device 10 of the present invention is driven is an application programming interface (API) for processing a plurality of key-value pairs. Use get method. Multi-get method or get method is an operation for the application to call a key in RocksDB. Multi-get allows multiple key values to be called at the same time. However, the conventional multi-get only repeats the get method internally, and there is a problem in that the internal parallelism of the NAND flash memory-based SSD cannot be properly used.

The key-value store in which the key value-based data access device 10 of the present invention is used can manage data more flexibly and efficiently than the conventional relational database, and uses SSD as main storage for improved performance. There are many cases. Key value stores can improve performance by taking advantage of the random I / O request characteristics of SSDs while reducing possible write amplifications. The key value-based data access device of the present invention not only delivers multiple I / O requests in a single system call, but also sorts the plurality of I / O requests. This allows parallel sequential access to data stored in memory cells including a plurality of flash chips, and enables a flash memory based storage device to achieve improved I / O performance.

That is, the key value-based data access apparatus 10 of the present invention enables multiple read accesses to data stored in memory cells in a single PSYNC system call using PSYNC I / O. By changing a pattern for accessing data stored in a key-value database from iterative random access to parallel sequential access, I / O performance of a flash memory based storage device may be improved. As a result, when using the data access device 10 based on the key value, not only can the data be accessed more quickly, but also the bandwidth of the SSD can be improved.

The server 100 receives a key list including at least one key value based on an input of the user 20, and outputs data accessed according to the key list. For example, the server 100 may be separately provided from the storage unit 200 and the control unit 300, but may be separately provided, but may be implemented as a platform for managing data in the storage unit 200. The server 100 may receive a key list from the user 20 and may receive and output a return data list, in which data accessed corresponding to the key list is stored, from the controller 300. The key list input by the server 100 of the present invention may be provided in a binary sequence indicating the location of data stored in the storage unit 200.

For example, the server 100 may receive the key list using at least one Key Value Operations Interface operating based on key values. The interface operating based on key values used by the server 100 includes an application programming interface (API) for processing a plurality of key-value pairs, and includes a multi-get method. Alternatively, it may be provided as a get method. The server 100 may receive a key list from a user by using a multi-get method or a get method, which is an operation for calling a key in RocksDB. It demonstrates with reference to FIG.

The storage unit 200 includes a storage area divided into a plurality of nonvolatile memory cells for storing data. For example, the memory cells included in the storage unit 200 include a NAND flash memory provided with a multi-channel for data transmission, and the flash memory includes a key range according to the size of the key values. ) May be pre-allocated and include a plurality of blocks for classifying and storing the data. The block may include data from a bloomfilter file and a bloom filter file that determine whether the data is included in the block by using a preset hash function having a hash value as an output. And an index file for outputting offset information on which position of the block the data is located.

The storage unit 200 manages data stored in the plurality of memory cells in a log structure merge tree (LSM tree) as described above. LSM Tree is a BaseTree algorithm used in key value stores such as LevelDB, RocksDB, and Hbase. It is a data structure designed for Write Intensive Data Workload, and can maintain low write latency. have.

For example, a log structure merge tree (LSM tree) in which data is hierarchically managed is flushed to disk storage only when memory is full, thereby achieving low write latency. The LSM tree may avoid accessing deleted data using Tomstone entries that separately mark deleted data items, instead of updating or deleting the data for low write latency.

Deletion of data stored in memory cells provided in the storage unit 200 of the present invention may be performed by an inter-layer merge operation of the LSM tree. The LSM tree may delete the data stored in the actual tree component by merging the tree component of the upper layer into the tree component of the lower layer when the size of the data stored in the tree component between layers is greater than or equal to a preset threshold. .

Referring to the movement of data from the log merge tree structure according to the embodiment, first, the disk area receives the data from the C ₀ level belonging to the In-Memory area, through the difference plane Immutable Memtable all of the memtable included in C ₀ level Flushed. If in the log merge tree structure levels included in the disk area C ₁ level stores the data flushed from C ₀ level, and larger than the size of data stored in the C ₀ level previously set, merge tree component in C ₁ level ( Merge). Similarly, when the size of data stored at the C ₁ level is greater than or equal to a preset threshold, the C ₁ level data may be merged to the C ₂ level.

When the merge cursor recognizes the tomstone data mark that marks the deleted data item in the above-described merge operation of the inter-layer tree components, deletion of data may occur. In general, many key value stores use the LSM tree to manage stored data to achieve fast write speeds, but due to the limitations of the multi-layered nature of the LSM tree, bLSM, LSM- to improve read performance without attenuation of write latency. Techniques like trie and SILT have been developed. In the present invention, the above-described memory and disk storage may be implemented by being included in the storage 200, but the memory may be implemented separately from the storage 200 and on a RocksDB processor on the controller 300. Of course.

RocksDB, in which the key value based data access device 10 of the present invention is driven, is based on LevelDB and uses the characteristics of SSD to optimize the performance of the key-value store. As a result, RocksDB operates on flash memory-based storage, making it more efficient to write data. In addition, RocksDB may use a parallel filter corresponding to a merge between layers in a bloom filter and the aforementioned LSM tree to improve read performance.

For example, RocksDB used as a data management scheme of a key value-based data access device 10 includes an in-memory table 224 and 226 to which a first in first out (FIFO) algorithm is applied. Data stored in 200 may be maintained in an LSM tree structure.

In the present invention, RocksDB may mean a data system stored in the storage unit 200 in an LSM tree manner, but may refer to a RoclsDB platform that performs a function of the controller 300 in the data access device 10 based on a key value. have. First, the data access apparatus 10 based on a key value receives data into an in-memory table. The in-memory table includes a memtable (224) and an immutable memtable (226) in which the data stored therein are not affected by the input of the data. When the size of the data stored in the memtable exceeds a predetermined threshold Data stored in memtable is transferred to immutable memtable which is not affected by data input / output. In this case, when the size of data stored in the immutable memtable is greater than or equal to a preset threshold, the data may be flushed to the storage 200.

Data stored in memory cells of the storage unit 200 may be managed by the RocksDB platform. LevelDB based system uses SST file to manage data hierarchically like LSM tree. Each level 262, 264, 266 of the LevelDB system corresponds to each tree component of the LSM tree, and the data of each level is sorted and stored in the SST file. The memory cells of the storage unit 200 of the present invention include flash-based memory cells. Therefore, when data is managed hierarchically based on an SST file, an efficient data writing operation is possible.

The SST file used in the LevelDB-based system includes metadata including the maximum minimum value of the data set, and the metadata includes information about a location where the data is stored, and includes a bloom filter file and an index file. That is, a LevelDB-based system can access metadata by using meta data as an index to access data stored in a specific block provided in memory cells.

For example, the storage unit 200 stores the data in the memory cells in a log structure merged tree method that processes the data hierarchically, and the data stored in the memory cells are stored in at least one layer. It can be divided and managed through the inter-layer compression. In the key-value store, compression is performed in parallel in the at least one layer, and the compression is periodically performed in consideration of the number of threads, and the storage unit 200 includes a plurality of key-values. The data may be stored in a log structure merged tree method using a pair).

In the present invention, when the storage unit 200 is implemented as a NAND flash chip-based SSD including a block having a plurality of pages, the storage unit 200 is a block for accessing data under the control of the controller 300. Includes the Bloom filter file and the index file as described above. For example, the bloom filter uses statistical logic to determine whether data of the corresponding key is stored in the SST file and returns the result. The data of the bloom filter of each SST file is loaded into the memory. Before accessing the SST file and generating disk I / O, it is first determined whether there is data in the SST file. For example, the bloom filter may determine whether there is data requested in a corresponding block by using hash values output from a predetermined number of hash functions.

For example, when it is determined that there is data in the SST file in the block, the key value-based data access device 10 searches for the index file. The index file has position information (file pointer and offset information) about where in the block the data corresponding to the key value is located. That is, the plurality of blocks included in each of the memory cells of the storage unit 200 include a bloom filter, and the controller 300 determines whether or not the requested data exists in each block. Including the index file, the control unit 300 may allow the controller 300 to search where the data is located.

The SST file of a block provided in the storage unit 200 and storing data may be gradually enlarged. In this case, the data access device 10 based on the key value may use the above-mentioned compression to generate the SST file of the target level. It can be integrated into the next level of SST file (for example, 262 to 264 or 264 to 266), and in this way, the number and size of data stored in the storage 200 can be adjusted. The key value based data access device 10 allows RocksDB to access data faster based on the above-mentioned compression.

In particular, in the present invention, the compaction may be processed by dividing the major compaction and the major compaction into a plurality of sub-compactions. In this case, in the sub-compaction, the compression range is divided by the number of threads, and each thread is assigned a small compaction. Can be treated as key-value pairs (KeyValuePairs). As a system for managing data stored in the storage unit 200, RocksDB may achieve superior performance than other key-value storage devices through multi-threaded compression and block-based data access. However, despite the above-described features, the conventional RocksDB multi-get has a limitation in that it cannot fully utilize the internal parallelism of the SSD. It demonstrates with reference to FIG. 3 and FIG.

The control unit 300 includes a storage data list generation unit 320 and a return data list storage unit 340. For example, the controller 300 requests and receives location information of data stored in memory cells in parallel, generates a stored data list from the received location information, and stores the generated stored data list and the key list. Compare and access data stored in the memory cells.

For example, the storage unit 200 storing data to be accessed by the controller 300 using a key list input from a user may be provided as an SSD as a flash memory based storage device. The SSD includes a flash chip including a plurality of planes, an SSD controller provided with a processor and a RAM. In the structure of the SSD shown in FIG. 3, the SSD controller may be provided in the controller 300 of the present invention, and an area in which a plurality of flash chips are provided may be provided as the storage unit 200 of the present invention. Of course, it can be integrated into the implementation.

The control unit 300 transmits the multi I / O REQUEST to a plurality of memory cells (flash chip) 272 274 included in the storage unit 200, and the memory cells transmit data to the control unit 300 using multiple channels. Can transmit According to an embodiment of the storage unit 200 in which data accessed by the control unit 300 is stored, the SSD stores data in page unit blocks, each page size may be provided as 4 to 8 KB, and the plurality of pages may be one Form a block. SSDs with a block-based structure can process data in units of pages, and deletion can only be done in units of blocks, not in units of pages. Due to the internal parallelism of the SSD, sequential access is efficient when the controller 300 attempts to access page-based data to the storage 200.

In the present invention, the control unit 300 collectively receives the position information of the data stored in the memory cells using a Parallel Synchronous I / O Interface (PSYNC I / O) operating based on an operation variable. For example, the operation variable used by the controller 300 may include at least one of a buffer, an I / O parameter, and a pointer set associated with the key values. In addition, the location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data, the I / O request set according to the location of data stored in the memory cells. Can be ordered sequentially. It demonstrates with reference to FIG.

Conventionally, in order to use the internal parallelism of an SSD in RocksDB where a key value based data access device is used, the PSYNC I / O interface is used to transmit multiple I / O requests in a single system call. While the techniques have been studied, the conventional technique merely delivers multiple I / O requests to the multi using PSYNC I / O, and attempts random access in attempting to access data in the page-based blocks of the SSD. It was. However, when the storage unit 200 is a flash memory-based storage device, since the sequential access is more efficient in view of storing data in a page-based block structure, the conventional technology does not fully utilize the internal parallelism of the SSD. Thus, there is still room for improving the performance of read operations in block-based data structures as described above.

The conventional key value-based data access apparatus 10 accesses data by repeating a get operation through a multiget method until the target key list is reached, and the get method performs a binary search. It is performed using the Index Iterator and the Block Iterator to access the data provided in the SST file in the aforementioned block. However, the conventional conventional key value-based data access device 10 transmits only one I / O request at a time and accesses data stored in the SSD, so that a large number of random patterns according to one I / O request are provided. Because of the access, the performance of the SSD could not be fully improved. In particular, the conventional key value-based data access device 10 causes inefficiency due to an increase in internal fragmentation due to small sized data when the block size is smaller than the page size of the SSD.

The stored data list generator 320 sequentially generates the stored data list from the lower layers of the data stored in the memory cells using the log structure merging tree method using the received position information. For example, the location information used by the storage data list generator 320 may include a bloom filter file, an index file, a file pointer, and offset information as information about a location of data stored in memory cells of the storage 200. have. The control unit 300 of the present invention includes a stored data list which is a list of all data stored in the memory cells of the storage unit 200 through the stored data list generation unit 320 and compares the stored data list with the key list.

The return data list storage unit 340 sequentially compares the generated stored data list with the key list from a lower layer of the data and stores the data stored at the position indicated by the position information corresponding to the key list in the return data list. do. For example, in comparing the stored data list and the key list, the return data list storage unit 340 starts the comparison from the list of the stored data list corresponding to the lower layer of the storage unit 200 in which data is stored in a hierarchical structure. Thus, the list of stored data lists corresponding to the top layer may be sequentially compared. It demonstrates with reference to FIG.

The key value-based data access device 10 of the present invention can fully utilize internal parallelism when the storage unit 200 is provided as a flash memory-based storage device by using the following Multipath Multiget Method algorithms. Multipath Multiget Method algorithms are as follows.

Algorithm 1 of Table 1 is an algorithm representing a PSYNC I / O process. Here, operation variables for the PSYNC I / O interface may be prepared in advance.

The operation variable includes a buffer, an I / O parameter and a set of pointers associated with the key values.

Algorithm 2 of Table 2 illustrates a multipath multiget method process used by the key value-based data access device 10 of the present invention. Algorithm 2 of Table 2 used by the key value based data access device 10 includes metadata collection for disk access in memory (lines 7 to 16), classification of collected metadata (lines 17), and I / O. Contains the request algorithm (lines 18-28).

First, the controller 300 using the multipath multiget method algorithm of the present invention collects location information for data access. Location information for data access includes file descriptors, offsets, information about the size of the data, and other sets of I / O requests. The data of the storage unit 200 of the present invention are stored in advance by dividing a key range (KeyRange) in consideration of minimum or maximum values of key values into pre-allocated memory cells, and more specifically, SST of an internal block provided in the memory cells. Each file is pre-assigned a key range (KeyRange) according to the size of the key value.

The control unit 300 of the present invention uses the location information including the above-described file descriptors, offsets, and other I / O request information sequentially arranged according to the location of the data, thereby using the memory cells of the storage unit 200. The data stored in can be accessed sequentially. More specifically, the controller 300 of the present invention uses the I / O request set among the location information collectively received using PSYNC I / O in order according to the location of the data to sequentially use the memory cells of the storage unit 200. The data stored in can be accessed sequentially.

2 illustrates data stored in the storage 200 in a log merge tree structure in the embodiment of FIG. 1. The storage unit 200 manages data stored in a plurality of memory cells as a tree component having a hierarchical structure, and the size of the data stored in each tree component is preliminary. If the value is equal to or greater than the set threshold, merging into lower tree components is as described above. In a key-value store such as LevelDB or RocksDB, which uses an LSM tree as a base tree algorithm, data is input from an in-memory area, and when data of an immutable memtable is filled, data is flushed to the storage unit 200 or disk storage.

In the present invention, the storage unit 200 includes memory cells in which a plurality of blocks are provided, and each block hierarchically stores data including an SST file as described above. In the memory cells of the storage unit 200, a key range according to the size of key values is pre-assigned to store data separately. More specifically, each block provided in the memory cells is configured in consideration of a maximum minimum value of data. Is pre-assigned, allowing data to be stored separately. For example, the SST file 1 indicated by file1 in L1 may have a key range of 100,200.

3 is an enlarged block diagram of the controller 300 in the embodiment of FIG. 1.

The control unit 300 includes a storage data list generation unit 320 and a return data list storage unit 340. For example, the controller 300 requests and receives location information of data stored in memory cells in parallel, generates a stored data list from the received location information, and stores the generated stored data list and the key list. In comparison, accessing data stored in the memory cells is the same as described above, and thus will be omitted.

4 is a conceptual diagram illustrating a structure of data stored in a flash memory based storage device.

The SSD may include an SSD controller 300 and a plurality of flash chips 272 and 274. However, as described above, in the present invention, the SSD controller may be implemented as the controller 300 and the region including the plurality of flash chips as the storage 200. The memory cells of the storage unit 200 may include a plurality of blocks and may store data in layers. For example, one block included in the memory cells of the storage unit 200 may include an SST file, and the SST file may include a bloom filter file and an index file to access data stored in the block.

5 shows a data access procedure performed in a conventional key value based data access apparatus.

A conventional key value-based data access device receives a key list including at least one key value from a user 20, and uses a single key I / O to store data stored in the storage unit 200 using the input key list. The O requests are used to access a repetitive random access pattern. Since SSD uses page-based blocks to store data, it is efficient to access the data in sequential patterns when accessing data. Conventional key value-based data access devices use a single I / O request to access data. As described above, the performance of the SSD was not fully utilized since the access was performed in a random access pattern.

6 illustrates a data access process performed in a key value based data access apparatus according to an embodiment of the present invention.

The key value-based data access apparatus 10 of the present invention uses a parallel synchronous I / O interface (PSYNC I / O) that operates based on an operation variable of position information of data stored in memory cells. Receive a batch. In addition, the location information received by the key value-based data access device 10 using the parallel synchronous I / O interface (PSYNC I / O) collectively includes an I / O request set for requesting input / output of data. I / O request sets may be sequentially arranged according to positions of data stored in the memory cells.

Accordingly, the key value-based data access device 10 of the present invention uses the PSYNC I / O interface to utilize the internal parallelism of the SSD in RocksDB. ), And sequential access in attempting to access data in the page-based blocks of the SSD, it is possible to improve the performance of the key value-based data access device. When the key value-based data access device 10 includes a flash memory-based storage device as a storage unit, the data access device 10 uses the internal parallelism of the SSD completely through efficient sequential access.

7 illustrates the performance of the data retrieval apparatus 10 based on key values measured according to the change in the number of multi-gets.

Unlike the conventional Original Multiget method, the performance of the key value-based data access device 10 using the Sequemtial Multiget method was measured using the RocksDB benchmark. In order to distinguish the disk performance difference, the OS buffer was set to disabled at the time of measurement, and specific hardware and software settings are as follows.

Table 3 above shows the hardware configuration. In order to measure the performance of the key value-based data access device 10, the CPU used Interl Core i7-6700K CPU @ 4.00 GHz, and measured using 64 GB of memory and Samsung 850 Pro 512 GB.

Table 4 above shows the software settings. In order to measure the performance of the key value based data access device 10, the benchmark tool was used in RocksDB version 4.4.1 environment, and the fillseq, fillrandom, and multiget benchmarks were used to measure the performance of RocksDB. For the measurement, the key value is set to 16 byte random integer, the value for each key is set to 512 bytes, and 8KB block is used considering the size of flash page is 8KB. In the charts of FIG. 7, the range of multiget entries is set to 1000 to 1000000 and the fetch size is set to 8MB to 8GB. 7 (a) and 7 (c), the key value-based data access device 10 of the present invention using the sequemtial multiget 704 has a lower elapsed timse than the conventional technique 702, and has a multiget. As the number of entries increases, the performance difference becomes noticeable. As described above, the difference in performance is due to the sequential access of the key value-based data access device 10 to the SSD using the aligned offset information. FIG. 7 (b) shows the time taken to classify the multi-gets per key. On the average, 0.1 us is consumed and the effect of the sorting cost is negligible as the amount of data increases.

8 shows the performance of the data retrieval device based on the key value measured according to the change of the multi-get ratio. 9 shows the performance of a key value based data retrieval apparatus measured according to the size of a data block.

Where multiget ratio is the multi-get ratio, number of multiget keys is the number of multiget keys, and number of total keys is the number of total keys. The performance of the key value based data access device 10 may vary depending on the multiget ratio. In the multi-get ratio varying from 1 to 100%, the performance of the key value based data access device 10 is as shown in FIG. The value size is fixed to 8KB, which is the same as the size of the page in the block, and as a result of changing the multi-get ratio, it can be seen that the performance of the key value-based data access device does not change significantly.

The multipath multi get (MPMG) used by the key value-based data access device of the present invention exhibited better performance than the case of using the conventional multiget method as the batch size increased. Is 10000, the performance of the key value-based data access device 10 using the MPMG method of the present invention is 100% improved compared to the case of using the conventional multiget method. The key value-based data access device 10 of the present invention has the most improved performance compared to the conventional multiget method when the value size is 256 bytes. Key value-based data access device 10 of the present invention operates effectively when processing a large number of multiple get operations, and parallelism of flash-based storage device because it uses multiple batch asynchronous i / o unlike conventional techniques Can be used completely. Accordingly, the key value-based data access device 10 may transmit a larger number of multiple i / o requests by accessing the disk file in a parallel access pattern, unlike the conventional method of simply accessing the disk file in a random access pattern. There is an advantage. When the key value-based data access device 10 of the present invention is used, the query execution time of the RocksDB benchmark can be reduced by 80%, and the larger the multiget entry size, the better the performance is as described above.

10 is a flowchart illustrating a key value based data retrieval method according to an embodiment of the present invention.

The key value based data retrieval method includes the following steps performed in time series in the key value based data retrieval apparatus 10.

In S100, the server 100 receives a key list including at least one key value based on an input of the user 20. For example, the server 100 may receive the key list using at least one Key Value Operations Interface operating based on the key values. The key release operation interface used by the server 100 may be provided by multiget (). In addition, as described above, the key values received by the server 100 may be provided in a binary sequence indicating a location of stored data.

In S200, the controller 300 requests position information of the data stored in a plurality of nonvolatile memory cells for storing data in parallel. As described above, the controller 300 may use PSYNC I / O in requesting location information of data stored in the plurality of nonvolatile memory cells.

In S300, the storage data list generation unit 320 generates a storage data list indicating a data list stored in the memory cells from the location information. For example, the stored data list generator 320 may sequentially generate the stored data list from lower layers of data stored in the memory cells using the log structure merging tree method using location information. In addition, the storage data list generation unit 320 collectively receives the position information of the data stored in the memory cells using a parallel synchronous I / O interface operating based on an operation variable, and performs the operation. As described above, the variable includes at least one of a buffer, an I / O parameter, and a set of pointers associated with the key values. Here, the location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data, the I / O request set according to the location of data stored in the memory cells. Can be ordered sequentially.

For example, the memory cells of the present invention include a NAND flash memory provided with a multi-channel for the data transfer, and the flash memory is pre-assigned by a key range according to the size of the key values. As described above, it may include a plurality of blocks that classify and store the data. In addition, the memory cells are stored in a log structure merged tree method for processing data hierarchically, and the stored data are divided into at least one layer and managed through the inter-layer compression. It may be as described above.

The block may include the bloom filter file determining whether the data is included in the block using a predetermined hash function having a hash value as an output, and the data is determined to include the data in the bloom filter file. And an index file for outputting offset information regarding a location of a block, wherein the memory cells use a plurality of key-value pairs to log the data into a log structure merged tree. Can be stored in such a way. In S400, the controller 300 compares the generated stored data list with the key list to access data stored in the memory cells. A detailed method of accessing the data stored in the storage 200 by comparing the generated data list with the key list by the controller 300 is as described above, and thus will be omitted.

FIG. 11 is an enlarged flowchart of accessing in the embodiment of FIG. 10; FIG.

In S420, the controller 300 sequentially compares the generated stored data list with the key list from a lower layer of the data. For example, since the data stored in the storage unit 200 are managed by the LSM tree as a hierarchical structure, the stored data list may also be created in a lower layer to a higher layer direction.

In S440, the controller 300 stores the data stored at the position indicated by the position information corresponding to the key list in the return data list according to the comparison result. In more detail, the return data list storage unit 340 provided in the control unit 300 stores the data stored at the position indicated by the position information corresponding to the key list in the return data list according to the comparison result. For example, in comparing the key list and the stored data list, the controller 300 may access the data stored in the memory cells by using the return data list in which the data is stored after the comparison to the top layer of the data.

All or part of the method of one embodiment of the present invention described above may be implemented in the form of a computer-executable recording medium (or a computer program product), such as a program module executed by a computer. Here, the computer readable medium may include a computer storage medium (eg, memory, hard disk, magnetic / optical media or solid-state drive, etc.). Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.

In addition, all or part of the method according to an embodiment of the present invention includes instructions executable by a computer, the computer program including programmable machine instructions processed by a processor, and a high-level programming language. Language, an object-oriented programming language, an assembly language, or a machine language.

Means or modules in the present specification may mean hardware capable of performing functions and operations according to each name described in the present specification, and computer program code capable of performing specific functions and operations. It may mean an electronic recording medium, for example, a processor or a microprocessor, on which computer program code capable of performing specific functions and operations may be implemented. In other words, a means or module may mean a functional and / or structural combination of hardware for performing the technical idea of the present invention and / or software for driving the hardware.

Thus, a method according to an embodiment of the present invention may be implemented by executing a computer program as described above by a computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device. Each of these components are connected to each other using a variety of buses and may be mounted on a common motherboard or otherwise mounted in a suitable manner.

The above description is merely illustrative of the technical idea of the present invention, and various modifications, changes, and substitutions may be made by those skilled in the art without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (20)

A server for receiving a key list including at least one key value based on a user's input and outputting data accessed according to the key list;
A storage unit including a storage area divided into a plurality of nonvolatile memory cells for storing the data; And
Requesting and receiving position information of data stored in the memory cells in parallel, generating a stored data list from the received position information, comparing the generated stored data list with the key list, and storing the stored data in the memory cells A control unit for accessing the control unit; Key value-based data access device comprising a. The method of claim 1, wherein the server
And a key list based on at least one Key Value Operations Interface operating based on the key values. The method of claim 1,
The memory cells may include a NAND flash memory provided with a multi-channel for data transmission, and the flash memory may be pre-allocated with a key range according to the size of the key values to distinguish the data. Key value-based data access apparatus comprising a plurality of blocks for storing. The method of claim 2, wherein the storage unit
Storing the data in the memory cells in a log structure merged tree method for hierarchically processing the data;
And data stored in the memory cells are divided into at least one layer and managed through the inter-layer compression. The method of claim 4, wherein the control unit
A storage data list generation unit configured to sequentially generate the storage data list from lower layers of data stored in the memory cells in the log structure merging tree manner using the position information; More,
And accessing data stored in the memory cells using the generated stored data list. The method of claim 4, wherein the control unit
A return data list storage unit for comparing the generated stored data list with the key list sequentially from a lower layer of the data and storing data stored at a position indicated by the position information corresponding to the key list in a return data list; More,
And returning the list of return data to the server after comparing to the highest layer of the layers. The method of claim 5, wherein the control unit
Collectively receive position information of data stored in the memory cells using a parallel synchronous I / O interface operating based on an operation variable,
And said operation variable comprises at least one of a buffer, an I / O parameter and a set of pointers associated with said key values. The method of claim 7, wherein
The location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data.
And the I / O request set is sequentially arranged according to the location of data stored in the memory cells. The method of claim 3, wherein the block is
If it is determined that the data is included in the bloom filter file and the bloom filter file that determine whether the data is included in the block by using a preset hash function having a hash value as an output, the data is located at any position of the block. Contains an index file that outputs offset information as to whether
And the key list is provided in a binary sequence indicating a position of the data. The method of claim 7, wherein
The compaction is performed in parallel in the at least one layers, and the compaction is periodically performed in consideration of the number of threads.
And the storage unit stores the data in a log structure merged tree method using a plurality of key-value pairs. Receiving a key list including at least one key value based on a user input;
Requesting location information of the data stored in a plurality of nonvolatile memory cells for storing data in parallel;
Generating a stored data list representing a data list stored in the memory cells from the location information; And
Comparing the generated stored data list with the key list to access data stored in the memory cells; Key value-based data access method comprising a. The method of claim 11, wherein the step of receiving the key list is
And receiving the key list by using at least one Key Value Operations Interface operating based on the key values. The method of claim 11,
The memory cells include a nand type flash memory provided with a multi-channel for the data transfer.
The flash memory has a key value-based data access method, characterized in that it comprises a plurality of blocks for pre-allocating the key range according to the size of the key values to store the data separately. The method of claim 12, wherein the memory cells
And storing the data hierarchically in a log structure merged tree method, wherein the stored data is divided into at least one layer and managed through the inter-layer compaction. Key value based data access method. 15. The method of claim 14, wherein said generating is
And generating the stored data list sequentially from a lower layer of data stored in the memory cells by using the location information in the log structure merging tree method. 15. The method of claim 14 wherein the accessing step is
Sequentially comparing the generated stored data list and the key list from a lower layer of the data; And
Storing data stored at a location indicated by the location information corresponding to the key list in a return data list according to the comparison result; More,
And accessing the data stored in the memory cells by using a return data list in which the data is stored after comparing to the uppermost layer of the data. The method of claim 15, wherein the generating step
Collectively receive position information of data stored in the memory cells using a parallel synchronous I / O interface operating based on an operation variable,
And said operation variable comprises at least one of a buffer, an I / O parameter and a set of pointers associated with said key values. The method of claim 17,
The location information collectively received using the parallel synchronous I / O interface includes an I / O request set for requesting input and output of data.
And the I / O request set is sequentially arranged according to the location of data stored in the memory cells. The method of claim 13, wherein the block is
If it is determined that the data is included in the bloom filter file and the bloom filter file that determine whether the data is included in the block by using a preset hash function having a hash value as an output, the data is located at any position of the block. Contains an index file that outputs offset information as to whether
The memory cells store the data in a log structure merged tree method using a plurality of key-value pairs.
And the key list is provided as a binary sequence indicating a position of the data. A program stored in a computer-readable recording medium which realizes the key value-based data access method according to any one of claims 11 to 19 through execution by a processor.

Images