KR20090065136A - Method of data storing in memory page with key-value data model - Google Patents

Abstract

A method of storing data in a memory page for a key value data model is provided to store data divided by a key value unit in a page inside a main page, thereby largely reducing cache miss. In page header information, an address of a low key page is searched(S301). A low key is searched from the low key page by using an offset table of a page(S302). When a key page of a corresponding column exists, a key page address of the corresponding column is obtained. An address of a value page is obtained based on the low key data(S305). Searching query data is performed by searching the value data(S306).

Description

Method of Data Storing In Memory Page With Key-Value Data Model}

The present invention relates to a method for storing data in a memory page that is suitable for the key-value data model and that can optimize the cache by configuring column pages into key pages and value pages.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. [Task Management Number: 2007-S-016-01, Title: Low-cost large-scale global Internet service solution Development].

In the existing disk-based relational database environment, the corresponding page stored in the disk is loaded into main memory for query processing, and the data structure stored in the page is interpreted, and a part of the data is loaded into the cache and processed.

At this time, since the access pattern for the data loaded in the main memory page is different according to the query pattern, it is necessary to construct a page that can minimize the cache miss as much as possible.

In general, commercial relational databases store data on disk on a record-based basis, which uses the N-ary Storage Model (NSM) to organize data records within memory pages. The NSM stores the data sequentially in record units in a memory page and places an offset table at the end of the page to store the position of each record. However, since most queries use only a part of a record, that is, some columns, data that is not required for operation is loaded into the cache, causing unnecessary I / O between the main memory and the cache.

In order to minimize this, a Decomposition Storage Model (DSM) has been proposed. DSM divides records consisting of n columns vertically, divides them into n sub relations, and stores them on disk. A DSM is a method of organizing memory pages by columns. However, when a query is composed of several columns, multiple page accesses are stored, and the performance decreases due to joins between sub-relations.

PAX (Partition Attributes Across) proposed to solve this problem of DSM is stored on disk based on records, but when loaded in memory, memory pages are divided into subpartitions and stored in the same partition for each column to access only specific columns. This method minimizes cache misses between main memory and the cache.

Recently, data management systems specialized for applications have been proposed instead of general-purpose relational models for managing large amounts of data. The key-value data model is a model that supports key-based access in consideration of an application environment that services based on a large amount of data. Therefore, an in-memory page configuration is required.

It is an object of the present invention to provide a method for optimizing the cache by providing a data storage method in a memory page suitable for the key-value data model.

To achieve the above object, the present invention provides a method for storing a page header that stores information for managing a page in memory, and generating a low key page including information about a row key and a storage location of each column. And generating a column page by dividing a key page for storing a cell key and a value page for storing data, and providing a data page in a memory page for cache optimization.

According to another aspect of the present invention, obtaining location information of a row key page from a page header, and obtaining row key data based on the location information, and position of a key page and a value page of a corresponding column based on the row key data. Obtaining information, retrieving a cell key based on location information of the key page, and retrieving corresponding value data based on location information of the value page; Provides a way to retrieve data within a page.

According to the present invention, by dividing data into key-value units and storing the data in a page in the main memory, the cache miss can be significantly reduced to optimize the cache.

Hereinafter, with reference to the accompanying drawings a preferred embodiment according to the present invention will be described in detail.

In the key-value data model, one row is composed of a key and several columns, and each column can store a plurality of data (herein, individual data is called a cell). A cell consists of a set of key-values with data values accessed by row keys and columns. In the simple model of accessing data based on a key, the data may be searched based on a row key or a part of a row of a row, and the data may be accessed based on a cell key of a specific column.

In addition, the key-value data model provides a way to store the column data used together to reduce disk I / O through optimized storage for access patterns. That is, a column group consisting of one or more columns can be defined and stored in disk in units of column groups.

In the above key-value model, memory is configured as follows to optimize cache performance.

The key-value data model stores data on disk in column group units to optimize the cache by loading only the data necessary for loading into main memory.

However, when storing data in column group units, two cases should be considered. First, when a column group consists of one column, a column-based memory page can be configured by storing column data in the page as it is. Secondly, when a column group is composed of two or more columns, it is not efficient when cell key-based access is performed in one column when stored in memory in a column group unit. Therefore, the page should be composed in consideration of cell key-based access, so it is divided into columns and stored in the page of memory. When storing by column, considering the key-based approach, the column page in the page is composed of the key page and the value page and stored to improve the data retrieval performance by the cell key.

1 is a schematic diagram illustrating an embodiment of a web table storing a web page, and FIG. 2 is a schematic diagram illustrating an embodiment of configuring a memory page based on the table of FIG. 1.

As illustrated in FIG. 1, when a contents column and an anchor column exist in a table, and each contents column and anchor column constitute a column group, a page may be configured in a memory as illustrated in FIG. 2.

The page header 100 stores information for managing a page such as a page ID, a start address (pointer) and a size of a subpage. As shown, the page header is four pointers because it consists of four subpages (e.g., low key page 200, contents column page 310, anchor key page 321, anchor value page 322). Save it.

The low key page 200 has a low key 210 and information 220 about the storage location of each column. The storage location information 220 of the column can be displayed by indicating a block that stores the start position of the column existing in the offset table of each column page with a pointer.

The column page 300 is composed of only a value page when the column does not have a key (for example, a contens column 310), and when there is a key, the column page 300 stores a cell key and data, respectively. (E.g. anchor column 320)

3 shows a flowchart of a method for retrieving or updating data in a page according to the present invention.

For example, if you process the following command, this command will retrieve all data in table T with "com.cnn.www" as a low key and "anchor" as a column.

get (T, "com.cnn.www", "anchor")

In order to process the command, first, the address of the low key page is found in the page header information and then moved to the low key page (S301). In the low key page, the low key is searched using the offset table of the page (S302). Since the low key data includes the low key, the address of the block having the pointer of the cell data in the offset table, and the number of key-value data, the cell key data can be retrieved.

Determine a corresponding column based on the row key data, and if a key page of the corresponding column exists, obtain a key page address of the corresponding column, move to the key page (S303), and search for a cell key in the key page. (S304).

As described above, an address of a value page may be obtained based on the row key data, the value page may be moved (S305), and the value data may be searched to search the query data (S306). In this case, the process of obtaining the key page address and the value page address of the corresponding column based on the row key data may occur together before retrieval of the cell key and value data, and may be performed immediately before the retrieval step of the cell key and value data, respectively. It may be.

As mentioned above, although the configuration of the present invention has been described in detail with reference to the preferred embodiments and the accompanying drawings, it is only an example and various modifications and changes are possible within the scope of the technical idea of the present invention.

Therefore, the scope of the present invention should be defined by the following claims.

1 is a schematic diagram illustrating one embodiment of a web table storing a web page;

FIG. 2 is a schematic diagram illustrating an embodiment in which memory pages are configured based on the table of FIG. 1; FIG.

3 is a flow chart of a method for retrieving or updating data in a page according to the present invention.

Claims (8)

In storing the key-value data model in a memory page, Storing a page header for storing information for managing pages in memory;  Generating a low key page containing information about the low key and the storage location of each column; Generating a column page by separating the key page storing the cell key from the value page storing the data Method of storing data in a memory page for cache optimization, comprising a. The method of claim 1, wherein the page header, Including at least one of a page ID, subpage start address, and subpage size A method of storing data in a memory page for cache optimization, characterized by the above-mentioned. The method of claim 1, wherein generating the column page comprises: If a key does not exist in the column, generating a column page using only a value page. Method of storing data in a memory page for cache optimization, comprising a. The method of claim 1, wherein generating the column page comprises: Generating an offset table comprising a block representing the start of a column The data storage method of the memory page for cache optimization further comprising a. The method of claim 4, wherein generating the low key page comprises: Generating a pointer to designate a block indicating the start position of the column in the offset table of each column to display the storage location information of the column Method of storing data in a memory page for cache optimization, comprising a. In retrieving a memory page based on a key-value data model, (a) obtaining location information of the low key page from the page header, and obtaining the low key data based on the location information; (b) obtaining location information of a key page and a value page of a corresponding column based on the row key data; (c) retrieving a cell key based on the location information of the key page; (d) retrieving corresponding value data based on the location information of the value page; The method of retrieving data in a memory page for cache optimization, comprising: a. According to claim 6, wherein step (a), Acquiring the low key data by using the offset table present in the low key page where the position is determined. The method of retrieving data in a memory page for cache optimization, comprising: a. The method of claim 6 or 7, wherein the low key data, At least one of a row key, an address of a block holding a pointer to cell data in an offset table, and a number of key-value data A method of retrieving data in a memory page for cache optimization, characterized by the above-mentioned.

Images