KR102275191B1 - Storage System and Method in Windows Operating Systems for the General-Purpose Data Storage - Google Patents

Abstract

The present invention relates to a storage system and method in a Windows operating system for general-purpose data storage, wherein the components of the key-value storage that generate each data item as a key-value pair (k, v) are stored in the Windows registry. WR-Store created by mapping to components; and a control unit that performs hash-based multi-level registry index function, and performs any one function among input (Put), inquiry (Get), or deletion (Delete) for data stored in WR-Store using Windows basic API includes

Description

Storage System and Method in Windows Operating Systems for the General-Purpose Data Storage

The present invention relates to a storage system and method in a Windows operating system for general-purpose data storage, and more particularly, by utilizing a data storage and API supported by the Windows operating system itself to store data without installing additional software. It relates to technology that provides a general-purpose data repository for data storage and management.

With the advent of big data, various types of data such as text, location, graph, and image are stored and managed in data stores, but in general, data types cannot be determined in advance in big data applications.

In addition, in a relational database, data types such as integer, date, and character are defined in the database schema, and only data conforming to the defined data type can be stored. Therefore, it is not possible to store various types of indeterminate data in a relational database.

If you use 'key-value storage' to solve this problem, all data can be expressed in the form of key and value pairs, so data management is easy. A key-value store stores all data in a key-value store, regardless of data type, by mapping the entire data to a value (such as a string or binary format) and assigning a unique key to the value. You can then use the stored data by parsing the values according to the type of data you need.

Representative key-value stores include Redis, Google LevelDB, Facebook RocksDB, Oracle Berkeley DB, and Memcached, and can be classified into persistent storage and memory storage.

Persistent storage provides data type flexibility to relational databases. This means that you can use key-value stores to manage a variety of indeterministic data types instead of a relational database.

Memory storage also provides efficiency by caching data stored in persistent storage. LevelDB, RocksDB, and BerkelyDB are classified as persistent key-value stores, while Redis and Memcache are stored in in-memory key-value stores.

An in-memory key-value store can act as a cache between an application and the underlying system, allowing it to work with other systems to improve performance.

As described above, conventional data storage technologies (persistent storage, in-memory key-value storage) such as Facebook's RocksDB, Google's LevelDB, and Oracle's BerkeleyDB are configured to operate independently on the Windows operating system, and accordingly, a separate installation process It works through

At this time, there is a problem that the size of the software installed in the operating system is large and requires a lot of storage space, and since the installed software operates independently of the operating system, it takes a lot of time for interaction between the software and the operating system.

Accordingly, the present applicant intends to propose a system for configuring a universal data storage for storing and managing data without installing additional software by utilizing the data storage and API supported by the Windows operating system.

Korean Patent Publication No. 10-2009-0075691 (published on Jul. 8, 2009)

An object of the present invention is to map the components of the key-value store to the components of the Windows registry, classify data in the key-value store to enable hash-based multi-level registry indexing, and to set the key through registry settings using APIs. -Light key-value using Windows built-in structure and basic API without installing additional libraries and applications by performing input (Put), inquiry (Get) and delete (Delete) functions for data stored in the value store to configure the repository.

An embodiment of the present invention for achieving this technical problem is a storage system in a Windows operating system for general-purpose data storage, and the configuration of a key-value store that generates each data item as a key-value pair (k, v) WR-Store created by mapping elements to components in the Windows registry; and a control unit that performs hash-based multi-level registry index function, and performs any one function among input (Put), inquiry (Get), or deletion (Delete) for data stored in WR-Store using Windows basic API It is characterized in that it includes.

Preferably, it is characterized in that it further comprises a migration unit that converts data stored in the WR-Store to be recognized in another operating system environment that supports the key-value store.

In the storage method in the Windows operating system for general-purpose data storage according to an embodiment of the present invention based on the above-described system, the control unit maps the components of the key-value store to the components of the Windows registry, and the WR-Store (a) generating a; (b) the control unit reconstructing the key-value data stored in the WR-Store into a hash-based multi-level index structure; and (c) in which the control unit performs any one of input (Put), inquiry (Get), or deletion (Delete) on the data stored in the WR-Store using the Windows basic API. .

According to the present invention as described above, the components of the key-value storage are mapped to the components of the Windows registry, and input (Put) and inquiry (Get) of data stored in the key-value storage through registry settings using APIs Alternatively, by providing WR-Store that performs either function during Delete, it has the effect of configuring a lightweight key-value store using the Windows built-in structure and basic API without installing additional libraries and applications.

And, as the size of the data set increases, the WR-Store according to the present invention is much more efficient than other key-value stores, and it can be applied to other operating systems through migration, so it has great scalability.

WRL에 따른 WR-Store의 성능 변화를 도시한 도면.
도 8은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 연속 작업 k의 수가 1 에서 1 백만으로 변할 때 1D_4L, 2D_2L 및 4D_1L의 처리 시간을 도시한 도면.
도 9는 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 k가 1 에서 100만으로 증가함에 따라 합성 데이터 세트 KVData3의 처리량을 도시한 도면.
도 10은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 데이터 세트의 크기가 10,000 에서 1,000만 으로 증가함에 따른 합성 데이터 세트의 처리량을 도시한 도면.
도 11은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 k가 1에서 1 백만으로 증가함에 따라 실제 데이터 세트의 처리량을 도시한 도면.
도 12는 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템의 다양한 실제 데이터 세트의 처리량을 도시한 도면.
도 13은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 p가 증가함에 따라 성능 변화를 도시한 도면.
도 14는 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템에서 실제 데이터 세트인 ID-HashTag를 저장하는 WR-Store에서 추출 된 데이터의 세부 사항을 도시한 도면.
도 15는 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템의 E-and-T 방법의 알고리즘을 도시한 도면.
도 16은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템의 E-then-T 및 E-and-T의 성능 분석을 도시한 도면.
도 17은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 시스템을 도시한 구성도.
도 18은 본 발명의 실시예에 따른 범용적인 데이터 저장을 위한 윈도우 운영체계에서의 저장 방법을 도시한 순서도.1 is a diagram showing a logical structure provided by an editor of the Windows registry.
2 is a diagram illustrating the concept of a WR-Store of a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
3 is a diagram illustrating a registry path of a hash-based multi-level registry index of WR-Store by a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
4 is a diagram illustrating a WR-Store processing algorithm of a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
5 is a diagram illustrating the performance of WR-Store as WRL is fixed to 1 and WRL is increased in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
6 is a diagram illustrating improved performance of WR-Store or WRL in a state where WRD is fixed to 1 in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
7 is a WRD in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention;

A diagram showing the performance change of WR-Store according to WRL.
8 is a diagram illustrating processing times of 1D_4L, 2D_2L and 4D_1L when the number of consecutive jobs k changes from 1 to 1 million in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention;
9 is a diagram illustrating the throughput of a composite data set KVData3 as k increases from 1 to 1 million in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
10 is a view showing the throughput of a synthetic data set as the size of the data set increases from 10,000 to 10 million in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
11 is a diagram illustrating the throughput of an actual data set as k increases from 1 to 1 million in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention;
12 is a diagram illustrating throughput of various actual data sets of a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention;
13 is a diagram illustrating performance changes as p increases in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention;
14 is a diagram illustrating details of data extracted from a WR-Store that stores an ID-HashTag, which is an actual data set, in a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
15 is a diagram illustrating an algorithm of an E-and-T method of a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
16 is a diagram illustrating performance analysis of E-then-T and E-and-T of a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
17 is a block diagram illustrating a storage system in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.
18 is a flowchart illustrating a storage method in a Windows operating system for general-purpose data storage according to an embodiment of the present invention.

The specific features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Prior to this, the terms or words used in the present specification and claims are based on the principle that the inventor can appropriately define the concept of the term in order to best describe his or her invention in the technical spirit of the present invention. It should be interpreted with the corresponding meaning and concept. In addition, it should be noted that, when it is determined that the detailed description of the well-known functions related to the present invention and its configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof is omitted.

1 is a diagram illustrating a logical structure provided by an editor of a Windows registry. As shown in Fig. 1, the left window indicates registry keys, where the path of the key structured in a tree structure is ""Computer\HKEY_CURRENT_USER\Control Panel\Keyboard"". In this case, each key may have several sub-keys and values, and a parent-child relationship may be established between the keys and sub-keys.

In addition, the right pane displays the value of each key, each of which consists of three properties: name, type, and data, the Name property uniquely identifies the value of the same key, the Type property determines the type of the value, , the Date attribute represents the actual value that follows the specified value type.

Representative value types include REG_BINARY for binary data, REG_DWORD for numbers, and REG_SZ for strings, which means that different types of data can be managed in the registry, including numbers, strings, and binary.

Windows registry types are (1) HKEY_LOCAL_MACHINE (hereinafter referred to as 'HKLM'), (2) HKEY_CLASSES_ROOT (hereinafter referred to as 'HKCR'), (3) HKEY _CURRENT_CONFIG (hereinafter referred to as 'HKCC'), (4) ) HKEY_USERS (hereinafter referred to as 'HKU'), and (5) HKEY_CURRENT_USER (hereinafter referred to as 'HKCU').

First, HKLM maintains system information, HKCR and HKCC are symbolic links to specific subkeys of HKLM, HKU maintains information about all users in the system, HKCU is symbolic to current user information in HKU it's a link For security reasons, we usually store information that requires limited privileges (eg administrators) in HKLM and other information that can be accessed without restrictions (eg logged-in users) in HKCU.

This Windows registry is stored in a special physical location called Hive to continuously store data, and the operating system synchronizes memory data with Hive. [Table 1] is a representative API for manipulating the Windows registry.

[Table 1]

Executables built using the API will work on all versions of Windows as it essentially creates new information in the registry, queries the registry for existing information, and then deletes that information.

Hereinafter, the configuration of the key-value store (hereinafter referred to as 'WR-Store') of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is as follows.

WR-Store is designed by mapping the components of the key-value store to the components of the Windows registry, and the mapping is performed as shown in [Table 2].

As shown in FIG. 2, the WR-Store consists of (1) a key-value data storage area and (2) an index structure area, and the right pane of FIG. 2 shows the key-value data storage of the WR-Store. As shown, several values are stored under the registry key.

As in the example shown in Fig. 2, the registry key displayed in the ""Computer\HKCU\kvs\fe\40\8a\96"" window stores two key-value pairs, and one key-value pair key is " "792479 "" and the associated value is ""#Scotland "". Another key-value pair has the key ""2149 "" and the value ""(-115.223125,36.232915) "".

The left window of FIG. 2 shows the index structure of the WR-Store, and has a structure of indexing keys and subkeys of the Windows registry. As a result, key-value data is stored in the leaf subkey of the registry path.

For example, in FIG. 2, ""fe"", ""40"", ""8a"", and ""96"" are paths to access key-value data of keys ""792479"" and ""2149"" to be. Because a registry key can store multiple values, the name attribute of the registry value identifies the actual key-value pair. At this time, you can set the names of keys and subkeys in the registry path from the given key.

If you create a new database for WR-Store under a specific root registry, you can change the root registry key according to the desired security level of the key-value store, since logged in users can access HKCU but only administrators have access to HKLM. You can choose. Therefore, create a database in either (1) ""Computer\HKLM\DBName"" or (2) ""Computer\HKCU\DBName"" path. At this time, the root key used in Fig. 2 is HKCU and the database name is ""kvs"".

Hereinafter, the hash-based multi-level registry indexing procedure is as follows.

To configure the index structure for WR-Store, you need to determine the names of keys and subkeys in the registry path. In this case, you can use the keys in the key-value store as in the Windows registry in a simple way.

For example, in FIG. 2, ""2149"" becomes a key in the registry, so the registry path of the ""2149"" key becomes ""Computer\HKCU\kvs\2149"". In this case, WR-Store has a one-level index structure. However, creating a scalable index requires a multi-level structure.

To create a multi-level structure, substrings of keys in the key-value store can be used as keys and subkeys in the Windows registry. For example, if you have the key "2149"" split it into two substrings ""21"" and ""49"", where ""21"" becomes the key in the first level registry structure and ""49"" becomes a key in the second level registry structure, consequently the path of the registry key is set to ""Computer\HKCU\kvs\21\49"".

Hereinafter, a procedure for uniformly distributing key-value data to a registry key having a multi-level structure is as follows.

First, a hash function is used to determine the registry path from the specified key in the key-value store. At this time, the hash function has the property of distributing data uniformly regardless of the distribution of the original key, and since multiple values can be stored in the same registry key in the Windows registry, hash collision is not a problem in WR-Store, and the name property is can be used to identify the value of a registry key.

In the embodiment of the present invention, MD5 is used for the hash function and the hash result is expressed in hexadecimal, and the hash result is divided into several substrings to create a multilevel structure, and each substring is divided into a registry for each level of the multilevel structure. used as a key.

3 is a diagram illustrating a registry path of a hash-based multi-level registry index of WR-Store. The registry path for indexing has a length of WRL of each hash value i (1 ≤ I ≤ WRD), and consists of a WRD hash value used for the i-th level index of the Windows registry. At this time, if WRD is defined as the registry depth and WRL is defined as the length of each subkey of the registry path, the total length of the registry path for indexing is WRL_WRD.

In addition, [Table 3] shows an example of the registry path of WR-Store. For example, assuming key ""2149"" and MD5(""2149"") = ""fe40a2892ab382cd"", WRD is 2, WRL is 2, and database name is ""kvs", hash result ""fe The first two bytes of "" become the first level registry key, and the next two bytes of the hash result ""40"" become the second level registry key. As a result, the final path to the registry key is ""Computer \ It is set to HKCU \ kvs \ fe \ 40"", and when WRD and WRL are different, the corresponding registry path is also changed as shown in [Table 3] to create a completely different index structure.

Here, the hash results of the ""214910"" and ""214911"" keys are completely different even though the original keys are similar. Therefore, the key-value data can be distributed uniformly in the Windows registry.

[Table 3]

Hereinafter, the processing algorithm for WR-Store is as follows.

4 shows the processing algorithm of WR-Store, RootKey and DBName are determined when the database is created. Here, RootKey is the root key of the registry and is determined by the security level, and DBName represents the database name and is a subkey of RootKey.

The Get() operation finds a key-value pair (key_in) as an input by passing a key, and returns the value of the found key-value pair (value_out) as a result. First, RegOpenKeyEx() is called by passing the RootKey and DBName as inputs, and the handle of the specified registry key hKey is indexed. After that, it calls RegQueryValueEx() by passing hKey and key_in as inputs, and gets value_out as a result. Here, value_out is the final result of Get() operation, and finally, RegCloseKey() is called to close the allocated resource.

The Put() operation also stores the new key-value pair by passing the key-value pairs key_in and value_in as inputs. First, call RegCreateKey() by passing RootKey and DBName. At this time, if the given DBName is not created as a registry key in the RootKey (that is, when data is inserted for the first time), the registry key of the DBName is created in the RootKey.

On the other hand, it opens the specified registry key (i.e. if the DBName has already been created) and returns a handle to the registry key hKey. It then calls RegSetValueEx() by storing the new key-value pair in the registry with hKey, key_in, REG_SZ, value_in, and sizeo f (value_in) as inputs. At this time, any data type can be saved as a string by using REG_SZ for the value type handling string, and finally, RegCloseKey() is called to close the allocated resource.

And, the Delete () operation deletes key_in as an input by passing the key of the key-value pair. First, call RegOpenKeyEx() passing the RootKey and DBName as input and get a handle to the specified registry key hKey. After that, by passing the RootKey, DBName, and key_in as inputs, RegDeleteKeyValue() is called to delete the key-value pair with the specified key, and finally, RegCloseKey() is called to close the allocated resource.

Hereinafter, the experimental data and environment of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described as follows.

The performance of WR-Store according to an embodiment of the present invention was compared with RocksDB of Facebook, BerkeleyDB of Oracle, and LevelDB of Google, which are representative key-value stores, and Get (), Put (), Delete () operations of WR-Store. The algorithm shown in FIG. 4 was used based on the Windows basic API to implement . In the case of RocksDB, the library for Windows was built according to the instructions provided by RocksDB (https://github.com/facebook/rocksdb/wiki/Building-on-Windows).

We used the source code released on the master branch of Github (https://github.com/facebook/rocksdb), and the source code version used is 5.14. For BerkeleyDB, a library for Windows was built according to the documentation provided by Oracle (https://docs.oracle.com/database/bdb181/index.html). For the source code, the source code of version 18.1 published on the official Oracle website (http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html) was used. For LevelDB, a library for Windows was built using the source code released on Github for the Windows version of LevelDB (https://github.com/ren85/leveldb-windows).

Also, for the sake of fairness, I use C++ for all methods and Visual Studio 15.0 on Windows 10 64-bit. Experiments were performed with different Windows versions in different environments to confirm that WR-Store can run in all Windows environments.

As for hardware specifications, all experiments were performed on a system with a quad-core Intel Xeon E5-2660 2.0 GHz processor, 8 GB of main memory, and Windows 2016 Server 64-bit. Immediately afterward, each experiment was performed.

[Table 4] shows the characteristics of the data set.

Using 8 data sets, including synthetic and real data sets,

Synthetic datasets are used to measure the scalability of a key-value store as the size of the dataset increases. Each key-value pair consists of a random string with a variable length of the value and a unique number for the key. Four data sets of various sizes can be created: (1) KVData1, (2) KVData2, (3) KVData3, and (4) KVData4.

[Table 4]

I tried crawling 4 real data sets on Twitter. To crawl real tweets, we use the Tweepy library (https://github.com/tweepy/tweepy) with different filtering conditions and post-processing for each dataset, all datasets such as geographic location, text and integer It is designed to have different data types.

In addition, [Table 5] shows a sample of the actual data set. (1) ID-Geo data set consists of tweet ID and location information of tweets, (2) ID-hashtag data set consists of tweet ID and hashtags of tweets, and (3) ID-Tweet data set consists of Consists of tweet ID and tweet text, and (4) user-tracker data set consists of user ID and number of user followers.

[Table 5]

Meanwhile, to evaluate the performance of the key-value store, we measure three kinds of operations. (1) read operations, (2) delete operations, and (3) write operations are the primary operations in the key-value store to measure each set of operations (e.g. Get() for read operations, Delete() for delete operations). , in the case of a write operation, put) is performed.

At this time, we change k from 1 to 1 million for the data set to see the change in performance as the continuous operation k increases, where for small (or large) k, the initial cost of the key-value store is large (or small) portion. We also used the Windows API QueryPerf ormanceCounter() to measure the processing time (in microseconds (ms)) and convert it to operations per second (simple OPS) for presentation.

Hereinafter, the size of the executable file of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is as follows.

Since WR-Store uses the Windows basic structure and API, it is basically light. To check this, the size of the executable file of the key-value store was measured, and the results are shown in [Table 6].

Executables in other key-value stores also contain large-sized source code and third-party libraries such as zlib, snappy, and lz4 to support full-featured key-value stores. Conversely, WR-Store uses only the Windows built-in structure and basic API.

As a result, it can be seen that the size of the executable file of WR-store is very small compared to other key-value stores. In particular, WR-Store increases the size of the executable file by 153.73 times compared to RocksDB, 72.69 times compared to BerkeleyDB, and LevelDB. It can be seen that it is 17.77 times smaller than the .

[Table 6]

Hereinafter, an empirical analysis of the WR-Store of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is as follows.

The performance of WR-Store depends on the internals of the Windows registry, and the performance of WR-Store is affected by two parameters: (1) the depth of the registry, WRD, and (2) the length of each subkey in the registry; WRL. Each setting of WR-Store is indicated by xD_yL. where x is the value of WRD and y is forWRL, using KVData2 as data set and measuring OPS (operations per second), while the number of consecutive operations is fixed at 1,000.

On the other hand, FIG. 5 is a view showing the performance of the WR-Store as WRL is fixed to 1 and WRL is increased, and FIG. 6 is a view showing that the performance of the WR-Store or WRL is improved while WRD is fixed to 1. It is a drawing.

WRD는 레지스트리 키 아래의 값 수를 결정하며 WRL 및 WRD 각각보다 WR-Store 성능에 더 중요한 요소인 것으로 결론을 내릴 수 있다. 따라서 WRL

WRD에 가장 효율적인 값을 찾은 다음 WRL

WRD에 대해 결정된 값에 대해 WRL 및 WRD에 적절한 값을 선택해야 한다.In both experiments, when WRL_WRD is 2, 3, and 4, it can be seen that the performance of WR-Store is relatively better than that of other cases. WRL

WRD determines the number of values under the registry key and it can be concluded that it is a more important factor for WR-Store performance than WRL and WRD respectively. So WRL

Find the most efficient value for WRD, then WRL

For the values determined for WRD, appropriate values for WRL and WRD must be chosen.

Hereinafter, the performance adjustment of the WR-Store of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is as follows.

Empirical analysis indicates that WR-Store has the best parameter settings for a given data set.

WRL에 따른 WR-Store의 성능 변화를 도시한 도면으로, 연속 작업 k의 개수가 1에서 1 백만으로 변할 때 WRD

WRL이 2, 4 및 6 인 WR-Store의 처리 시간을 알 수 있으며, 이때 KVData를 데이터 세트로 사용한다.7 is WRD

A diagram showing the performance change of WR-Store according to WRL. WRD when the number of consecutive jobs k changes from 1 to 1 million.

The processing time of the WR-Store with WRLs of 2, 4 and 6 is known, in which case KVData is used as the data set.

WRL이 증가함에 따라 별개의 레지스트리 키 수가 증가하기 때문에 각 키의 값이 적을 수 있다. 따라서 WRD

WRL이 작을 경우 지정된 레지스트리를 찾은 후 많은 값을 처리하는 비용이 발생하여 동일한 키에 대해 많은 수의 값을 처리해야 한다. 반대로, WRD

WRL이 크면 동일한 키에 대해 적은 수의 값 (예 : 각 키에 대해 하나의 값) 만 가지므로 많은 하위 키가 존재하고, 큰 레지스트리 인덱스에 액세스하는 데 비용이 발생하기 때문에 큰 레지스트리 인덱스가 필요하다. 결과적으로 WR-Store의 최고 성능을 나타내는 WRD

WRL에 최적의 값이 포함된다.WRD

As the WRL increases, the number of distinct registry keys increases, so each key may have a smaller value. So WRD

If the WRL is small, the cost of processing a large number of values after finding a given registry is incurred, requiring processing a large number of values for the same key. Conversely, WRD

A large WRL requires a large registry index because it only has a small number of values for the same key (e.g. one value for each key), so many subkeys exist, and there is a cost to accessing the large registry index. . As a result, WRD representing the best performance of WR-Store

The WRL contains the optimal value.

WRL이 변경됨에 따라 WR-Store의 성능 변동이 매우 심하다는 것을 관찰했다. 또한 2D_2L이 가장 안정적이고 효율적인 성능을 확인하였다.Looking at the experimental results, WRD

We observed that the performance of the WR-Store fluctuates very significantly as the WRL changes. In addition, 2D_2L was confirmed to have the most stable and efficient performance.

WRD 인덱싱을 위한 레지스트리 경로의 전체 길이는 4로 고정된다. 이후, 다음 WRL 및 WRD를 변경하여 각각의 최적 조합을 찾는다. 본 발명의 실시예에서는 WRL과 WRD의 세 가지 조합 ((1) 1D_4L, (2) 2D_2L 및 (3) 4D_1L)을 도출하였다.As such, the performance of WR-Store asWRL and WRD varies, and according to previous results, WRL

The total length of the registry path for WRD indexing is fixed at 4. Then, the next WRL and WRD are changed to find an optimal combination of each. In the embodiment of the present invention, three combinations of WRL and WRD ((1) 1D_4L, (2) 2D_2L, and (3) 4D_1L) were derived.

WRL이 변경되는 경우보다 작더라도 WRL 및 WRD가 변경 될 때 의미 있는 성능 차이가 있다.Meanwhile, FIG. 8 is a diagram showing the processing times of 1D_4L, 2D_2L, and 4D_1L when the number of consecutive jobs k changes from 1 to 1 million. KVData was used as a data set, and the overall performance difference was WRD.

There is a significant performance difference when WRL and WRD change, even if they are smaller than when WRL changes.

First, in the case of a read operation, 1D_4L is the most efficient, and when k is 1 million, it can be seen that 1D_4L is 2.21 times and 4D_1L 1.81 times better than 2D_2L.

Also, for write operations, 1D_4L is slightly more efficient than the other settings, and when k is 1 million, 1D_4L outperforms 2D_2L by 1.64 times and 4D_1L by 1.39 times.

And, in the case of deletion, 1D_4L is the most efficient, and when k is 1 million, it can be seen that 1D_4L is 1.55 times better than 2D_2L and 4D_1L is 1.55 times better.

According to the results of the two experiments, it was confirmed that the performance fluctuation of WR-Store was very serious when WRL and WRD were changed (especially in the case of WRD WRL). Therefore, the best parameter setting for WR-Store, namely 1D_4L, was used for the rest of the experiments on the synthetic data set.

In addition, the best parameters are derived by performing performance evaluation on four real data sets. [Table 7a] and [Table 7b] show the throughput measured in the real data set when WRL and WRD are changed.

Here we simply compute the average throughput by assigning equal weights to read, write, and delete operations, resulting in finding the best parameter settings in bold. In addition, for the rest of the experiments on real data sets, the parameter settings best suited for each data set can be used, as shown in [Table 7a] and [Table 7b].

[Table 7a]

[Table 7b]

Hereinafter, the experimental results for the synthetic data set of the storage system in the Windows operating system for general-purpose data storage according to the embodiment of the present invention are as follows.

9 is a diagram showing the throughput of the composite data set KVData3 as k increases from 1 to 1 million.

As the number of consecutive jobs k increases, the throughput results show that WR-Store shows constant performance regardless of k, but RocksDB, BerkeleyDB and LevelDB do not. As a result, WR-Store is particularly efficient for small k, i.e. 1 to 1,000. In particular, it can be seen that WR-Store outperforms RocksDB by 59.259479.95 times for read operations, 1.596182.61 times for write operations, and 2.365978.49 times for delete operations.

Additionally, WR-Store provides 2.0910.24x performance for read operations, 1.3238.72x performance for write operations, and 15.2854.14x performance for delete operations over BerkeleyDB. WR-Store outperforms LevelDB by 25.291091.79 times for read operations, 1.78695.29 times for write operations, and 3.76642.65 times for delete operations. This comes from the fact that other key-value stores have a high initial cost to do their job. As a result, when k is small, the performance is significantly inefficient compared to WR-Store, and the advantage is that no initial cost is required because the Windows registry is always ready.

As the value of k grows, we can observe some cases where other key-value stores are more efficient than WR-Store. Specifically, RocksDB performs 5.03 times for read operations, 14.02 times for write operations, and 9.10 times for delete operations when k is 1 million.

LevelDB performs better than WR-Store even when the value of k is large. In other words, LevelDB is 9.16 times more efficient than WR-Store for write operations and 4.89 times for delete operations when k is 100,000. However, for read operations, WR-Store consistently outperforms LevelDB at all values of k for write and delete operations. At this time, WR-Store surpasses LevelDB again when the value of k increases (ie, one million).

Furthermore, WR-Store is comparable to BerkeleyDB even at large k, which is 44.34% more efficient than BerkeleyDB for delete operations when k is 1 million, BerkeleyDB is 83.09% for read operations and 5.31% for delete operations when k is 1 million. It can be seen that it is more efficient than WR-Store.

It is also important to check the performance of loading additional data into the WR-Store as data is being stored. Data set KVData3 is stored here, and the performance of additional data writes increased from 1 to 1 million in size is measured, and the results show that the performance of WR-Store is very constant even when the size of newly inserted data increases. This means that the WR-Store is stable even under additional data loads.

Meanwhile, FIG. 10 is a diagram illustrating the throughput of the composite data set as the size of the data set increases from 10,000 (ie, KVData1) to 10 million (ie, KVData4).

In the embodiment of the present invention, the number of consecutive operations of the throughput is fixed at 10,000 as the size of the data set increases, and the most important point of this result shows the scalability of the WR-Store. In other words, it can be seen that the performance of WR-Store is fairly constant as the size of the data set increases and the performance of other key-value stores deteriorates significantly.

When measuring the performance degradation when the data set is KVData4 compared to KVData1, the performance of WR-Store is 1.05 times for read operations, 2.30 times for delete operations, and 16.25 times for write operations, respectively.

At this time, RocksDB was 52.11 times, 55.02 times, and 49.63 times, respectively, Berkeley DB was 4365.17 times, 230.23 times, and 1.48 times, respectively, and LevelDB was 239.80 times, 725.28 times and 766.18 times, respectively. You can also see that BerkeleyDB can scale with write operations, but the performance hit is much more severe on read and delete operations.

Hereinafter, the experimental results for the actual data set of the storage system in the Windows operating system for general-purpose data storage according to the embodiment of the present invention are as follows.

11 is a diagram illustrating the throughput of an actual data set as k increases from 1 to 1 million. In this case, ID-Geo was used as the data set for the experiment on the throughput k according to the increase of the continuous operation k.

The overall trend is similar to the results of the synthetic data set shown in Fig. 9 mentioned earlier, where the performance of WR-Store is fairly constant, while other key-value stores require high initial costs. Thus, it can be seen that WR-Store is much more efficient for small k, and RocksDB and LevelDB outperform WR-Store only for some large k.

12 is a diagram showing the throughput of various actual data sets, and the number of consecutive operations is fixed at 10,000 for throughput experiments of various actual data sets.

For read and delete operations, WR-Store outperforms other key-value stores for larger data sets (eg ID-Geo and ID-Tweet). For write operations, BerkeleyDB and LevelDB show fairly consistent performance, but WR-Store still shows similar performance.

Hereinafter, the stress test in the intensive registry workload of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is as follows.

Windows applications always access the registry to store and retrieve data. Since the WR-Store according to the embodiment of the present invention is based on the registry, it is important to observe the performance of the WR-Store in the intensive registry workload.

First, the Process Monitor (https://docs.microsoft.com/en-us/sysinternals/downloads), a tool that analyzes the registry access of other processes while WR-Store is running, and monitors all I/O events in Windows. /procmon) to collect all registry operations that occur in the Windows operating system. (including registry access provided by Microsoft)

Table 8 shows a summary of registry operations collected by Process Monitor, which means that over 3.8 million registry operations are collected in 626.7 seconds. Also, you can see that the Windows process is working with WR-Store and is actively accessing the registry. At this time, the average time interval between registry operations in all processes is 12.96 milliseconds for each process.

Table 9 shows the top 10 processes accessing the registry in some order, with an average time interval between registry operations in the top 10 processes of 1.95 milliseconds.

[Table 8]

[Table 9]

Then, we measured the performance of WR-Store in an intensive registry workload environment. Here we write each synthesizing process so that it can access the registry once every 1.95 milliseconds. This is equal to the average frequency of the top 10 processes accessing the registry.

Then, the number of concurrently running processes p was changed from 1 to 1000. 13 is a diagram illustrating a change in performance as p increases. Here, the actual data set, ID-Geo, was used as the data set and the number of consecutive operations was corrected to 10,000. As a result, it can be seen that the performance of WR-Store is continuously maintained like other key-value stores that do not depend on the registry even when 1000 processes that access the registry intensively are running simultaneously.

Hereinafter, an efficient method for ETL (Extract-Transform-Load) in the WR-Store of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described as follows.

Because WR-Store is based on the Windows registry, it is only portable if a Windows operating system is running. However, in other environments without a Windows operating system, a method for migrating data from WR-Store to another environment is required.

A key-value store processes data in a very simple format consisting of key and value pairs. However, WR-Store stores data in a multi-level structure based on the Windows registry. Therefore, it is necessary to extract data from a multi-level structure and transform it into the form of key-value pairs.

First, data is extracted from the registry path of WR-Store to a file, and then the contents of the file are converted into the form of key-value pairs. (Hereinafter, this key-value conversion is referred to as "E-then-T".) The E-then-T method depends on the built-in extraction function provided by the Windows registry. In other words, if you run the Windows command ""regedit /E"", the data stored in a specific registry path is extracted to a file.

14 is a diagram showing details of data extracted from a WR-Store that stores ID-HashTag, which is an actual data set. It represents a multi-level registry path, where each registry path contains a corresponding key and value pair. You can then specify the key part and the value part by parsing the content from this file based on a pattern matching method such as a regular expression.

The E-then-T method requires two separate phases, and a second phase (i.e. transform) must be performed on the result of the first phase (i.e. extraction). Therefore, it is necessary to design an ETL method that is more efficient than a method that directly transforms the data while extracting the data, which is referred to as extraction and transformation (E-and-T).

The E-and-T method requires direct access to the Windows registry using native APIs, which can be used to navigate the entire structure in the registry path that stores data for the WR-Store. Whenever it finds a key and value during traversal, it converts each key and value directly into the form of a key-value pair.

15 is a diagram illustrating an algorithm of the E-and-T method. First, get information about the specified registry path to know the number of subregistries and the number of key and value pairs. If subkeys exist, we must recursively call this algorithm to examine each subkey. Otherwise, you can get a list of key and value pairs stored in the WR-Store in the form of extracting key and value pairs one by one.

In addition, several experiments were performed to show the performance of the proposed E-and-T method. 16 is a diagram showing the performance analysis of two ETL methods proposed for WR-Store, namely E-then-T and E-and-T. Here, we used four real data sets and corrected the number of consecutive operations to 10,000.

The elapsed time of the E-then-T method consists of (1) the time of the extraction process and (2) the time of the conversion process, measuring each part individually and comparing each part with the elapsed time of the E-and-T method. did. Figure 16 (a) shows the elapsed time of extraction, transformation, E-then-T and E-and-T, it can be seen that E-and-T is much more efficient than E-then-T. In particular, E-and-T performed 1.24 to 3.85 times higher than E-then-T when the data set was changed.

Also, it can be seen that E-and-T requires slightly more time than the export-only process, whereas E-then-T requires a lot of time for the conversion process involving pattern matching. Compared to E-then-T, the performance improvement of E-and-T is larger for large data sets such as ID-GEO and ID-Tweet, and the performance improvement is 3.30 times and 3.85 times, respectively.

In addition, (b) of Figure 16 shows the E-and-T and E-then-T parts of the entire ETL process, starting to extract data stored in the WR-Store and loading it into another key-value store (for example, one of the following). indicates

E-then-T accounts for 11.78% 84.93% of all ETL processes; E-and-T accounts for 3.89% 63.04%. BerkeleyDB and LevelDB's E-and-T account for 11.21% and 3.89% of the overall ETL (extract-transform-load) process, respectively.

Hereinafter, the related work of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described as follows.

Since WR-Store uses the internal structure of the window based on the Windows registry as an index structure, there is a structural difference in that an additional data structure required for the existing key-value store is not required.

In addition, LSM-trees are widely used as index structures for key-value stores (eg LevelDB, RocksDB, HBase, and Cassandra), and there has been a lot of work to improve LSM trees.

bLSM is an LSM tree with the advantages of B-Tree and log structured approaches using Bloom filters to improve index performance. VT-tree extends LSM-tree to efficiently handle sequential and file system workloads, and LSM-Trie has a triple structure of LSM-Tree for more efficient structure compression. In addition, WiscKey is a persistent LSM tree-based key-value store optimized for SSD storage, Flame DB proposes a grouped level structure to reduce compression overhead in the LSM-tree, and adopts an improved bloom filter for false positives of the bloom filter. can reduce

There are also various key-value stores based on hash tables, FAWN-KV uses an in-memory hash table and log structured data store, and provides not only the high performance of flash storage, but also replication and consistency. FlashStore uses a variant of the cuckoo hash table and compresses key signatures to effectively store key-value pairs in flash storage, while SkimpyStash uses a linear chain to index key-value pairs in flash storage using an in-memory hash table. do.

With a very low RAM footprint, SILT combines three primary key-value stores: a log store, a hash store, and an ordered store, while Mercury reduces DRAM access with linked lists and chained hash tables.

There are operations based on other data structures, Masstree is an in-memory structure that combines a B+-tree and an attempt, which provides fast random access and supports range queries. NVMKV is a key-value store with advanced FTL features, while ForestDB uses HB + tree, which combines a disk-based tree structure with a B + tree. Efficient for indexing and retrieving keys of varying lengths, Tucana uses extensions of the B#-tree to provide asymptotic properties for insert operations.

Hereinafter, the function of the WR-Store of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described as follows.

[Table 10] shows some limitations of the Windows registry. The registry key is used for indexing by WR-Store, and 255 characters of the key name and 512 depth of the key are sufficient to create a scalable index.

As the hash result is expressed in hexadecimal, 16255 combinations can be created for one level of registry, and 512 levels of registry can be created. Also, since the size limit of the value name is 16,383, the maximum length of the key in WR-store is 16,383. Keys are typically represented as decimal numbers, which can handle 1016383 keys, which is enough to store large key-value pairs.

[Table 10]

Even though the size limit of the Windows registry value is stated as available memory, it is recommended to store values that are less than 2 MB in size for efficiency. Common modern sources of core value data, such as social networking services and search engines, process data much smaller than 2 MB in size.

For example, Twitter allows 280 bytes (i.e. less than 1 KB) for tweets, Facebook allows 63,206 bytes (i.e. about 61 KB) for posts, and Google allows webpages of 500 KB in maximum size (https:// Crawl to developers.google.com/search/reference/robots_txt).

You can also work with the file system to store large values. In other words, it stores large values as files on the file system and only keeps the file paths in the Windows registry.

In terms of migration, as all versions of Windows support the Windows registry, data stored in WR-Store can be easily migrated to another system only if the Windows operating system is running on the target system.

Using Windows built-in commands, you can easily export data stored in one WR-Store and import it to another WR-Store in another system, and save the exported data. In addition, for other environments that do not support the registry, the aforementioned ETL (Extract-Transform-Load) method can be used to migrate data from WR-Store to another environment that supports existing key-value stores.

In terms of compatibility, the Windows Registry and related APIs can work with any Windows version, so WR-Store can be used on all Windows versions (eg Windows 7/8/8.1/10 and Windows Server 2008/2012/2016). . Executable files compiled in one environment can work in another Windows environment without modification, so compatibility is very good.

In terms of stability, since the Windows registry is always accessed by Windows applications, system processes such as explorer.exe and services.exe always work even if the user does not run the application at all, and use the built-in API provided by Windows. to access the Windows registry shown in [Table 1].

WR-Store according to an embodiment of the present invention was also developed based on the same API, and as shown in FIG. 4, all algorithms for basic operations (eg, Get, Put, and Delete) of the key-value store use the built-in API. It can be seen that based on

As a result, it is desirable to understand WR-Store as one of the general Windows applications that use the Windows registry, and secures safety even in extreme cases where intensive updates to the Windows registry occur at the same time.

Hereinafter, a detailed configuration of the storage system S in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described with reference to FIG. 17 .

Specifically, the storage system (S) in the Windows operating system for general-purpose data storage according to an embodiment of the present invention is a key-value store that generates each data item as a key-value pair (k, v). WR-Store (100), which is created by mapping elements to components of the Windows registry, and hash-based multi-level registry indexing functions, and input for data stored in WR-Store (100) using Windows native API It is configured to include a control unit 200 that performs any one function among (Put), inquiry (Get), and deletion (Delete).

In addition, the data stored in the WR-Store 100 is configured to further include a migration unit 300 that converts the data stored in the key-value store so that it can be recognized in another operating system environment that supports the key-value store. At this time, migration is performed by ETL (Extract-Transform-Load).

Further, a key-value store according to an embodiment of the present invention represents each data item as a key-value pair (k, v), where the key k uniquely identifies the entire data item, and the value v represents the actual data item. It includes description and performs input (Put), inquiry (Get) and delete (Delete) functions.

Here, the Get function takes a key as input and returns the value associated with the provided key from the key-value store, the Put function gets the key and value pair and stores the key-value pair in the key-value store, and the Delete function takes a key as input and performs an operation to delete a data item using the specified key.

Hereinafter, with reference to FIG. 18, a procedure for configuring the key-value storage of the storage system in the Windows operating system for general-purpose data storage according to an embodiment of the present invention will be described as follows.

First, the control unit 200 creates the WR-Store 100 by mapping the components of the key-value store to the components of the window registry (S1702).

Then, the control unit 200 reconstructs the hash-based multi-level index structure so that the key-value data stored in the WR-Store 100 can be distributed in a balanced manner and accessed efficiently (S1704).

Subsequently, the control unit 200 performs any one of input (Put), inquiry (Get), or deletion (Delete) on the data stored in the WR-Store 100 using the Windows basic API (S1706).

Next, the control unit 200 migrates the data stored in the WR-Store 100 to the key-value store of another operating system by applying ETL (Extract-Transform-Load) (S1708). Through this, the scope of application of WR-Store can be expanded not only in the environment where the Windows operating system runs, but also in other environments that support the existing key-value store.

Then, the control unit 200 performs empirical analysis to verify the performance of the WR-Store 100 to adjust the performance (S1710).

According to the storage system in the Windows operating system for general-purpose data storage according to the embodiment of the present invention as described above, there are advantages as follows.

First, WR-Store is a lightweight key-value store because it uses Windows built-in structures and native APIs without installing additional libraries and applications. Experimental results show that WR-Store reduces the size of executable files by 17.77153.73 times compared to other key-value stores.

Second, extensive experimentation with synthetic and real data sets made WR-Store comparable or even more efficient with modern systems (i.e., RocksDB, BerkeleyDB, and LevelDB). In particular, WR-Store outperforms other key-value stores even with a small number of consecutive operations, WR-Store is much more efficient than other key-value stores as the size of the data set grows, and WR-Store is scalable. has been verified. It also shows that the performance of WR-Store is maintained even for intensive registry workloads where 1000 processes accessing the registry are running concurrently.

As such, the advantages of WR-Store are (1) lightness, (2) efficiency, and (3) scalability. The proposed technique can be used in any environment running Windows operating system with minimal effort. In addition, in the case of another environment without the Windows operating system, data can be easily migrated to another environment supporting the existing key-value store by using the ETL (Extract-Transform-Load) method according to the embodiment of the present invention. .

Although described and illustrated in relation to a preferred embodiment for illustrating the technical idea of the present invention above, the present invention is not limited to the configuration and operation as shown and described as such, and deviates from the scope of the technical idea. It will be apparent to those skilled in the art that many changes and modifications can be made to the invention without reference to the invention. Accordingly, all such suitable alterations and modifications and equivalents are to be considered as falling within the scope of the present invention.

S: A storage system in the Windows operating system for general-purpose data storage.
100: WR-Store
200: control unit
300: migration unit

Claims (3)

WR-Store created by mapping the components of the key-value store that creates each data item as a key-value pair (k, v) to the components of the Windows registry; and
A control unit that performs a hash-based multi-level registry index function and performs any one function among input (Put), inquiry (Get), or deletion (Delete) for the data stored in the WR-Store using the Windows basic API including,
The control unit performs empirical analysis of the WR-Store based on the depth WRD of the registry and the length WRL of each subkey of the registry,
A storage system in the Windows operating system for general-purpose data storage, characterized in that it is provided to adjust the performance of the generated WR-Store based on the result of empirical analysis of the received WR-Store. According to claim 1,
A migration unit that converts the data stored in the WR-Store so that it can be recognized in other operating system environments that support key-value storage
A storage system in the Windows operating system for general-purpose data storage, characterized in that it further comprises (a) generating, by the control unit, the WR-Store by mapping the components of the key-value store to the components of the Windows registry;
(b) the control unit reconstructing the key-value data stored in the WR-Store into a hash-based multi-level index structure; and
(c) the control unit performs any one of input (Put), inquiry (Get), or deletion (Delete) on the data stored in the WR-Store using the Windows basic API,
(d) After step (c), the control unit performs empirical analysis of the WR-Store based on the depth WRD of the registry and the length WRL of each subkey of the registry, and
A storage method in the Windows operating system for general-purpose data storage, characterized in that it further comprises the step of adjusting the performance of the WR-Store based on the empirical analysis result for the WR-Store.

Images