CN111026329A - Key value storage system based on host management tile record disk and data processing method - Google Patents
Key value storage system based on host management tile record disk and data processing method Download PDFInfo
- Publication number
- CN111026329A CN111026329A CN201911125920.5A CN201911125920A CN111026329A CN 111026329 A CN111026329 A CN 111026329A CN 201911125920 A CN201911125920 A CN 201911125920A CN 111026329 A CN111026329 A CN 111026329A
- Authority
- CN
- China
- Prior art keywords
- layer
- data
- value storage
- key value
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a key value storage system based on a host management tile record disk and a data processing method, and belongs to the technical field of information storage. The method comprises the following steps: the LSM tree key value storage module is used for receiving a request of a user, each layer of space of the LSM tree key value storage module is composed of partitions of a plurality of host management tile record disks, and data of each layer are strictly separated; and the space management module is used for connecting the LSM tree key value storage module and the host management tile recording disk and is responsible for space distribution and recovery of the host management tile recording disk. The invention converts random writing into sequential writing, accords with the characteristics of a tile record disk, and each layer of data of the LSM tree key value storage system is separately stored in the disk.
Description
Technical Field
The invention belongs to the technical field of information storage, and particularly relates to a key value storage system and a data processing method based on a host management tile record disk.
Background
With the continuous development of internet technology, the data volume generated by people in the process of using the internet is larger and larger, the data structure is more and more complex, and the total amount of stored information is rapidly increased every year. In terms of storage devices, hard disks have always been dominant because of their large storage capacity, low unit storage cost and mature storage technology, but their storage capacity is limited due to the superparamagnetic effect of magnetic disks. In terms of storage systems, with the advent of the big data age, traditional relational databases are increasingly weak in handling such data. In the face of massive and high-concurrency data, a storage device with large storage capacity and low unit cost is needed, and a storage system with flexible data processing and high performance is also needed.
In recent years, the storage surface density of a magnetic disk can be improved by 2 to 3 times by the appearance of a tile recording magnetic disk, so that the storage capacity of the magnetic disk is increased, and the tile recording magnetic disk has the storage characteristics of large storage capacity and low cost as a novel storage device. The LSM tree key value storage system is used as a non-relational database, is widely applied to key value storage systems, can adapt to various application scenarios, converts random writing into sequential writing, has extremely high performance characteristics, and is an excellent key value storage system. There are three main types of shingle recording disks: a Drive-Managed tile recording disk (Drive-Managed SMR), a Host-Managed tile recording disk (Host-Managed SMR), and a Host-Aware tile recording disk (Host-Aware SMR). The host management tile recording disk is not internally provided with a conversion layer and is only a data storage device, and a user can set a proper garbage collection strategy and read-write operation according to the application requirement of the user, which is a highly recommended use mode in the industry at present. However, such shingled disks are not compatible with existing system software and require management of data storage for use.
However, in the prior art, the random writing performance of the shingle recording disk is poor due to the overlapping of the tracks, how to avoid random writing is one of the hot spots of the shingle recording disk, most of the existing shingle recording disks are divided into a series of continuous partitions with the same size, each of which can only be written sequentially, and the need to migrate the data of the original partition to the new partition upon reclamation, which can be a significant overhead, particularly for the drive management tile recording disks, the garbage collection is carried out in the magnetic disk, so that the great write amplification can be caused, the host perception type tile recording magnetic disk has compatibility and a complex structure, so that extra expenses can be brought, a garbage collection strategy can be set by a user through the host management tile recording magnetic disk, the host management tile recording magnetic disk is a large number of tile recording magnetic disks used in the industry, and the research hotspot is to reduce the garbage collection expenses of the tile recording magnetic disks.
The existing LSM tree key value storage system is based on a file system, and the data of the LSM tree key value storage system is directly stored on a host management tile record disk, so that the data distribution is scattered, the space utilization rate is low, the overhead of garbage recovery is increased, and the key problem is that the key value storage system is correctly stored on the tile record disk.
Disclosure of Invention
Aiming at the defects and improvement requirements of the prior art, the invention provides a key value storage system based on a host management tile record disk and a data processing method, aiming at correctly storing the data of the key value storage system to the tile record disk, ensuring that the tile record disk avoids random writing, simultaneously improving the performance of the key value storage system and correctly combining the two.
To achieve the above object, according to a first aspect of the present invention, there is provided a key-value storage system based on a host management shingle recording disk, the system comprising:
the LSM tree key value storage module is used for receiving a request of a user, each layer of space of the LSM tree key value storage module is composed of partitions of a plurality of host management tile record disks, and data of each layer are strictly separated;
and the space management module is used for connecting the LSM tree key value storage module and the host management tile recording disk and is responsible for space distribution and recovery of the host management tile recording disk.
Specifically, a space is applied from the memory for storing a bitmap, the bitmap is used for recording the use condition of each partition of the disk recorded by the host management tile, a bit of 1 indicates use, and a bit of 0 indicates non-use.
To achieve the above object, according to a second aspect of the present invention, there is provided a data processing method based on the key-value storage system of the first aspect, the method including the steps of:
s1, judging the type of a user request received by an LSM tree key value storage module, if the user request is a write request, a delete request or a modify request, entering a step S2, and if the user request is a read request, entering a step S3;
s2, sequentially storing the key value pairs and the operation type keys of the requests into a cache region of the LSM tree key value storage module, when the cache region exceeds a threshold value, sequentially writing all data of the cache region into a host management tile recording disk, if the space of the layer is enough, directly and sequentially writing the data into the layer, otherwise, when the LSM tree key value storage module applies for a new partition from the space management module, writing the partition into the layer, and returning the requests after the writing is successful;
s3, searching a corresponding value in the cache region according to the requested key, if the corresponding value is found, directly returning the value corresponding to the key to the user, otherwise, searching layer by layer in the multilayer structure of the LSM tree key value storage module, and returning the search result to the user.
Specifically, when applying for a partition, the lookup bit is traversed from the beginning until the first free partition is found, assigned to the LSM tree key-value storage module, and then assigned a value of 1.
Specifically, if the data of the layer exceeds the threshold, merging the first data file of the first partition of the layer with the data file in the next layer, where the first data file intersects with the key of the data file, writing the merged data file into the last partition of the next layer, and directly recovering the merged data when the first partition of the layer has no valid data.
Specifically, when a partition is deleted, its corresponding bit is reassigned to 0.
Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:
(1) aiming at the fact that the layering on a key value storage system in the prior art is only logical layering, the disk space management adopts the idea of zone allocation according to partitions, space allocation and layered storage modes are combined, data of each layer of an LSM tree key value storage system is stored in a plurality of partitions, data of the same layer are distributed in a plurality of partitions of a disk, each partition is only in one layer, the data of each layer are strictly separated, a plurality of data files are sequentially written into each partition, most of data of each layer on a physical address space are together, so that data of each layer are more concentrated in disk storage, physical addresses are layered, disk storage space is saved, and space utilization rate is improved.
(2) When the host management tile records that no effective data exists in the partition of the disk, the bit corresponding to the partition is set to be 0 in the bitmap of the space management module, so that the partition can be allocated for use again, data migration is not needed, and the expense of garbage recovery is reduced.
Drawings
Fig. 1 is a schematic structural diagram of a key value storage system based on a host management tile record disk according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a hierarchical structure of a key value storage system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a disk space management system for a host management tile according to an embodiment of the present invention;
fig. 4 is a flowchart of a data processing method based on the key value storage system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in fig. 1, the present invention provides a key-value storage system based on a host management shingle recording disk, the system comprising:
bottom-host management tile recording disk
The underlying data storage employs existing host management tile recording disks. The host management tile recording disk is divided into a plurality of continuous partitions with the same size, each partition is managed independently, the inside of each partition can only be written in sequence, and the partitions are not interfered. The partition size is fixed for the mainframe management tile recording disk product, typically 256 MB.
Upper-layer LSM tree key value storage module
The LSM (Log-Structured Merge) tree key value storage module is used for receiving a request of a user and managing metadata, wherein the metadata indicates a storage position of data, and each layer of space of the LSM tree key value storage module is allocated according to a partition unit of a host management tile recording disk.
According to the key value storage system (similar to the open source levelDB) based on the LSM tree, when a write request comes, the write request records request information into a log file and writes the request information into a buffer area of a memory, wherein the name of the buffer area is MemTable, the information of the two parts is consistent, so that the content of the write request needs to be written into the log file, and the loss of database information caused by the factors of sudden shutdown of the system and the like can be avoided. After writing the fragmented data to MemTable, when the threshold of MemTable is reached (e.g., 8MB), the whole MemTable is converted to read-only ImmuTable (data is not allowed to be written again), and a new MemTable is applied for storing new data. The read-only ImmuTable is then converted to SSTable (the structure stored to disk in the level DB), and the SSTable can be saved to disk. Typically newly written data is written first to L0And the layer sends a merging request when the data size of a certain layer reaches a threshold value, and pushes down the data to the next layer in a merging mode.
SSTable is a data structure for storing an ordered series of key-value data to a disk in a key value storage system, the SSTable exists in the disk in a file mode, a file with a suffix name of ldb is arranged in the SSTable file, a plurality of key-value data are stored in the SSTable file, and the arrangement sequence of the key-value data is ordered according to the range of keys, so that the data in a single SSTable file is ensured to be ordered. The SSTable structure also contains the information of the size, the key range, the check code and the like of the file, the size of the SSTable file is set in the configuration information max _ file _ size of the database, the SSTable file is set to be 4MB, and the SSTable file is moderate in size and suitable for file operation.
As shown in fig. 2, space allocation of each layer of the key-value storage system is allocated in a zone unit, when a certain layer needs to write data but has insufficient free space, a zone is applied to the host management tile recording disk, data is written in a strict sequential writing mode in the zone, when no valid data exists in one zone, the zone is recycled, and the zone is waited to be allocated next time. Therefore, the data of each layer can be ensured to be concentrated, and the disk space can be effectively utilized. And the data of each layer is divided on the address space according to zone as a unit, when a certain layer of data needs to be read and written, the moving distance of a read-write pointer of a magnetic disk is reduced, and the performance of the system can be accelerated.
The invention relates to a key value storage system, in particular to a logical layering method of a key value storage system.
Middle layer-space management module
The space management module is used for connecting the LSM tree key value storage module and the host management tile recording disk and is responsible for space distribution and recovery of the host management tile recording disk.
As shown in fig. 3, a space is requested from the memory for storing a bitmap, where the bitmap is used to record the usage of each partition of the disk by the host management tile: each partition corresponds to a bit, and if the partition is used, the corresponding bit is 1; if the partition belongs to a free block, the corresponding bit is 0.
Firstly, a user performs data operation through an external interface of the system, a read-write request enters a key value storage interface, then the key value storage system processes the request, when data read-write is needed, the read-write interface enters a disk storage part, and the data of a host management tile recording disk can be read and written through a specific read-write interface.
As shown in fig. 4, the present invention provides a data processing method based on the above key value storage system, which includes the following steps:
step S1, judging the type of the user request received by the LSM tree key value storage module, if the user request is a write request, a delete request or a modify request, entering step S2, and if the user request is a read request, entering step S3.
And S2, sequentially storing the requested key value pairs and the operation type keys into a cache region of the LSM tree key value storage module, when the cache region exceeds a threshold value, sequentially writing all data of the cache region into a host management tile recording disk, if the space of the layer is enough, directly and sequentially writing the data into the layer, otherwise, when the LSM tree key value storage module applies for a new partition from the space management module, writing the partition into the layer, and returning the request after the writing is successful.
The new partition location of the application is determined by the bitmap. When applying for partition, traversing and searching bit from beginning until finding the first free partition, distributing the first free partition to the LSM tree key value storage module, and then assigning the bit as 1. When a partition is deleted, the value of 0 is reassigned to the corresponding bit. The bitmap management can save the memory space, and the bit operation can accelerate the operation speed and improve the performance of the system.
And S3, searching a corresponding value in the cache region according to the requested key, if the corresponding value is found, directly returning the value corresponding to the key to the user, otherwise, searching layer by layer in the multilayer structure of the LSM tree key value storage module, and returning the search result to the user.
The read operation includes the steps of:
(1) when a read request exists in the key value storage system, the key value storage system firstly searches whether the key value pair exists in a memory buffer area, and if the key value pair exists in the memory buffer area, the key value storage system directly returns data;
(2) if not, continuing to search the key value pair in a read-only static buffer area, and if the key value pair is found, returning data;
(3) if not found, the data can only be searched in the hierarchical structure of the key-value storage system, firstly in the file at the L0 level, and if found, the data is returned;
(4) if not, searching the next layer, firstly judging whether the layer has files, and if not, returning to the step of failing to search;
(5) if the layer has files, searching key value pairs in the layer, and if the key value pairs are found, returning data; if not, jump to step (4).
The merging operation is mainly divided into a minor type and a major type, and the key value storage system sorts and compresses existing data through the merging operation, so that some key-value data which are not effective can be deleted, the redundancy of the data is reduced, the number of files stored in a disk is reduced, and the like, and the overall performance of the system is improved.
minor type means writing write request data into MemTable in memory, converting into read-only ImmuTable when MemTable reaches threshold, converting into SSTable file and writing into L of disk0In a layer. For deleted records, the records are not directly deleted in the memory in the minor type, because only deleted key records need to be marked, if the key-value data needs to be completely deleted in the database, the records need to be found out in all data of the system, and great overhead is brought to the system, so that the records are not directly deleted in the minor type, only the key is marked and then written into a file, and the records are really deleted in the major type.
The major type means that when the size of storage data of a certain layer in a key value storage system exceeds a threshold value, SSTable files need to be selected from the layer to be pressed down to the next layer, SSTable files overlapped with the SSTable files in the lower layer need to be selected according to the key range of the SSTable files when the SSTable files are pressed down, then the SSTable files are subjected to multi-path merging and sorting, invalid key-value data are removed in the sorting process, a plurality of SST files formed by the sorted results are written into the lower layer, and balanced distribution of the SSTable files is completed. When the sizes of the stored data of the multiple layers exceed the threshold value of the layer, selecting which layer to perform the merging operation is selected according to the ratio of the total size of the stored data of the current layer to the threshold value of the layer, and preferentially performing the merging operation when the ratio is large.
There are two triggering conditions for a major type of merge operation:
1) the total SSTable file size at a layer in a key-value storage system exceeds a threshold for that layer. In this case, it is stated that the layer does not allow storing more SSTable files, and partial data of the layer needs to be stored to the lower layer through a merge operation, so that the hierarchical structure of the system can be balanced.
2) The number of times of reading certain SSTable files at a certain layer in the key value storage system is too many and exceeds a set value. When a certain SSTable file is read each time, the allowed read value of the file is reduced, and when the value is reduced to a certain degree, the file is triggered to be pressed down to a lower layer.
A merge operation of a key-value store system, comprising the steps of:
and S4, when the data of a certain layer exceeds a threshold value, selecting SSTable in the layer as an object combined to the lower layer and recording the SSTable as compactSSTable, and selecting SSTable overlapped with the key value range of the compactSSTable from the lower layer and recording the SSTable as Overlap SSTable.
Step S4 includes the following steps:
and S41, calculating the total data quantity of SSTable of each layer of the key value storage system/the threshold value of the layer, and recording as Scale.
S42, selecting the largest Scale and the number of layers with the Scale larger than 1, indicating that the layer needs to combine data to a lower layer; otherwise, directly returning without merging operation.
S43, selecting a first SSTable in a first partition at the layer as compact-SSTable;
s44, searching the SSTable overlapped with the key value pair range of the compactSSTable at the lower layer as Overlap SSTable.
And S5, merging the key value data of the compactSSTable and the Overlap SSTable into a new SSTable to be written into the lower layer.
Step S5 includes the following steps:
and S51, reading both the compactSSTable and Overlap SSTable files into a memory, and performing multi-path merging and sequencing on key value pairs of each file.
S52, in the sorting process, if the key value pairs of the same key exist, the new key value pairs and the old key value pairs are compared, the old key value pairs are deleted, and the new key value pairs are requested to be written into the files of the next layer.
S53, if the space of the layer is insufficient, applying for a new partition to add the layer, and marking the partition as 1 in the bitmap to indicate that the partition is used.
S54, a new file is created in the partition, and the key value pairs are written into the new file.
S55, when the size of the file reaches the threshold value, a new file is requested to be applied, and the step S53 is entered.
And S56, until the key value pairs of the compactSSTable and Overlap SSTable files are combined, and writing the new key value pairs into the next layer.
And S6, deleting the SSTable before merging after merging is completed, and recovering the partition if no SSTable in the partition is detected in the deleting process, wherein the partition can be allocated again.
Step S6 includes the following steps:
s61, when the compactSSTable and the Overlap SSTable are combined, deleting one of the files.
S62, after the file is deleted, whether the file still exists in the partition where the file is located is detected.
S63, if the file still exists in the partition, no operation is needed; if there is no file, the value of the partition in the bitmap needs to be set to 0, and the partition is reset, indicating unused.
If the data of the layer exceeds the threshold value, merging the first data file of the first partition of the layer and the data file in the next layer, which has intersection with the key of the data file, writing the merged data file into the last partition of the next layer, and directly recovering the data when the first partition of the layer has no valid data. Compared with the prior art of randomly selecting and merging files, the method and the device for merging the data files merge the first data file of the first partition of the layer with the data file in the next layer, which has intersection with the key of the data file, can recover the partition more quickly, and improve the space utilization rate.
After the files are combined, deleting the files one by one; after deleting a file, detecting whether the partition where the file is located has the file; if the partition still has files, no operation is needed; if there is no file, the value of the partition in the bitmap needs to be set to 0, and the partition is reset, indicating unused.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (6)
1. A key-value storage system based on a host management shingle recording disk, the system comprising:
the LSM tree key value storage module is used for receiving a request of a user, each layer of space of the LSM tree key value storage module is composed of partitions of a plurality of host management tile record disks, and data of each layer are strictly separated;
and the space management module is used for connecting the LSM tree key value storage module and the host management tile recording disk and is responsible for space distribution and recovery of the host management tile recording disk.
2. The system of claim 1, wherein a space is requested from the memory for storing a bitmap, the bitmap is used for recording the use condition of each partition of the disk by the host management tile, a bit of 1 indicates use, and a bit of 0 indicates non-use.
3. A data processing method based on the key-value storage system of any one of claims 1 to 2, characterized in that the method comprises the steps of:
s1, judging the type of a user request received by an LSM tree key value storage module, if the user request is a write request, a delete request or a modify request, entering a step S2, and if the user request is a read request, entering a step S3;
s2, sequentially storing the key value pairs and the operation type keys of the requests into a cache region of the LSM tree key value storage module, when the cache region exceeds a threshold value, sequentially writing all data of the cache region into a host management tile recording disk, if the space of the layer is enough, directly and sequentially writing the data into the layer, otherwise, when the LSM tree key value storage module applies for a new partition from the space management module, writing the partition into the layer, and returning the requests after the writing is successful;
s3, searching a corresponding value in the cache region according to the requested key, if the corresponding value is found, directly returning the value corresponding to the key to the user, otherwise, searching layer by layer in the multilayer structure of the LSM tree key value storage module, and returning the search result to the user.
4. The method of claim 3, wherein upon application for a partition, a lookup bit is traversed from scratch until a first free partition is found, assigned to the LSM tree key-value storage module, and then assigned a value of 1.
5. The method of claim 3, wherein if the data in the layer exceeds the threshold, merging the first data file in the first partition of the layer with the data file in the next layer that intersects the key of the data file, writing the merged data file into the last partition of the next layer, and directly recycling the merged data when the first partition of the layer has no valid data.
6. The method of claim 5, wherein a partition is deleted with its corresponding bit reassigned to 0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911125920.5A CN111026329B (en) | 2019-11-18 | 2019-11-18 | Key value storage system based on host management tile record disk and data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911125920.5A CN111026329B (en) | 2019-11-18 | 2019-11-18 | Key value storage system based on host management tile record disk and data processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026329A true CN111026329A (en) | 2020-04-17 |
| CN111026329B CN111026329B (en) | 2021-04-20 |
Family
ID=70200370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911125920.5A Active CN111026329B (en) | 2019-11-18 | 2019-11-18 | Key value storage system based on host management tile record disk and data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026329B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131140A (en) * | 2020-09-24 | 2020-12-25 | 北京计算机技术及应用研究所 | SSD-based key value separation storage method supporting efficient storage space management |
| CN112395212A (en) * | 2020-11-05 | 2021-02-23 | 华中科技大学 | Method and system for reducing garbage recovery and write amplification of key value separation storage system |
| CN113157600A (en) * | 2021-03-08 | 2021-07-23 | 网宿科技股份有限公司 | Space allocation method of shingled hard disk, file storage system and server |
| CN113495889A (en) * | 2021-07-07 | 2021-10-12 | 浙江大华技术股份有限公司 | Distributed object storage method and device, electronic equipment and storage medium |
| CN113704260A (en) * | 2021-08-25 | 2021-11-26 | 中山大学 | Data storage method and system based on improved LSM tree structure |
| CN114996275A (en) * | 2022-06-22 | 2022-09-02 | 华侨大学 | Key value storage method based on multi-tree conversion mechanism |
| CN115904264A (en) * | 2023-03-13 | 2023-04-04 | 浪潮电子信息产业股份有限公司 | Method, device and medium for recovering garbage data of blocks in storage system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765656A (en) * | 2015-04-23 | 2015-07-08 | 浪潮电子信息产业股份有限公司 | Backup method and device for cold data |
| CN106201353A (en) * | 2016-07-12 | 2016-12-07 | 曹蕊 | A kind of data transfer apparatus for key assignments storage system and method |
| CN108804019A (en) * | 2017-04-27 | 2018-11-13 | 华为技术有限公司 | A kind of date storage method and device |
| CN109521959A (en) * | 2018-11-01 | 2019-03-26 | 西安交通大学 | One kind being based on SSD-SMR disk mixing key assignments memory system data method for organizing |
| CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
| CN109582222A (en) * | 2018-10-31 | 2019-04-05 | 华中科技大学 | A kind of method for cleaning persistently cached in host perception watt recording disc |
| CN109697033A (en) * | 2018-12-19 | 2019-04-30 | 中国人民解放军国防科技大学 | Tile record disk sensing storage caching method and system |
| CN110268394A (en) * | 2017-02-09 | 2019-09-20 | 美光科技公司 | KVS tree |
-
2019
- 2019-11-18 CN CN201911125920.5A patent/CN111026329B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765656A (en) * | 2015-04-23 | 2015-07-08 | 浪潮电子信息产业股份有限公司 | Backup method and device for cold data |
| CN106201353A (en) * | 2016-07-12 | 2016-12-07 | 曹蕊 | A kind of data transfer apparatus for key assignments storage system and method |
| CN110268394A (en) * | 2017-02-09 | 2019-09-20 | 美光科技公司 | KVS tree |
| CN108804019A (en) * | 2017-04-27 | 2018-11-13 | 华为技术有限公司 | A kind of date storage method and device |
| CN109582222A (en) * | 2018-10-31 | 2019-04-05 | 华中科技大学 | A kind of method for cleaning persistently cached in host perception watt recording disc |
| CN109521959A (en) * | 2018-11-01 | 2019-03-26 | 西安交通大学 | One kind being based on SSD-SMR disk mixing key assignments memory system data method for organizing |
| CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
| CN109697033A (en) * | 2018-12-19 | 2019-04-30 | 中国人民解放军国防科技大学 | Tile record disk sensing storage caching method and system |
Non-Patent Citations (2)
| Title |
|---|
| EAGLE992018: "LSM树原理介绍", 《CSDN》 * |
| 林清鑫: "一种轻量级键值存储系统的研究与实现", 《中国硕士学位论文全文数据库 信息科技辑》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131140A (en) * | 2020-09-24 | 2020-12-25 | 北京计算机技术及应用研究所 | SSD-based key value separation storage method supporting efficient storage space management |
| CN112131140B (en) * | 2020-09-24 | 2023-07-14 | 北京计算机技术及应用研究所 | SSD-based key value separation storage method supporting efficient storage space management |
| CN112395212A (en) * | 2020-11-05 | 2021-02-23 | 华中科技大学 | Method and system for reducing garbage recovery and write amplification of key value separation storage system |
| CN113157600A (en) * | 2021-03-08 | 2021-07-23 | 网宿科技股份有限公司 | Space allocation method of shingled hard disk, file storage system and server |
| CN113495889A (en) * | 2021-07-07 | 2021-10-12 | 浙江大华技术股份有限公司 | Distributed object storage method and device, electronic equipment and storage medium |
| CN113495889B (en) * | 2021-07-07 | 2024-03-15 | 浙江大华技术股份有限公司 | Distributed object storage method and device, electronic equipment and storage medium |
| CN113704260A (en) * | 2021-08-25 | 2021-11-26 | 中山大学 | Data storage method and system based on improved LSM tree structure |
| CN113704260B (en) * | 2021-08-25 | 2023-09-29 | 中山大学 | Data storage method and system based on improved LSM tree structure |
| CN114996275A (en) * | 2022-06-22 | 2022-09-02 | 华侨大学 | Key value storage method based on multi-tree conversion mechanism |
| CN114996275B (en) * | 2022-06-22 | 2024-08-16 | 华侨大学 | Key value storage method based on multi-tree conversion mechanism |
| CN115904264A (en) * | 2023-03-13 | 2023-04-04 | 浪潮电子信息产业股份有限公司 | Method, device and medium for recovering garbage data of blocks in storage system |
| CN115904264B (en) * | 2023-03-13 | 2023-06-13 | 浪潮电子信息产业股份有限公司 | Garbage data recovery method, device and medium for blocks in storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026329B (en) | 2021-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111026329B (en) | Key value storage system based on host management tile record disk and data processing method | |
| US10303596B2 (en) | Read-write control method for memory, and corresponding memory and server | |
| CN110825748B (en) | High-performance and easily-expandable key value storage method by utilizing differentiated indexing mechanism | |
| US9584312B2 (en) | Methods and systems for storing and retrieving data | |
| CN104035729B (en) | Block device thin-provisioning method for log mapping | |
| US8161240B2 (en) | Cache management | |
| CN106502587B (en) | Hard disk data management method and hard disk control device | |
| CN106548789A (en) | Method and apparatus for operating stacked tile type magnetic recording equipment | |
| KR101678868B1 (en) | Apparatus for flash address translation apparatus and method thereof | |
| CN109582593B (en) | FTL address mapping reading and writing method based on calculation | |
| WO2017149592A1 (en) | Storage device | |
| CN102646069A (en) | Method for prolonging service life of solid-state disk | |
| CN110968269A (en) | SCM and SSD-based key value storage system and read-write request processing method | |
| CN113626431A (en) | LSM tree-based key value separation storage method and system for delaying garbage recovery | |
| CN104050057B (en) | Historical sensed data duplicate removal fragment eliminating method and system | |
| US20230075437A1 (en) | Techniques for zoned namespace (zns) storage using multiple zones | |
| CN111443874B (en) | Solid-state disk memory cache management method and device based on content awareness and solid-state disk | |
| CN114416646A (en) | Data processing method and device of hierarchical storage system | |
| CN110532228A (en) | A kind of method, system, equipment and the readable storage medium storing program for executing of block chain reading data | |
| CN111984604B (en) | Method for reducing fragments of log-structured file system and flash memory storage system | |
| CN113253932A (en) | Read-write control method and system for distributed storage system | |
| US20240220470A1 (en) | Data storage device and storage control method based on log-structured merge tree | |
| CN114969069A (en) | Heat perception local updating method applied to key value storage system | |
| CN112527197A (en) | Smart card fragment storage space arrangement method, smart card and system | |
| Gong et al. | A write-optimized B-tree layer for NAND flash memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |