CN116762054A - Method for storing data pages in a data storage device using similarity-based data reduction - Google Patents
Method for storing data pages in a data storage device using similarity-based data reduction Download PDFInfo
- Publication number
- CN116762054A CN116762054A CN202180083933.XA CN202180083933A CN116762054A CN 116762054 A CN116762054 A CN 116762054A CN 202180083933 A CN202180083933 A CN 202180083933A CN 116762054 A CN116762054 A CN 116762054A
- Authority
- CN
- China
- Prior art keywords
- page
- data
- pages
- storage device
- received
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 124
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000009467 reduction Effects 0.000 title description 3
- 238000012545 processing Methods 0.000 claims abstract description 13
- 230000006835 compression Effects 0.000 claims description 40
- 238000007906 compression Methods 0.000 claims description 40
- 238000004590 computer program Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 33
- 238000007726 management method Methods 0.000 description 13
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000001965 increasing effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000004065 semiconductor Substances 0.000 description 4
- 235000005282 vitamin D3 Nutrition 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 239000012634 fragment Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000000872 buffer Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 239000011647 vitamin D3 Substances 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 235000019633 pungent taste Nutrition 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
- G06F3/0641—De-duplication techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A method of storing a received data page (202) in a data storage device (102) is provided. The method comprises (i) obtaining a set of samples comprising a set of samples when the received data page is received, wherein the set of samples comprises two or more samples of the received data page, (ii) calculating a new hash value for each of the two or more samples, (iii) identifying one or more page identifiers (302 AA to 302NN, 404AA to 404 NN) associated with one or more pre-calculated hash values in a key value store (300), (iv) sorting the identified page identifiers by the number of times they are identified, (v) determining a similarity between the received data page and one or more pages corresponding to one or more sorting identifiers, wherein the similarity is measured by a plurality of matching data substrings, the substrings being a block or a sequence of bytes in a page, (vi) processing the received data page according to the similarity.
Description
Technical Field
The present invention relates generally to a method of storing received data pages in a data storage device, and more particularly to a control unit for causing a data storage device to store received data pages. The invention further relates to a data storage device comprising a control unit for storing received data pages.
Background
Modern computers use limited memory to hold code and data. Computer memory behaves differently in terms of access speed and (non-) volatility. Currently, volatile memory is faster than nonvolatile memory, and is therefore suitable for encoding and storing data to speed up computation. For example, random access memory (random access memory, RAM) is commonly used as volatile memory. Some use cases, such as internet of things (Internet of Things, ioT), may require variable application data to be placed in non-volatile memory. To increase the execution speed, part of the code and data may also be put into a central processing unit (Central Processing Unit, CPU) code cache (C-cache) and a data cache (D-cache), which are faster but slower than RAM. The CPU cache may be divided into several levels, with different sizes and access speeds. In addition, small amounts of data may be placed and stored in CPU registers that are closest to the CPU execution unit, with the fastest access speed.
In current computer designs, memory access speed is determined by the technical cost, a tradeoff between usage technology and associated costs: register memory is the fastest and most expensive, while non-volatile memory layers are the slowest and least expensive. The registers are statically allocated by the compiler and the CPU can dynamically reallocate registers at the microcode level. The CPU memory controller dynamically manages the allocation of buffers for executing programs.
When the RAM capacity is insufficient to execute the program, the memory manager will attempt to free up memory. With existing memory management systems, non-volatile RAM pages will be deleted from RAM and read later when needed. The immutable RAM pages include program pages in RAM or pages in RAM that are associated with kernel code. A variable RAM page with modified content, with minimum recent access or minimum frequency of use, may be swapped directly from RAM to the swap device. The modified RAM page is marked as a dirty page. ZRAM operates using anonymous pages that are subsets of dirty pages. With the ZRAM design, anonymous pages do not include pages allocated by the file system for files and the like. Anonymous pages may be compressed and placed in ZRAM instead of swapped from ZRAM to a swap device. If anonymous pages are not compressible, they may be placed into ZRAM as is, or written to the switching device as is. If it is a duplicate anonymous page, i.e., a page filled with patterns, it is not compressed. The duplicate pages are represented by a small amount of metadata, which is a reference to an existing page. A page filled with patterns (e.g., 8 bytes repeated over the entire page) is represented by a small number of pattern bytes.
Another existing memory management system implements an adaptive strategy for the compressed cache to detect when the compressed cache must change its memory usage to provide additional benefits and reduce costs. The basis of the existing memory management system is to strictly compress the cache and not allocate redundant memory. Only when the compressed cache is full, compressed, will a new amount of memory be allocated to the compressed cache in order to free up compressed pages to store new pages.
Yet another existing memory management system implements a similarity detection algorithm by implementing a similarity signature for large data blocks and variable length blocks.
Yet another existing memory management system supports sub-page sharing and memory compression of infrequently accessed pages. Yet another existing memory management system employs parameterization techniques/algorithms to identify similar data pages based on 64 byte portions of each page. Specifically, such existing memory management systems hash the contents of (kxs) 64 byte blocks at randomly selected locations on a page, and then group these hashes into k groups of s hashes each. The algorithm uses each group as an index to the hash table. In other words, larger s values capture local similarity, while larger k values include global similarity. Pages that match at least one of the two blocks will be selected as similar pages. In this technique, the number of candidates specifies how many different pages the hash table tracks for each hash signature. For one candidate, existing memory management systems store only the first page found using each hash signature; for a larger number of candidates, the memory management system will reserve multiple pages in the hash table for each hash signature.
Existing memory management systems exchange patched and compressed data pages. However, the swapped pages are not available for reference. The cost of the swap operation is high due to the disk input or output involved.
Yet another existing memory management system maintains a two-dimensional array of signatures for each page as a heatmap. The heat map has S rows and V columns, where V is the total number of possible signature values for the cache sub-block. Each 4 Kilobyte (KB) cache block is divided into 8 sub-blocks of 512 bytes, resulting in 8 sub-signatures representing the contents of the cache block. Unlike many existing content addressable storage systems, each sub-signature is 1 byte, representing the sum of 4 bytes at offsets 0, 16, 32, and 64, respectively, in a sub-block. In this way, the computational overhead is significantly reduced compared to the hash value calculation of the entire sub-block. The column of the heat map includes signature values for sub-pages. Each time a block is read or written, its 8 1-byte sub-signature is retrieved and the 8 hotness values of the corresponding entry in the heatmap are incremented by 1. The spectrum of the content is key to identifying the reference block. Such existing memory management systems use the spectrum of content to identify reference pages. Existing memory management systems cannot identify the same page as compared to the identification of similar pages. In addition, the likelihood of finding similar pages is also low.
Existing data similarity detection techniques/algorithms include fingerprint recognition, string matching, bag of word analysis, citation-based plagiarism detection, genre measurement, etc., which are applied to an entire file (e.g., document) and read the entire source file to calculate corresponding similarity feature values. Thus, these techniques/algorithms require a large amount of CPU cycles and memory space and incur a large amount of disk access. Another existing data similarity detection technique samples blocks of data from both the head and tail of the modulated file to avoid positional shifts caused by modifications.
However, the above prior art does not use similarity detection (except when detecting full copies) and delta compression in dirty anonymous pages to achieve better compression ratios. Furthermore, the above prior art techniques employ techniques such as page deduplication (e.g., ZRAM page deduplication), compression, and pattern detection alone. Furthermore, ZRAMs and memory swap techniques do not use dry run statistics to detect page similarity in ZRAMs, deduplication, delta compression, and reduce writes to swap nonvolatile storage—possibly to increase the life of swap nonvolatile storage.
Accordingly, there is a need to address the above-described deficiencies of the prior art for efficient similarity detection of data pages in a memory system.
Disclosure of Invention
It is an object of the present invention to provide a method of storing a received data page in a data storage device, a control unit for causing a data storage device to store said received data page in said data storage device, and a data storage device comprising a control unit for storing a received data page in a data storage device system, while avoiding one or more of the disadvantages of the prior art.
The above object is achieved by the features of the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.
The present invention provides a method of storing a received data page in a data storage device, a control unit for causing a data storage device to store the received data page in the data storage device, and a data storage device comprising a control unit for storing the received data page in the data storage device system.
According to a first aspect, a method of storing a received data page in a data storage device is provided. The data storage device includes pre-stored data blocks or pages. The data storage device includes a key value store having a plurality of identifiers of pre-stored data blocks or data pages. The identifier associates a plurality of pre-computed hash values computed from samples of the pre-stored data blocks or pages of data stored in the data storage device with one or more blocks or pages, the one or more blocks or pages being a source of retrieval of hash values. The data storage device further includes metadata, wherein the metadata includes data representing a physical address of each pre-stored data block or data page in the data storage device. The method includes, when the received data page is received, obtaining a sample set comprising a first set of samples, wherein the sample set comprises a first set of samples of two or more samples of the received data page. The samples are uniformly spaced, i.e. at a fixed distance from each other. The method includes calculating a new hash value for each of two or more samples. The method includes identifying one or more identifiers associated with one or more pre-computed hash values in the key-value store and matching a new hash value. The method includes ordering the identified identifiers by number of times they are ordered. The method includes determining a degree of similarity between a received page of data and one or more pages corresponding to one or more ordering identifiers, wherein the degree of similarity is measured by a plurality of matching data substrings, the substrings being a sequence of bytes in a block or page. The method includes processing the received data page according to the similarity.
A method of storing received pages of data in a data storage device such as random access memory (random access memory, RAM), storage class memory (storage class memory, SCM) or the like is provided. The data storage device includes pre-stored data blocks or pages. The data storage device includes a control unit, such as a hash table, with key value storage. The key value store uses keys and values as follows. A key is a hash calculated with a certain offset for a selected sample of a page. The value is an identifier of the page, combined with the example offset. The key value store may associate keys of a plurality of samples in pages pre-stored in the data storage device with identifiers of the pages, and each page identifier is combined with a respective sample offset. The data storage device further includes metadata, wherein the metadata maps each page identifier to a physical address of each pre-stored data block or data page in the data storage device. Optionally, the metadata is organized as vectors, b+ trees, etc.
The method includes, upon receipt of a data page, obtaining a sample set of the data page including a first set of samples, wherein the sample set includes a first set of two or more samples of the received data page. The samples are at a fixed distance from each other. The method includes calculating a new hash for each of two or more samples. The method includes determining zero, one, or more of the page identifiers of the pre-stored data pages that match each new hash generated by a lookup in a key-value store (e.g., hash table). All new hashed determined page identifiers calculated for the received pages are ordered by their number of occurrences. For example, if two samples are extracted from a received data page, two hashes are calculated for the two samples. Each hash is used as a key to find a key value store. For example, for the first key, N values or N page identifiers are retrieved from the key value store. For the second key, M page identifiers are retrieved from the key value store. Of the n+m page identifiers retrieved, page identifier a appears x times, page identifier B appears y times, and no other page identifier is found, so n+m= x+y. Then, assuming x < y, this makes it possible to sort the page identifiers a and B deterministically according to the number of occurrences of the page identifiers a and B in the page identifier set retrieved from the key value store. The method includes determining, using a hash value calculated for a sample of a received data page, a similarity measured by a plurality of equal data segments found in the received data page and a pre-stored data page corresponding to a page identifier extracted from a key value store. The data fragment for the similarity measurement consists of a plurality of (S) sequential bytes in the data page, e.g., s= 16 bytes. The method includes processing the received data page according to the similarity. Specifically, if all the data fragments of a received data page are equal to one of the data fragments in the pre-stored data page, then both pages are repeated. If only a portion of the data segments in the received data page and the pre-stored data page are equal, the pages are similar and the received page may be delta compressed relative to the pre-stored data page. If none of the data segments in the received page is equal to any of the data segments in the pre-stored data page, the latter is found by a key-value store lookup, the received data page is considered unique and compressed separately (i.e., independent of any pre-stored data page).
The detection of the similarity of the data pages is followed by delta compression, improving the latency and throughput of the data storage device. The method can perform deduplication or similarity detection on the data page using the same data structure (e.g., key value store) and the same computation (e.g., page sampling and computing hash values of the page samples) to avoid additional dedicated data structures or computations for deduplication or similarity detection. The similarity detection, in combination with delta compression and deduplication, increases the effective capacity of the data storage device due to the higher compression ratio of delta compression compared to the compression ratio of independently compressed data pages. In addition to the above, the method uses the dry run statistics to adjust its parameters for data page similarity detection, deduplication, and delta compression to improve the efficiency of the similarity detection method.
In summary, the method reduces the amount of data written to the non-volatile memory, thereby extending the life of the data storage device.
According to a second aspect, a computer program product for controlling writing of pages to a data storage device is provided. The computer program product comprises computer readable code means which, when a received page is received, causes the data storage system to perform the above method.
According to a third aspect, a control unit for a data storage device system is provided for causing a data storage device to perform the method when a page is received.
In a fourth aspect, a data storage device is provided that includes a control unit. Optionally, the data storage device is a RAM memory, e.g., a ZRAM memory.
Thus, unlike the related art, according to the method of storing the received data page in the data storage device, the control unit causes the data storage device to store the received data page in the data storage device, and performing the similarity detection on the data page after the delta compression improves the latency and throughput of the data storage device. The method determines the similarity between the received data pages and the pre-stored data pages that have been used to efficiently process the received pages and the pre-stored data pages. Based on the similarity, the method may imply delta compression. The similarity detection, in combination with delta compression and deduplication, increases the effective capacity of the data storage device due to the higher compression ratio of delta compression compared to the compression ratio of independently compressed data pages. In addition to the above, the method uses the dry run statistics to adjust its parameters for data page similarity detection, deduplication, and delta compression to improve the efficiency of the similarity detection method. In summary, the method reduces the amount of data written to the non-volatile memory, thereby extending the life of the data storage device.
These and other aspects of the invention are apparent from and will be elucidated with reference to one or more embodiments described hereinafter.
Drawings
Implementations of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
FIG. 1 is a block diagram of a control unit for a data storage device provided by one implementation of the present invention;
FIG. 2 is an exemplary view of sampling a received data page provided by one implementation of the present invention;
FIG. 3 is an exemplary view of a key store as a key store provided by one implementation of the present invention;
FIG. 4 is an exemplary view of determining the similarity of received data pages using key value store provided by one implementation of the present invention;
FIGS. 5A and 5B are flowcharts of a method provided by one implementation of the present invention for storing a received data page in a data storage device that includes pre-stored data blocks or data pages;
FIGS. 6A-6C are flowcharts of a method of processing received data pages provided by one implementation of the present invention;
FIG. 7 is a diagram of a computing device used by one implementation of the invention.
Detailed Description
Embodiments of the present invention provide a method of storing a received data page in a data storage system and a data storage device comprising a control unit for storing a received data page in a data storage device.
In order that those skilled in the art will more readily understand the solution of the present invention, the following implementation of the invention is described in conjunction with the accompanying drawings.
The terms first, second, third and fourth (if any) in the description of the invention, in the claims and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequence or order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the implementations of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to encompass non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to the particular steps or elements recited, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a block diagram illustrating an implementation of the control unit 104 of the data storage device 102. The data storage device 102 includes a key value store (e.g., hash table). The key value store includes a plurality of identifiers of pre-stored data blocks or data pages. The identifier associates a plurality of pre-computed hash values computed from samples of pre-stored data blocks or pages of data stored in the data storage device 102 with one or more blocks or pages that are sources of sample extraction and retrieval of the corresponding hash values. The data storage device 102 also includes metadata, wherein the metadata includes data representing the physical address of each pre-stored data block or data page in the data storage device 102. The control unit 104 is configured to, when a received data page is received, cause the data storage device 102 to obtain a sample set comprising a set of samples, the set of samples comprising two or more samples of the received data page. The samples are at a fixed distance from each other. The data storage device 102 calculates a new hash value for each of the two or more samples. The data storage device 102 identifies one or more identifiers associated with one or more pre-computed hash values in the key-value store. The data storage device 102 sorts the identified identifiers according to the number of times they are identified. The data storage device 102 determines a degree of similarity between a received page of data and one or more pages corresponding to one or more ordering identifiers, wherein the degree of similarity is measured by a plurality of matching data substrings, the substrings being a sequence of bytes in a block or page. The data storage device 102 processes the received data pages according to the similarity.
The data pages sent to the data storage device are delta compressed or swapped out directly in the kernel level after the similarity detection, improving the latency and throughput of the data storage device. The control unit 104 enables the data storage device 102 to perform deduplication or similarity detection on received pages using the same data structure (e.g., key value store) and the same computation (e.g., page sampling and computing hash values of page samples) to avoid additional dedicated data structures or computations for deduplication or similarity detection. The similarity detection, in combination with delta compression and deduplication, increases the effective capacity of the data storage device 102 due to the higher compression ratio of delta compression as compared to the compression ratio of independently compressed pages. In addition to the above, the data storage device 102 uses the dry run statistics to adjust its parameters for page similarity detection, deduplication, and delta compression. In summary, the data storage device 102 reduces the amount of data written to the non-volatile memory, thereby extending the life of the data storage device 102.
Optionally, the step of processing the received data page further comprises (i) if the similarity between the received data page and the one or more pre-stored data pages is above a first threshold, the data storage device 102 selecting the one or more pre-stored data pages having a similarity above the first threshold. The data storage device 102 performs the following steps for the selected one or more pre-stored data pages: (a) Comparing the received data page with the contents of the selected one or more pre-stored data pages; (b) Storing a new entry in metadata of the received page if the received data page and the selected pre-stored data page are the same, (c) delta-compressing the received data page using the selected pre-stored data page as a dictionary if the received data page and the selected pre-stored data page are similar but not identical, writing the compression result on the storage medium and storing the address of the written data page or block in metadata as a block/page identifier, and adding one or more entries to the key value store, the one or more entries associating the received data page with the block/page identifier and an offset of the calculated hash value. Alternatively, if the similarity is below the first threshold, the data storage device 102 compresses the received data pages separately, writes the compressed received data pages into the data storage device 102, stores the addresses of the written pages in metadata, and adds a new page identifier in combination with the sample offset for computing the hash value to the key store.
When the received data page is similar to the selected pre-stored data page, the received data page is compressed, thereby eliminating additional data structures and saving memory of the data storage device 102. When the received data page is similar to the selected pre-stored data page, the received data page is compressed, but the received data page is not swapped out, thereby improving the overall performance of the data storage device 102. Storing the compressed received data pages in the data storage device 102 also improves the overall performance of the data storage device 102.
Optionally, the similarity between the received data block or data page and the particular pre-stored data block or data page is determined as the overall size of the one or more matching substrings, wherein the comparison begins at the offset of the respective samples in the received and pre-stored data pages and extends left and right from the respective offsets. The determination of the similarity omits the additional calculation of the similarity detection. The comparison in the similarity detection may determine one or more parameters. Adjusting one or more parameters may achieve a higher data reduction rate.
Alternatively, the threshold is calculated as a static constant for all pages, or dynamically recalculated for any pair of new and old data blocks or pages by calculating the total size of new pages at offsets where samples are found to be the same matching equal substrings.
Optionally, the selected one or more pre-stored data blocks or data pages are selected from the one or more identified pre-stored data pages based on a plurality of page identifiers obtained from entries of the key-value store when the key-value store is looked up by hash values calculated at respective offsets and samples of the received data pages.
The hash values calculated at the respective offsets and the number of page identifiers obtained from the entries of the key value store can easily apply the similarity detection, thereby reducing the computational complexity.
Alternatively, if the selected pre-stored data page is found to be neither similar nor identical to a particular pre-stored data block or data page, the data storage device 102 updates the key value store with the page identifier calculated for the sample of received data pages at the offset of the fixed distance interval. Each key-value storage entry to be updated is selected using the calculated hash value as an index to the key-value storage, and the selected key-value storage entry is updated using a combination of the received page identifier and the corresponding offset of the calculated hash value. When a selected pre-stored data page is neither similar nor identical to a particular pre-stored data block or data page, the selected pre-stored data page is not referenced by delta compression. Finally, the page identifier of such selected pre-stored data pages is overwritten in the key-value store, thereby increasing the capacity of the data storage device and improving the performance of the similarity detection.
Optionally, the metadata for each page includes a reference count indicating the number of identical blocks/pages and the number of data blocks or pages that were previously delta compressed using the corresponding page as a reference. The reference count indicating the same number of blocks/pages is used for delta compression of similar blocks/pages, thereby eliminating additional data structures and saving memory in the data storage device 102.
Alternatively, if the received page identifier combined with the sample offset should be stored into the key store and the key store is full, the data storage device 102 compares the reference counts for all entries corresponding to the respective hash values, and identifies the minimum reference count, and selects the entry with the minimum reference count, and overwrites the entry of the selected page with the received page identifier combined with the sample offset. The data page whose reference count is equal to 1 (minimum constant, selected by the implementation) is called the unique page.
Alternatively, if more than one page or block is found to have the smallest reference count, the data storage device 102 selects the entry to overwrite in the page or block having the smallest reference count based on the time since the page was written to the storage medium. The entry of the page or block with the smallest reference count that has the least recent storage ("oldest" page or block) is selected for overwriting in the key value store.
FIG. 2 is an exemplary view of sampling a received data page 202 provided by one implementation. When the received data page 202 is received, the control unit is for making available to the data storage device a sample set comprising a first set of samples. For example, the first set of samples may include four samples 204A-204D, each sample being 16 bytes in size. If the first sample 204A of the received data page 202 may start from 0 bytes as an offset, the second sample 204B may start from 1024 bytes, the third sample 204C may start from 2048 bytes, and the fourth sample 204D may start from 3072 bytes. The data storage device calculates new hash values for the four samples 204A through 204D. The new hash value may be used as an index into the key value store. The control unit extracts zero or more page identifiers from the key store for four new hash values. The extracted page identifiers are grouped by equality (i.e., have the same value). The data storage device creates an array for the page identifier and four new hash values. If a group includes at least the smallest page identifier, the corresponding page with that page identifier may be similar to the received page of data. This group is referred to as a similarity candidate group. The control unit may repeatedly sample the received data page 202 a maximum number of times by increasing the offset by 1 to find at least one set of similarity candidates. The incremental sample set for samples 0, 1024, 2048, 3072 may be (1, 1025, 2049, 3073), (2, 1026, 2050, 3074), etc. If a large number of similarity candidate sets are found after sampling, the data storage device orders the similarity candidate sets in descending order according to their size. The results of the ordering are stored in an array. The data storage device places the largest similarity candidate set at the beginning of the array and the smallest similarity candidate set at the end of the array. The largest similarity candidate set may be considered a better similarity candidate set. The number of similarity candidate sets may use parameter constraints. If the parameter is equal to 1, the data storage device considers only one set of similarity candidates with the largest number of page identifiers, and the data pages in the selected set of similarity candidates can be read from flash or ZRAM. The data storage device compares the received data page with the pre-stored page to find a duplicate page. If the received data page 202 is duplicate, the data storage device replaces the reference to another page. Otherwise, the data storage device compresses the received data page with the other page as a dictionary. For example, if two pages of size 4 Kilobytes (KB) are copied one by one into an 8KB buffer, the first page is scanned as a dictionary to collect statistics. The data storage device then compresses the second page using the dictionary collected from the first page. Higher compression ratios are achieved using delta compression. The dictionary may be used for compression even if the page is incompressible without the dictionary, where the dictionary contains data similar to that in the incompressible data.
FIG. 3 is an exemplary view of a key value store 300 provided by one implementation. The key-value store 300 includes one or more page identifiers 302 AA-302 NN of pre-stored data blocks or data pages, which are associated with a plurality of pre-computed hash values. Key store 300 includes memory space and is indexed by page sample hash values. Each memory space includes one or more entries. Each entry includes a page identifier (pageID) 302AA to 302NN. The page identifiers 302AA to 302NN are combined with offsets of Ha Xiye samples. The pageIDs in the same memory space may correspond to different pages. The same pageID may be in different storage spaces.
Optionally, in processing the received data page, if the received data page is similar to but not identical to the selected pre-stored data page, the selected pre-stored data page is delta compressed as a dictionary, the compression result is written on a storage medium, and the address of the written data page or data block is stored in metadata as a block/page identifier. One or more entries are added to the key value store 300 that associate the received data page with the block/page identifier and an offset to calculate the hash value. Alternatively, if the similarity is below the first threshold, the received data page is compressed alone, the compressed received data page is written to the data storage device, the address of the written page is stored in metadata, and the new pageID combined with the sample offset is added to the key store 300 associated with the hash value of the sample extracted from the received data page.
Alternatively, if the selected pre-stored data page is found to be neither similar nor identical to the particular pre-stored data block or data page, the key store 300 with the page identifiers 302 AA-302 NN or identifiers calculated for samples of the received page at fixed distance interval offsets is updated. The key store entry to be updated is selected using the calculated hash value as an index to the key store 300, and the selected key store entry is updated using a combination of the new page identifier and the corresponding offset of the calculated hash value.
When a received data page is neither similar nor identical to a particular pre-stored data block or data page, the received data page is considered unique and compressed alone. Finally, the page identifiers 302 AA-302 NN may be overwritten in the key-value store (e.g., hash table) 300, thereby increasing the data storage device capacity and enhancing the performance of the similarity detection.
FIG. 4 is an exemplary view of determining similarity of received data pages using key-value store provided by one implementation. The exemplary view includes a data storage device 400. The data storage device 400 includes a key value store. The key value store may be a two-dimensional array. The key value store includes a plurality of identifiers 404AA to 404AN, 404CA to 404CN of pre-stored data blocks or pages, wherein the plurality of identifiers 404AA to 404AN, 404CA to 404CN are associated with a plurality of pre-computed hash values. The plurality of pre-computed hash values are computed from samples of pre-stored data blocks or pages of data stored in the data storage device 400 to one or more blocks or pages from which the hash values were obtained. The data storage device 400 also includes metadata, wherein the metadata represents the physical address of each pre-stored data block or data page in the data storage device 400. The exemplary view depicts logical block indirection (Logical Block Indirection, LBI) tables 402A-402E. Each LBI entry is an address of a compressed page location in ZRAM or flash. The page identifiers 404 AA-404 AN identify entries in the LBI table.
Fig. 5A and 5B are flowcharts of a method provided by one implementation of storing a received data page in a data storage device that includes pre-stored data blocks or data pages. The data storage device includes a key value store having a plurality of identifiers of pre-stored data blocks or data pages. The identifier associates a plurality of pre-computed hash values computed from samples of pre-stored blocks or pages of data stored in the data storage device with one or more blocks or pages that are sources of sample extraction and retrieval of the corresponding hash values. The data storage device further comprises metadata, wherein the metadata comprises data representing a physical address of each pre-stored data block or data page in the data storage device. In step 502, when a received data page is received, a sample set is obtained that includes a set of samples that includes two or more samples of the received data page. The samples are at a fixed distance from each other. In step 504, a new hash value is calculated for each of the two or more samples. In step 506, one or more identifiers associated with one or more pre-computed hash values in the key-value store are identified. In step 508, the identified identifiers are ordered by the number of times they are identified. In step 510, a similarity between the received page of data and one or more pages corresponding to one or more ordering identifiers is determined, wherein the similarity is measured by a plurality of matching data substrings, the substrings being a sequence of bytes in a block or page. In step 512, the received page is processed according to the similarity.
The detection of the similarity of the data pages is followed by delta compression, improving the latency and throughput of the data storage device. The method can perform deduplication or similarity detection on the data page using the same data structure (e.g., key value store) and the same computation (e.g., page sampling and computing hash values of the page samples) to avoid additional dedicated data structures or computations for deduplication or similarity detection. The similarity detection, in combination with delta compression and deduplication, increases the effective capacity of the data storage device due to the higher compression ratio of delta compression compared to the compression ratio of independently compressed data pages. In addition to the above, the method uses the dry run statistics to adjust its parameters for data page similarity detection, deduplication, and delta compression to improve the efficiency of the similarity detection method. In summary, the method reduces the amount of data written to the non-volatile memory, thereby extending the life of the data storage device.
Fig. 6A-6C are flowcharts of a method of processing received data pages provided by one implementation. When a received data page is received, the control unit processes the received data page. In step 602, a hash of N samples in a received data page of 4 Kilobytes (KB) is computed. In step 604, if the similarity between the received data page and the one or more pre-stored data pages is above a first threshold, one or more pre-stored data pages having a similarity above the first threshold are selected. In step 606, for the selected one or more pre-stored data pages, the received data pages are compared to the contents of the selected one or more pre-stored data pages. In step 608, if the sample hash of the received data page is the same as the sample hash of the one or more selected pre-stored data pages, the corresponding pre-stored data page is read. In step 610, the received data page is compared to one or more pre-stored data pages to identify similarity. In step 612, if a duplicate is not identified, a similarity comparison is applied. If no similarity is identified in step 614, then in step 616 the unique page is compressed and stored in the data storage device while the key value store is updated using the intermediate sample set. In step 618, if the received data page and the selected pre-stored data page are the same, a new entry in the metadata of the received page is stored, the new entry associating the received page with the selected pre-stored data page. In step 620, if the received data page and the selected pre-stored data page are similar but not identical, delta compression is performed on the received data page using the selected pre-stored data page as a dictionary, the compression result is written on the storage medium and the address of the written data page or block is stored in metadata as a block/page identifier, and one or more entries are added to the key value store, the one or more entries associating the received page identifier with an offset of the calculated hash value.
When the received data page is similar to the selected pre-stored data page, the received data page is compressed, but the received data page is not swapped out, thereby improving the overall performance of the data storage device.
Optionally, the similarity between the received data block or data page and the particular pre-stored data block or data page is determined as the overall size of the one or more matching substrings, wherein the comparison begins at the offset of the respective samples in the received and pre-stored data pages and extends left and right from the respective offsets.
The threshold may be calculated as a static constant for all pages or dynamically recalculated for any pair of new and old data blocks or pages by calculating the total size of new pages at offsets where samples are found to be the same matching equal substrings.
When the key store is looked up by the hash value calculated at the corresponding offset and sample of the received data page, the selected one or more pre-stored data blocks or pages may be selected from the one or more identified pre-stored data pages according to the plurality of page identifiers obtained from the entries of the key store.
Alternatively, if the selected pre-stored data page is found to be neither similar nor identical to a particular pre-stored data block or page, the key value store is updated with page identifiers calculated for samples of the received page at fixed distance intervals, as follows: each key-value storage entry to be updated is selected using the calculated hash value as an index to the key-value storage, and the selected key-value storage entry is updated using a combination of the received page identifier and the corresponding offset of the calculated hash value.
Alternatively, the metadata for each page may include a reference count indicating the number of identical blocks/pages and the number of data blocks or pages that were previously delta compressed using the corresponding page as a reference.
The reference count, which indicates the sum of the same block/page numbers, is used for delta compression of similar blocks/pages, thereby eliminating additional data structures and saving memory in the data storage device.
Optionally, if the received page identifier combined with the sample offset should be stored in the key store and the key store is full, the method further comprises: (i) Comparing reference counts of all entries corresponding to the respective hash values and identifying a minimum reference count, and (ii) selecting the entry with the minimum reference count and overwriting the entry of the selected page with a combination of the received page identifier and the sample offset of the calculated hash value.
Alternatively, if more than one page or block is found to have the smallest reference count, then the entry to be overwritten is selected in the page or block having the smallest reference count based on the time since the page was written to the storage medium.
In one implementation, a computer program product for controlling writing pages into a data storage device. The computer program product comprises computer readable code means which, when a received page is received, causes the data storage system to perform the above method.
Fig. 7 is an illustration of an exemplary computing device 700 in which the various architectures and functions of the various previous implementations may be implemented. As shown, computing device 700 includes at least one processor 704 coupled to bus 702, wherein computing device 700 may be implemented using any suitable protocol, such as the peripheral component interconnect (peripheral component interconnect, PCI) standard, PCI-Express, accelerated graphics port (Accelerated Graphics Port, AGP), hyperTransport, or any other bus or point-to-point communication protocol. Computing device 700 also includes memory 706.
Control logic (software) and data are stored in memory 706, and memory 706 may take the form of random-access memory (RAM). In this description, a single semiconductor platform may refer to the only single semiconductor-based integrated circuit or chip. It should be noted that the term single semiconductor platform may also refer to a multi-chip module with increased connectivity that emulates a module on a chip with increased connectivity that emulates on-chip operation and is a substantial improvement over using conventional central processing unit (central processing unit, CPU) and bus implementations. Of course, the various modules may also be placed individually or in various combinations of semiconductor platforms, depending on the desires of the user.
Computing device 700 may also include secondary memory 710. For example, secondary memory 710 includes hard disk drives and removable storage drives representing floppy disk drives, magnetic tape drives, compact disk drives, digital versatile disk (digital versatile disk, DVD) drives, recording devices, universal serial bus (universal serial bus, USB) flash memory. The removable storage drive reads from and/or writes to a removable storage unit in a well known manner.
Computer programs or computer control logic algorithms may be stored in memory 706 and/or secondary memory 710. When executed, enable the computing device 700 to perform the various functions described above. Memory 706, secondary memory 710, and any other memory are possible examples of computer-readable media.
In one implementation, the architecture and functionality described in the various previous figures may be implemented in the processor 704, a graphics processor coupled to the communication interface 712, an integrated circuit (not shown) capable of having at least a portion of the capabilities of both the processor 704 and the graphics processor, a chipset (i.e., a set of integrated circuits designed to operate and sell as a unit to perform related functions, etc.).
Furthermore, the architecture and functionality described in the various previous figures may be implemented in the context of general purpose computer systems, circuit board systems, game console systems dedicated for entertainment purposes, application specific systems. For example, computing device 700 may take the form of a desktop computer, a laptop computer, a server, a workstation, a gaming machine, an embedded system.
In addition, computing device 700 may take the form of various other devices including, but not limited to, personal digital assistant (personal digital assistant, PDA) devices, mobile telephone devices, smartphones, televisions, and the like. Further, although not shown, the computing device 700 may be coupled to a network (e.g., a telecommunications network, a local area network (local area network, LAN), a wireless network, a wide area network (wide area network, WAN), such as the internet, a peer-to-peer network, a cable network, etc.) for communication purposes via the I/O interface 708.
It should be understood that the arrangement of components shown in the described figures is exemplary and that other arrangements are possible. It should also be appreciated that the various system components (and modules) defined by the claims described below and shown in the various block diagrams represent components in some systems configured in accordance with the subject matter disclosed herein. For example, one or more of these system components (and modules) may be implemented in whole or in part by at least some of the components shown in the arrangements shown in the described figures.
Furthermore, while at least one of these components is at least partially implemented as an electronic hardware component, and thus constitutes a machine, other components may be implemented in software, which when included in an execution environment constitutes a machine, hardware, or a combination of software and hardware.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (15)
1. A method of storing a received data page (202) in a data storage device (102) comprising pre-stored data blocks or data pages, characterized in that,
the data storage device (102) comprises a key value store (300), wherein the key value store (300) has a plurality of identifiers (302 AA to 302NN, 404AA to 404 NN) of pre-stored data blocks or data pages, the identifiers (302 AA to 302NN, 404AA to 404 NN) associating a plurality of pre-computed hash values computed from samples of the pre-stored data blocks or data pages stored in the data storage device (102) with one or more blocks or pages, the one or more blocks or pages being a source of sample extraction and retrieval of respective ones of the pre-computed hash values;
The data storage device (102) further comprises metadata, wherein the metadata comprises data representing a physical address of each pre-stored data block or data page in the data storage device (102), the method comprising the steps of, when the data storage device (102) receives a data page (202):
obtaining a uniformly spaced sample set, wherein the uniformly spaced sample set comprises two or more samples of the received data page (202);
calculating a new hash value for each of the two or more samples;
identifying one or more identifiers (302 AA to 302NN, 404AA to 404 NN) associated with one or more pre-computed hash values in the key-value store (300);
sorting the identified identifiers by the number of times they are identified;
determining a degree of similarity between the received data page (202) and one or more pages corresponding to one or more ordering identifiers, wherein the degree of similarity is measured by a plurality of matching data substrings, the substrings being a sequence of bytes in a block or page;
-processing the received data page (202) according to the similarity.
2. The method of claim 1, wherein the key-value store (300) is a hash table.
3. The method according to claim 1, wherein the step of processing the received data page (202) comprises:
if the similarity between the received data page (202) and the one or more pre-stored data pages is above a first threshold, selecting one or more pre-stored data pages having a similarity above the first threshold, and performing the following steps on the selected one or more pre-stored data pages:
comparing the received data page (202) with the contents of the selected one or more pre-stored data pages;
-if the received data page (202) and the selected pre-stored data page are identical, storing a new entry in the metadata of the received data page (202), the new entry associating the received data page (202) with the selected pre-stored data page;
if the received data page (202) and the selected pre-stored data page are similar but not identical, delta-compressing the received data page (202) using the selected pre-stored data page as a dictionary, writing the compression result on a storage medium and storing the address of the written data page or block in the metadata as a block/page identifier, and adding one or more entries to the key value store (300), the one or more entries associating the received data page (202) with a block/page identifier and an offset to calculate the hash value;
-if the similarity is below the first threshold value, compressing the received data page (202) alone, writing the compressed received data page into the data storage device (102), storing the address of the written page in the metadata, and adding a new page identifier combined with the sample offset of the calculated hash value to the key value store (300).
4. A method according to claim 3, characterized in that the similarity between the received data block or data page (202) and a particular pre-stored data block or data page is determined as the overall size of one or more matching substrings, wherein the comparison is started at the offset of the respective samples in the received and pre-stored data page and is extended to the left and right from the respective offset.
5. A method according to claim 3 or 4, characterized in that the threshold is calculated as a static constant for all pages or dynamically recalculated for any pair of old and new data blocks or data pages by calculating that the total size of the new page matches an equal substring at an offset where samples are found to be the same.
6. The method according to claim 4 or 5, wherein the selected one or more pre-stored data blocks or data pages are selected from the one or more identified pre-stored data pages based on the plurality of page identifiers obtained from the entries of the key-value store (300) when the key-value store (300) is looked up by a hash value calculated with the respective offset and sample of the received data pages (202).
7. A method according to any one of the preceding claims, characterized in that the method comprises:
the selected pre-stored data page is found to be neither similar nor identical to the particular pre-stored data block or data page, the key store (300) is updated with page identifiers calculated for evenly spaced samples of the received data page (202), wherein each key store entry to be updated is selected using the calculated hash value as an index to the key store (300), and the selected key store entry is updated using a combination of the received page identifier and the corresponding offset of the calculated hash value.
8. The method of any of the preceding claims, wherein the metadata of each page includes a reference count indicating the number of identical blocks/pages and the number of data blocks or data pages that have been previously delta compressed using the corresponding page as a reference.
9. The method of claim 8, wherein if the received page identifier combined with a sample offset should be stored into the key value store (300) and the key value store (300) is full, the method further comprises:
Comparing reference counts of all entries corresponding to the respective hash values and identifying a minimum reference count;
the entry with the smallest reference count is selected and the entry for the selected page is overwritten using the received page identifier in combination with the sample offset for calculating the hash value.
10. The method of claim 9, wherein if a plurality of pages or blocks are found to have a minimum reference count, selecting an entry to overwrite in the page or block having the minimum reference count based on a time since the page was written to the storage medium.
11. A computer program product for controlling writing of pages to a data storage device (102), characterized in that it comprises computer program product comprising computer readable code means which, when a received data page (202) is received, cause the data storage device (102) to perform the method according to any of the preceding claims.
12. A control unit (104) for a data storage device, characterized in that the control unit (104) is adapted to cause the data storage device (102) to perform the method according to any one of claims 1 to 11 when a received data page (202) is received.
13. A data storage device (102), characterized by comprising a control unit (104) according to claim 12.
14. The data storage device (102) of claim 13, wherein the data storage device is a random access memory (random access memory, RAM).
15. The data storage device (102) of claim 14, wherein the RAM memory is a ZRAM memory.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| RU2020000740 | 2020-12-22 | ||
| RUPCT/RU2020/000740 | 2020-12-22 | ||
| PCT/RU2021/050215 WO2022139626A1 (en) | 2020-12-22 | 2021-07-14 | Method for storing a data page in a data storage device using similarity based data reduction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116762054A true CN116762054A (en) | 2023-09-15 |
Family
ID=77627490
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202180083933.XA Pending CN116762054A (en) | 2020-12-22 | 2021-07-14 | Method for storing data pages in a data storage device using similarity-based data reduction |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116762054A (en) |
| WO (1) | WO2022139626A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024030041A1 (en) * | 2022-08-02 | 2024-02-08 | Huawei Technologies Co., Ltd. | Method for data compression and electronic device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9413527B2 (en) * | 2009-04-30 | 2016-08-09 | HGST Netherlands B.V. | Optimizing signature computation and sampling for fast adaptive similarity detection based on algorithm-specific performance |
| US20150010143A1 (en) * | 2009-04-30 | 2015-01-08 | HGST Netherlands B.V. | Systems and methods for signature computation in a content locality based cache |
| US9582222B2 (en) * | 2009-04-30 | 2017-02-28 | Western Digital Technologies, Inc. | Pre-cache similarity-based delta compression for use in a data storage system |
| US20170038978A1 (en) * | 2015-08-05 | 2017-02-09 | HGST Netherlands B.V. | Delta Compression Engine for Similarity Based Data Deduplication |
| US10719253B2 (en) * | 2018-10-31 | 2020-07-21 | EMC IP Holding Company LLC | Efficient compression of data in storage systems through offloading computation to storage devices |
| US11093176B2 (en) * | 2019-04-26 | 2021-08-17 | EMC IP Holding Company LLC | FaaS-based global object compression |
-
2021
- 2021-07-14 WO PCT/RU2021/050215 patent/WO2022139626A1/en active Application Filing
- 2021-07-14 CN CN202180083933.XA patent/CN116762054A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022139626A1 (en) | 2022-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6725671B2 (en) | Adaptive Value Range Profiling for Extended System Performance | |
| US10089360B2 (en) | Apparatus and method for single pass entropy detection on data transfer | |
| US9471500B2 (en) | Bucketized multi-index low-memory data structures | |
| KR102034833B1 (en) | Apparatus for Accessing Data Using Internal Parallelism of Flash Storage based on Key-Value and Method thereof | |
| KR100734823B1 (en) | Method and apparatus for morphing memory compressed machines | |
| US10224957B1 (en) | Hash-based data matching enhanced with backward matching for data compression | |
| US9940060B1 (en) | Memory use and eviction in a deduplication storage system | |
| US8886616B2 (en) | Blocklet pattern identification | |
| US11409663B2 (en) | Methods and systems for optimized translation of a virtual address having multiple virtual address portions using multiple translation lookaside buffer (TLB) arrays for variable page sizes | |
| JP6608468B2 (en) | Storage apparatus and control method thereof | |
| AU2015215974B1 (en) | Apparatus and method for inline compression and deduplication | |
| US20200225882A1 (en) | System and method for compaction-less key-value store for improving storage capacity, write amplification, and i/o performance | |
| KR102219948B1 (en) | Method and system of searching for data stored in memory | |
| CN116762054A (en) | Method for storing data pages in a data storage device using similarity-based data reduction | |
| Tulkinbekov et al. | CaseDB: Lightweight key-value store for edge computing environment | |
| US11868615B2 (en) | Compression device and control method | |
| US20230076729A2 (en) | Systems, methods and devices for eliminating duplicates and value redundancy in computer memories | |
| CN112347060B (en) | Data storage method, device and equipment of desktop cloud system and readable storage medium | |
| CN111796757B (en) | Solid state disk cache region management method and device | |
| KR100894845B1 (en) | Method for Address Translation using the Flash Translation Layer Module | |
| CN111723266A (en) | Mass data processing method and device | |
| US11829341B2 (en) | Space-efficient persistent hash table data structure | |
| US20230409222A1 (en) | System and method for indexing a data item in a data storage system | |
| US20240004549A1 (en) | Memory system | |
| Wang | OpenKV: LSM-tree-based key-value store for open-channel SSD |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |