CN106682184B - Lightweight merging method based on log merging tree structure - Google Patents
Lightweight merging method based on log merging tree structure Download PDFInfo
- Publication number
- CN106682184B CN106682184B CN201611246968.8A CN201611246968A CN106682184B CN 106682184 B CN106682184 B CN 106682184B CN 201611246968 A CN201611246968 A CN 201611246968A CN 106682184 B CN106682184 B CN 106682184B
- Authority
- CN
- China
- Prior art keywords
- key value
- dtable
- metadata
- dtables
- merging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a lightweight merging method based on a log merging tree structure, which uses an ordered key value pair table with dynamically changed size in the log merging tree structure, and simultaneously stores metadata of a lower ordered key value pair table in an upper ordered key value pair table with overlapped key value ranges; selecting a corresponding ordered key value pair table as a merging object according to the balance degree of each ordered key value in the log merging tree to the surface layer; merging the metadata in the selected ordered key value pair table, and writing the merged metadata and the corresponding key value pair data into a lower ordered key value pair table with overlapped key value ranges in an additional mode; adjusting the key value range of the ordered key value pair table to enable the ordered key value pair table load at the same level to reach a balanced state; and a segmented dichotomy searching mode is used for quickly and efficiently searching the key value data. The method can reduce the merging operation overhead of the log merging tree, reduce write amplification and improve the performance of the log merging tree structure.
Description
Technical Field
The invention belongs to the technical field of information storage, and particularly relates to a lightweight merging method based on a log merging tree structure.
Background
With the rapid development of information technology and the popularization of the internet, the data volume has a dramatic increase, the data storage scale of the application service and the data access volume are increased, and the availability requirement and the performance requirement on the database storage system are further improved. The traditional relational database has clear requirements on a stored data structure, so that the high concurrent reading and writing requirements of current service application on the database, the requirements on high-efficiency storage and access of mass data and the requirements on high expandability and high availability of the database cannot be met. To address such problems, non-relational databases have evolved.
In recent years, a Log-Structured-Merge-Tree (LSM Tree) is widely used in a NoSQL system, and is a mainstream storage model in a key-value storage system. The log merged tree architecture provides an algorithm of delayed updating and batch writing, random writing is converted into batch writing, and the moving overhead of a disk arm is reduced, so that the writing performance of the database is greatly improved, and the current high-concurrency and high-performance service requirements are better met.
In the prior art, although the log merge tree can provide good write performance, in order to maintain good read performance, it is necessary to perform a merge operation continuously in the background to order data, which also makes the merge overhead of the log merge tree become a performance bottleneck in the key-value storage system based on the log merge tree algorithm. Therefore, in order to reduce the impact of the merge operation of the log merge tree on the performance of the key-value storage system, an optimization method capable of reducing the overhead of the merge operation of the log merge tree is needed.
Disclosure of Invention
Aiming at the defects or the improvement requirements of the prior art, the invention provides a lightweight merging method based on a log merging tree structure, which aims to reduce read-write amplification when the log merging tree is subjected to background Dtable merging; storing metadata in a lower layer overlay data in an upper layer Victim data; balancing the additional data amount of the Dtables by adjusting the key value range of the Dtables; therefore, the expense of the merging operation of the log merging tree is reduced, and the technical problem of reading, writing and amplifying of the log merging tree is solved.
In order to achieve the above object, the present invention provides a lightweight merge method based on a log merge tree structure, which comprises the following steps:
(1) an ordered key value pair table with dynamically changed size and an upper threshold value is used and recorded as a Dtable, wherein the upper threshold value is in a value range of 1MB to 256MB, and the preferred value is 4 MB; the Dtable contains two types of data, including metadata and key-value pair data: the metadata can be divided into two parts, and if the Dtable is in the Li layer of the log merge tree, and one or more Dtables with key value ranges overlapped with the Dtable in the Li +1 layer are marked as overlap Dtables, the metadata of the Dtable will contain the metadata of the Dtable and the metadata of the overlap Dtables;
the key value range of the Dtable of the Li +1 th layer is overlapped with the key value range of one or more Dtables of the Li +1 th layer, and the key value range of the Dtable of the Li +1 th layer is only overlapped with the key value range of one Dtable of the Li +1 th layer; the storage sequence of the metadata and the key value pair data in the Dtable in the table is sequentially the metadata of the overlap Dtables, the key value pair data and the metadata of the Dtable;
(2) selecting corresponding Dtables as merging objects according to the balance degree of each Dtable layer in the log merging tree, and recording the Dtables as victim Dtables;
(3) in the merging process of the victim Dtable and the corresponding overlay Dtables, firstly merging metadata of the victim Dtable stored in the victim Dtable and metadata of the corresponding overlay Dtables, and then writing the merged metadata and corresponding key value pair data into the corresponding overlay Dtables in an additional mode;
further, in the lightweight merging method, load imbalance may exist in the Dtables at the same layer in the merging process, and under the condition of not causing extra overhead, the key value range of each Dtable is adjusted as much as possible, so that the load of the Dtables at the same layer reaches a balanced state.
Furthermore, in the lightweight merging method, data is added without modifying old data in the merging process, so that key value ranges cover the data added several times in the Dtable, key value pairs in the Dtables are not ordered any more, and a segmented binary search method is provided by utilizing the orderliness of single added data in order to improve the data search speed in the Dtable. And searching each metadata section by using a binary search method from back to front according to the adding sequence of the metadata sections until a final result is obtained.
Further, the step (2) comprises the following sub-steps:
(21) calculating the balance of each layer of the log merging tree, wherein the balance of the Li-th layer is equal to the ratio of the size and the number of Dtables of the Li-th layer;
(22) selecting a Dtable layer with the lowest balance degree as a merging layer;
(23) selecting Dtables in one merging layer as the victim Dtables in sequence, and ending the selection of the victim Dtables until all Dtables of the merging layer are selected.
Further, the step (3) comprises the following sub-steps:
(31) reading the victim Dtable selected in the step (2) into an internal memory;
(32) merging the metadata of the victim Dtable into the metadata of each overlap Dtable;
(33) dividing key value pair data in the victim Dtables into corresponding data blocks according to the key value range of the metadata of each overlap Dtables combined in the step (32);
(34) respectively writing the metadata of each overlap Dtables merged in the step (32) and the data blocks divided in the step (33) into each overlap Dtables in an additional writing mode;
(35) and (4) rewriting the metadata of each overlap Dtable merged in the step (32) back to the victim Dtable, and ending the merging operation.
Further, the step (32) comprises the sub-steps of:
(321) merging metadata of the victim Dtable, and deleting data indexes of key value range overlapping parts in the metadata;
(322) dividing the metadata of the victim Dtable merged in the step (321) into corresponding metadata sections according to the key value range of each overlap Dtables, and then respectively adding the metadata sections to the tail of the metadata of the corresponding overlap Dtables to finish the metadata merging operation.
Further, the step (33) comprises the sub-steps of:
(331) merging the key value pair data of the victim Dtable, and deleting invalid key value pair data;
(332) and (2) dividing the key value pair data of the victim Dtable combined in the step (331) into corresponding data blocks according to the key value range of the metadata of each overlap Dtable, and ending the key value pair data division operation.
Further, aiming at the condition of unbalanced load of the Dtable at the same layer, the invention adjusts the load of the Dtable and comprises the following substeps:
(41) the Dtable with unbalanced load can be divided into two types, one type is caused by unreasonable distribution of the key value range of the metadata, the other type is caused by overlarge data amount of the key value, if the key value range of the Dtable is unreasonable, the step (42) is carried out, and if the key value range of the Dtable is overlarge, the step (44) is carried out;
(42) under the condition of not causing key value pair data pair migration, reducing the key value range of the Dtables, and simultaneously expanding the key value range of the adjacent Dtables by equal amount, so that the key value range of each adjusted Dtable is the same as the original key value range;
(43) if the key value range of the adjacent Dtable is not reasonably distributed due to the current adjustment, the step (42) is carried out, otherwise, the adjustment of the key value range of the Dtable is finished;
(44) reading the Dtable into a memory, and sequencing key value data in the Dtable in a key value sequence to generate new metadata;
(45) and equally dividing the sorted Dtable into a plurality of new Dtables according to the capacity, and finishing the load adjustment of the Dtables.
Generally, compared with the prior art, the technical scheme of the invention has the following technical characteristics and beneficial effects:
(1) according to the lightweight merging method based on the log merging tree structure, reading and writing amplification is reduced when the log merging tree is subjected to background Dtable merging, and merging efficiency is improved;
(2) the invention provides a lightweight merging method based on a log merging tree structure, which reduces disk I/O (input/output) when the log merging tree performs lightweight merging by storing metadata in lower overlay data in an upper Victim data;
(3) the invention provides a lightweight merging method based on a log merging tree structure, which provides a zero-overhead load balancing strategy, balances the additional data volume of Dtables by adjusting the key value range of the Dtables, and maintains the balance of the log merging tree.
Drawings
FIG. 1 is a schematic diagram of a logical layout of Dtable data in an embodiment of the present invention;
FIG. 2 is a flow chart of Victim Dtable selection in an embodiment of the present invention;
FIG. 3 is a schematic diagram of a lightweight merge process in an embodiment of the invention;
FIG. 4 is a schematic diagram illustrating key value range adjustment of a Dtable according to an embodiment of the present invention;
fig. 5 is a schematic diagram of splitting the Dtable in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The overall thought of the invention is as follows: the log merge tree structure uses an ordered key value table with dynamically changed size, and the metadata of the lower ordered key value table is stored in the upper ordered key value table with overlapped key value ranges; selecting a corresponding ordered key value pair table as a merging object according to the balance degree of each ordered key value in the log merging tree to the surface layer; merging the metadata in the selected ordered key value pair table, and writing the merged metadata and the corresponding key value pair data into a lower ordered key value pair table with overlapped key value ranges in an additional mode; adjusting the key value range of the ordered key value pair table to enable the ordered key value pair table load at the same level to reach a balanced state; and a segmented dichotomy searching mode is used for quickly and efficiently searching the key value data.
Fig. 1 is a schematic diagram illustrating a Dtable data logic layout in the embodiment of the present invention. The data in the Dtable may be divided into three categories: are respectively
Metadata of the lower overlay data;
a key-value pair data block of the Dtable;
metadata of the Dtable itself.
Wherein, the Overlap Dtables corresponding to the Dtable may have one or more, so the metadata of the Overlap Dtables is equivalent to the set of metadata of one or more Dtables;
the key value pair data blocks of the Dtable comprise data blocks during initialization and additional data blocks in subsequent merging operation;
the metadata of the Dtable itself includes the initialized data block index and the appended data block index in the subsequent merge operation.
The lightweight merging method based on the log merging tree structure provided by the embodiment comprises the following steps:
(1) an ordered key value pair table with dynamic change of size and an upper threshold value of 4MB is used and is marked as Dtable; the Dtable contains two types of data, including metadata and key-value pair data. The metadata can be divided into two parts, and if the Dtable is in the Li layer of the log merge tree, and one or more Dtables with key value ranges overlapped with the Dtable in the Li +1 layer are marked as overlap Dtables, the metadata of the Dtable will contain the metadata of the Dtable and the metadata of the overlap Dtables; the key value range of the Dtable of the Li +1 th layer is overlapped with the key value range of one or more Dtables of the Li +1 th layer, and the key value range of the Dtable of the Li +1 th layer is only overlapped with the key value range of one Dtable of the Li +1 th layer; the storage sequence of the metadata and the key value pair data in the Dtable in the table is sequentially the metadata of the overlaptables, the key value pair data and the metadata of the Dtable;
(2) selecting corresponding Dtables as merging objects according to the balance degree of each Dtable layer in the log merging tree, and recording the Dtables as victim Dtables;
(3) in the merging process of the victim Dtable and the corresponding overlay Dtables, firstly merging metadata of the victim Dtable stored in the victim Dtable and metadata of the corresponding overlay Dtables, and then writing the merged metadata and corresponding key value pair data into the corresponding overlay Dtables in an additional mode;
fig. 2 is a flow chart illustrating selection of a Victim Dtable in the embodiment of the present invention. When the lightweight merge operation selects the Victim Dtable, a merge layer is selected according to the balance degree of each Dtable layer, and the worse the balance degree, the more necessary the Dtable layer is to perform the merge operation.
In an embodiment, step (2) comprises the sub-steps of:
(21) calculating the balance of each layer of the log merging tree, wherein the balance of the Li-th layer is equal to the ratio of the size and the number of Dtables of the Li-th layer;
(22) selecting a Dtable layer with the lowest balance degree as a merging layer;
(23) selecting Dtables in one merging layer as the victim Dtables in sequence, and ending the selection of the victim Dtables until all Dtables of the merging layer are selected.
Fig. 3 is a schematic diagram illustrating a lightweight merging method according to an embodiment of the present invention. In FIG. 3, the Dtable with key value ranges a-c in L1 is selected as the victim Dtable for merge operation; in the L2 layer, there are three key value ranges and a victim Dtable that overlap each other, the key value ranges are a, b, and c, respectively, to form an overlap Dtables set corresponding to the victim Dtable. The left and right sides of fig. 3 show the states of the Dtable layers before and after lightweight merging.
In an embodiment, step (3) comprises the following sub-steps:
(31) reading the victim Dtable selected in the step (2) into an internal memory;
(32) merging the metadata of the victim Dtable into the metadata of each overlap Dtable;
(33) dividing key value pair data in the victim Dtables into corresponding data blocks according to the key value range of the metadata of each overlap Dtables combined in the step (32);
(34) respectively writing the metadata of each overlap Dtables merged in the step (32) and the data blocks divided in the step (33) into each overlap Dtables in an additional writing mode;
(35) and (4) rewriting the metadata of each overlap Dtable merged in the step (32) back to the victim Dtable, and ending the merging operation.
Wherein, the step (32) further comprises the following substeps:
(321) merging metadata of the victim Dtable, and deleting data indexes of key value range overlapping parts in the metadata;
(322) dividing the metadata of the victim Dtable merged in the step (321) into corresponding metadata sections according to the key value range of each overlap Dtables, and then respectively adding the metadata sections to the tail of the metadata of the corresponding overlap Dtables to finish the metadata merging operation.
Wherein, the step (33) further comprises the following substeps:
(331) merging the key value pair data of the victim Dtable, and deleting invalid key value pair data;
(332) and (2) dividing the key value pair data of the victim Dtable combined in the step (331) into corresponding data blocks according to the key value range of the metadata of each overlap Dtable, and ending the key value pair data division operation.
In the embodiment, in the lightweight merging method, in the merging process, there may be load imbalance in the Dtables at the same level, so that the method further includes a step (4) of adjusting the key value range of each Dtables as much as possible without causing additional overhead, so as to make the load of the Dtables at the same level reach a balanced state.
Fig. 4 is a schematic diagram illustrating key value range adjustment of a Dtable in the embodiment of the present invention. In order to reduce the extra overhead caused by adjusting the key value range, the key value range adjustment of the Dtable is generally performed after the lightweight merge operation, and at this time, the Dtable does not contain relevant key value pair data, and the key value range adjustment does not cause the migration of the key value pair data. In fig. 4, the Li layer Dtable1 is distributed unevenly in the key value ranges before merging, so the key value ranges of Dtable1 and Dtable2 are adjusted after the merging operation, so that the key value ranges of Dtable1 and Dtable2 are adjusted from a-c and d before merging to a-b and c-d.
Fig. 5 is a schematic diagram illustrating splitting of a Dtable in the embodiment of the present invention. In fig. 5, the Dtable size of the Li layer key value range in e-f reaches the upper limit threshold 4MB, and unilateral key value range adjustment cannot be performed, so that two dtables in the layer need to be split equally into the key value range and the key value pair data in the Dtable, such as the two dtables with the key value ranges of e and f in the Li layer split in fig. 5.
In an embodiment, step (4) comprises the following sub-steps:
(41) the Dtable with unbalanced load can be divided into two types, one type is caused by unreasonable distribution of the key value range of the metadata, the other type is caused by overlarge data amount of the key value, if the key value range of the Dtable is unreasonable, the step (42) is carried out, and if the key value range of the Dtable is overlarge, the step (44) is carried out;
(42) under the condition of not causing key value pair data pair migration, reducing the key value range of the Dtables, and simultaneously expanding the key value range of the adjacent Dtables by equal amount, so that the key value range of each adjusted Dtable is the same as the original key value range;
(43) if the key value range of the adjacent Dtable is not reasonably distributed due to the current adjustment, the step (42) is carried out, otherwise, the adjustment of the key value range of the Dtable is finished;
(44) reading the Dtable into a memory, and sequencing key value data in the Dtable in a key value sequence to generate new metadata;
(45) and equally dividing the sorted Dtable into a plurality of new Dtables according to the capacity, and finishing the load adjustment of the Dtables.
In the embodiment, in the lightweight merging method, data is added without modifying old data in the merging process, so key range coverage exists between data added several times in a Dtable, key value pairs in the Dtables are not ordered any more, and a segmented binary search method is provided by utilizing the orderliness of single added data in order to improve the data search speed in the Dtable. And searching each metadata section by using a binary search method from back to front according to the adding sequence of the metadata sections until a final result is obtained.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A lightweight merging method based on a log merging tree structure is characterized by comprising the following steps:
(1) an ordered key value pair table with dynamically changed size and an upper threshold is used and is marked as a Dtable; the Dtable includes metadata and key-value pair data: the metadata comprises metadata of a Dtable and metadata of overlay Dtables, wherein if the Dtable is in the Li layer of the log merging tree, one or more Dtables with key value ranges overlapped with the Dtable in the Li +1 layer of the log merging tree are marked as overlay Dtables;
(2) selecting corresponding Dtables as merging objects according to the balance degree of each Dtable layer in the log merging tree, and recording the Dtables as victim Dtables;
(3) merging the victim Dtable and the corresponding overlay Dtables, merging the metadata of the victim Dtable and the metadata of the corresponding overlay Dtables which are stored in the victim Dtable, and writing the merged metadata and the corresponding key value pair data into the corresponding overlay Dtables in an additional mode;
the step (2) comprises the following substeps:
(21) calculating the balance of each Dtable layer of the log merge tree, wherein the balance of the Li-th layer is calculated according to the ratio of the size and the number of each Dtable of the Li-th layer;
(22) selecting a Dtable layer with the lowest balance degree as a merging layer;
(23) selecting Dtables in one merging layer as the victim Dtables in sequence, and ending the selection of the victim Dtables until all Dtables of the merging layer are selected.
2. The log merge tree structure-based lightweight merge method according to claim 1, wherein in the merge process, there may be load imbalance in the same layer of Dtables, and under a condition that no additional overhead is caused, the key value range of each Dtables is adjusted to make the same layer of Dtables load reach a balanced state.
3. The log merge tree structure-based lightweight merge method according to claim 1, wherein in the merge process, the lightweight merge method adds data without modifying old data, and searches each metadata segment by using a binary search method from back to front according to the addition sequence of the metadata segments until key value pair data is found or not found.
4. A lightweight merging method based on a log merging tree structure according to claim 1 or 2, wherein said step (3) comprises the following sub-steps:
(31) reading the merge object victim Dtable into a memory;
(32) merging the metadata of the victim Dtable into the metadata of each overlap Dtable;
(33) dividing key value pair data in the victim Dtables into corresponding data blocks according to the key value range of the metadata of each overlapped Dtable after combination;
(34) respectively writing the combined metadata of each overlap Dtables and the divided data blocks into each overlap Dtables in an additional writing mode;
(35) and rewriting the metadata of each overlapped Dtable after merging back to the victim Dtable, and ending the merging operation.
5. A log merge tree structure-based lightweight merging method according to claim 4, wherein the sub-step (32) comprises the sub-steps of:
(321) merging metadata of the victim Dtable, and deleting data indexes of key value range overlapping parts in the metadata;
(322) dividing the combined metadata of the victim Dtable into corresponding metadata sections according to the key value range of each overlap Dtables, respectively adding the metadata sections to the tail of the metadata of the corresponding overlap Dtables, and ending the metadata combination operation.
6. A log merge tree structure-based lightweight merging method according to claim 4, wherein the sub-step (33) comprises the sub-steps of:
(331) merging the key value pair data of the victim Dtable, and deleting invalid key value pair data;
(332) dividing the key value pair data of the merged victim Dtable into corresponding data blocks according to the key value range of the metadata of each overlap Dtable, and finishing the key value pair data division operation.
7. The log merge tree structure-based lightweight merge method according to claim 2, wherein the load adjustment of the Dtable comprises the following sub-steps:
(41) judging the reason of the Dtable load imbalance, if the reason is caused by unreasonable distribution of the metadata key value range, entering a step (42); if the key value pair data size is too large, entering the step (44);
(42) under the condition of not causing key value pair data pair migration, reducing the key value range of the Dtables, and simultaneously expanding the key value range of the adjacent Dtables by equal amount, so that the key value range of each adjusted Dtable is the same as the original key value range;
(43) if the key value range of the adjacent Dtable is not reasonably distributed due to the current adjustment, the step (42) is carried out; otherwise, ending the load adjustment of the Dtable;
(44) reading the Dtable into a memory, and sequencing key value data in the Dtable in a key value sequence to generate new metadata;
(45) and equally dividing the sorted Dtable into a plurality of new Dtables according to the capacity, and finishing the load adjustment of the Dtables.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611246968.8A CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611246968.8A CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106682184A CN106682184A (en) | 2017-05-17 |
| CN106682184B true CN106682184B (en) | 2019-12-20 |
Family
ID=58872790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611246968.8A Active CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106682184B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558084B (en) * | 2018-11-29 | 2021-08-17 | 文华学院 | Data processing method and related equipment |
| US10983975B2 (en) | 2019-06-13 | 2021-04-20 | Ant Financial (Hang Zhou) Network Technology Co., Ltd. | Data block storage method and apparatus, and electronic device |
| CN110377227B (en) * | 2019-06-13 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data block storage method and device and electronic equipment |
| CN113688096A (en) * | 2021-07-15 | 2021-11-23 | 三星（中国）半导体有限公司 | Storage method, storage device and storage system |
| CN116595015B (en) * | 2023-07-18 | 2023-12-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN105045891A (en) * | 2015-07-31 | 2015-11-11 | 中国科学院计算技术研究所 | Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus |
| CN105159915A (en) * | 2015-07-16 | 2015-12-16 | 中国科学院计算技术研究所 | Dynamically adaptive LSM (Log-structured merge) tree combination method and system |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160179865A1 (en) * | 2014-12-17 | 2016-06-23 | Yahoo! Inc. | Method and system for concurrency control in log-structured merge data stores |
-
2016
- 2016-12-29 CN CN201611246968.8A patent/CN106682184B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
| CN105159915A (en) * | 2015-07-16 | 2015-12-16 | 中国科学院计算技术研究所 | Dynamically adaptive LSM (Log-structured merge) tree combination method and system |
| CN105045891A (en) * | 2015-07-31 | 2015-11-11 | 中国科学院计算技术研究所 | Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106682184A (en) | 2017-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106682184B (en) | Lightweight merging method based on log merging tree structure | |
| CN105117415B (en) | A kind of SSD data-updating methods of optimization | |
| CN110083601B (en) | Key value storage system-oriented index tree construction method and system | |
| CN110347852B (en) | File system embedded with transverse expansion key value storage system and file management method | |
| CN109683811B (en) | Request processing method for hybrid memory key value pair storage system | |
| CN108600321A (en) | A kind of diagram data storage method and system based on distributed memory cloud | |
| CN105117417A (en) | Read-optimized memory database Trie tree index method | |
| CN109582593B (en) | FTL address mapping reading and writing method based on calculation | |
| CN108897761A (en) | A kind of clustering storage method and device | |
| CN102129458A (en) | Method and device for storing relational database | |
| CN101751406A (en) | Method and device for realizing column storage based relational database | |
| CN109376156A (en) | Read the method with the hybrid index of storage perception | |
| JP7087825B2 (en) | Learning device and learning method | |
| CN104361113A (en) | OLAP (On-Line Analytical Processing) query optimization method in memory and flesh memory hybrid storage mode | |
| CN112015741A (en) | Method and device for storing massive data in different databases and tables | |
| US6745198B1 (en) | Parallel spatial join index | |
| JP2019212243A (en) | Learning identification device and learning identification method | |
| CN104158744A (en) | Method for building table and searching for network processor | |
| CN107426315B (en) | Distributed cache system Memcached improvement method based on BP neural network | |
| WO2016038749A1 (en) | A method for efficient one-to-one join | |
| CN113268457B (en) | Self-adaptive learning index method and system supporting efficient writing | |
| CN107133183A (en) | A kind of cache data access method and system based on TCMU Virtual Block Devices | |
| JP7069897B2 (en) | Learning identification device and learning identification method | |
| WO2015129109A1 (en) | Index management device | |
| CN111695685B (en) | On-chip storage system and method for graph neural network application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |