CN106682184A - Light-weight combination method based on log combination tree structure - Google Patents
Light-weight combination method based on log combination tree structure Download PDFInfo
- Publication number
- CN106682184A CN106682184A CN201611246968.8A CN201611246968A CN106682184A CN 106682184 A CN106682184 A CN 106682184A CN 201611246968 A CN201611246968 A CN 201611246968A CN 106682184 A CN106682184 A CN 106682184A
- Authority
- CN
- China
- Prior art keywords
- dtable
- metadata
- key
- value pair
- dtables
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a light-weight combination method based on a log combination tree structure. The method disclosed by the invention is characterized in that sequenced key value pair tables with dynamic size changes are used in the log combination tree structure, and metadata of the lower-layer sequenced key value pair table is stored in an upper-layer sequenced key value pair table with an overlaid key value scope; according to the balance degree of each sequenced key value pair table layer in the log combination tree, corresponding sequenced key value pair tables are taken as cooperation objects; the metadata in the selected sequenced key value pair tables is combined, and the combined metadata and corresponding key value pair data are written into the lower-layer sequenced key value pair table with the overlaid key value scope through supplementation; key value scopes of the sequenced key value pair tables are adjusted, so loads of the sequenced key value pair tables on the same level could be balanced; and a subsection dichotomy search manner is used to conduct rapid and efficiency search of the key value pair data. The method has the advantages that the combination operation cost of the log combination tree can be reduced; write amplification can be reduced; and performance of the log combination tree structure can be enhanced.
Description
Technical field
The invention belongs to technical field of information storage, more particularly, to a kind of light weight for merging tree construction based on daily record
Level merging method.
Background technology
High speed development and the popularization of the Internet with information technology, data volume occurs in that rapid growth, using clothes
The data storage size of business and the visit capacity of data also increase therewith, and the availability requirement and performance to database storage system will
Ask and also further improve.Traditional relevant database is distinctly claimed due to the data structure to storing, so cannot meet
Current service application reads and writes demand to the high concurrent of data base, high efficiency storage to mass data and the demand for accessing and right
The enhanced scalability of data base and the demand of high availability.Such issues that in order to solve, non-relational database is arisen at the historic moment.
And in recent years, daily record merges tree (Log-Structured-Merge Tree, LSM Tree) in NoSQL systems
Obtain widely applying, the storage model for becoming main flow has been even more especially in key assignments storage system.Daily record merges tree framework and carries
Go out the algorithm for postponing to update and batch writes, random write has been converted to into batch and is write, reduce the mobile expense of disk arm, so as to
The write performance of data base has been greatly improved, current high concurrent and high performance demand for services has preferably been met.
In prior art, although daily record merges tree and can provide good write performance, in order to maintain good reading
Can, needs constantly merge operation on backstage, make data ordering, this be also caught daily record merge set merging expense into
Merge the performance bottleneck in the key assignments storage system of tree algorithm in order to be based on daily record.So, in order to mitigate the conjunction that daily record merges tree
And impact of the operation to key assignments performance of storage system, it would be desirable to it is a kind of to reduce the excellent of the union operation expense that daily record merging is set
Change method.
The content of the invention
For the disadvantages described above or Improvement requirement of prior art, the invention provides a kind of merge tree construction based on daily record
Lightweight merging method, its object is to merge when tree carries out backstage Dtable merging in daily record and reduces read-write amplification；By inciting somebody to action
Metadata in lower floor overlap Dtables is stored in the Victim Dtable of upper strata；By the key assignments model for adjusting Dtable
Enclose the supplemental data amount of balance Dtables；Thus the expense that daily record merges tree union operation is reduced, the reading that daily record merges tree is eliminated
Write amplifying technique problem.
For achieving the above object, the invention provides a kind of lightweight merging method for being merged tree construction based on daily record, is somebody's turn to do
Method is comprised the following steps：
(1) use size dynamic change and there is the orderly key-value pair table of upper limit threshold, be designated as Dtable, the wherein upper limit
Threshold value span is 1MB to 256MB, and preferred value is 4MB；Dtable includes two kinds of data, including metadata and key assignments logarithm
According to：Wherein metadata can be divided into two parts again, it is assumed that the Dtable merges the Li layers of tree in daily record, while by Li
In+1 layer range of key values one or more Dtables Chong Die with the Dtable are designated as overlap Dtables, then Dtable
Metadata will include the metadata of Dtable itself, and the metadata of overlap Dtables；
The range of key values of the Dtable of Li layers can occur with the range of key values of one or more Dtable of Li+1 layers
Overlap, the range of key values of the Dtable of Li+1 layers only can overlap with the range of key values of Li layers Dtable；
Storage order of the metadata and key-value pair data in Dtable in table is followed successively by, the metadata of overlap Dtables, key
Value is to data, the metadata of Dtable；
(2) merge the balance degree of each Dtable layer in tree according to daily record, choose corresponding Dtable right as merging
As being designated as victim Dtable；
(3) in the merging process of victim Dtable and corresponding overlap Dtables, merge be stored in first
The metadata of the metadata of the victim Dtable in victim Dtable and corresponding overlap Dtables, then will
Metadata and corresponding key-value pair data after merging is write in corresponding overlap Dtables in the way of adding；
Further, the lightweight merging method there may be load uneven in merging process with layer Dtables
Weighing apparatus, in the case where overhead is not caused, adjusts as far as possible the range of key values of each Dtables, makes the Dtables of same level
Load reaches equilibrium state.
Further, the lightweight merging method is in merging process, supplemental data and do not change legacy data, therefore
There is range of key values to cover between data additional several times in Dtable, the key-value pair inside Dtables is no longer orderly, is raising
The speed of data search in Dtable, using the order of single supplemental data, proposes segmentation binary chop method.According to first number
According to the additional order of section, back to front each metadata section is made a look up using binary chop, until obtaining final result.
Further, the step (2) is including following sub-step：
(21) equilibrium degree that daily record merges each layer of tree is calculated, the equilibrium degree of Li layers is equal to the size of the Dtable of Li layers
With the ratio of quantity；
(22) the minimum Dtable layers of equilibrium degree are selected as merging layer；
(23) Dtable merged in layer is chosen in order as victim Dtable, until merging all of layer
After Dtable is selected, terminate the selection of victim Dtable.
Further, the step (3) is including following sub-step：
(31) the victim Dtable for choosing step (2) are read in internal memory；
(32) metadata of victim Dtable is merged in the metadata of each overlap Dtables；
(33) range of key values of the metadata of each overlap Dtables after being merged according to (32) step,
Key-value pair data in victim Dtable is divided into corresponding data block；
(34) number that will be divided in the metadata and (33) step of each overlap Dtables after the merging of (32) step
It is respectively written in each overlap Dtables in the form of additional writing according to block；
(35) metadata of each overlap Dtables after the merging of (32) step is write back into victim again
Dtable, terminates union operation.
Further, the step (32) is including following sub-step：
(321) merge the metadata of victim Dtable, delete the data rope of range of key values lap in metadata
Draw；
(322) by the metadata of the victim Dtable after merging in (321) step according to each overlap
Dtables range of key values, is divided into corresponding metadata section, and the unit of corresponding overlap Dtables is then appended to respectively
The end of data, terminates metadata union operation.
Further, the step (33) is including following sub-step：
(331) merge the key-value pair data of victim Dtable, delete invalid key-value pair data；
(332) by the key-value pair data of the victim Dtable after merging in (331) step according to each overlap
The range of key values of the metadata of Dtables, is divided into corresponding data block, terminates key-value pair data division operation.
Further, the present invention for layer Dtable load imbalances situation, adjust Dtable load include as
Lower sub-step：
(41) Dtable of load imbalance can be divided into two classes, and a class is by metadata range of key values unreasonable distribution
Cause, it is another kind of to be caused by key-value pair data amount is excessive, if the range of key values of Dtable is unreasonable, into step
(42), if the key-value pair data amount of Dtable is excessive, into step (44)；
(42) in the case where key-value pair data is not caused to migration, the range of key values of Dtable is reduced, simultaneous equal expands
The range of key values of big adjacent Dtable, make each Dtable after adjustment range of key values and originally as；
(43) if the range of key values of adjacent Dtable also occurs the phenomenon of distribution unreasonable distribution because of this adjustment,
Into step (42), otherwise terminate the range of key values adjustment of Dtable；
(44) Dtable is read in internal memory, key value order is ranked up to the key-value pair data in Dtable, generates
New metadata；
(45) Dtable after sequence is divided into into by measure multiple new Dtable, the load for terminating Dtable is adjusted
It is whole.
In general, by the contemplated above technical scheme of the present invention compared with prior art, it is special with following technology
Levy and beneficial effect：
(1) present invention provide it is a kind of based on daily record merge tree construction lightweight merging method, daily record merge set into
Read-write is reduced when row backstage Dtable merges to amplify, and lifts combined efficiency；
(2) a kind of lightweight merging method for merging tree construction based on daily record that the present invention is provided, by by lower floor
Metadata in overlap Dtables is stored in the Victim Dtable of upper strata, and reducing daily record merging tree carries out lightweight conjunction
And when magnetic disc i/o；
(3) a kind of lightweight merging method for merging tree construction based on daily record that the present invention is provided, it is proposed that zero-overhead
Load balancing, the supplemental data amount of Dtables is balanced by adjusting the range of key values of Dtable, maintains daily record to merge tree
It is harmonious.
Description of the drawings
Fig. 1 is the Dtable mathematical logic schematic layout patterns in the embodiment of the present invention；
Fig. 2 is that the Victim Dtable in the embodiment of the present invention select flow chart；
Fig. 3 is the lightweight merging method schematic diagram in the embodiment of the present invention；
Fig. 4 is the range of key values adjustment schematic diagram of the Dtable in the embodiment of the present invention；
Fig. 5 is the fractionation schematic diagram of the Dtable in the embodiment of the present invention.
Specific embodiment
In order that the objects, technical solutions and advantages of the present invention become more apparent, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the present invention, and
It is not used in the restriction present invention.As long as additionally, technical characteristic involved in invention described below each embodiment
Not constituting conflict each other just can be mutually combined.
The present invention Integral Thought be：The orderly key-value pair of size dynamic change is merged used in tree construction in daily record
Table, the metadata of the orderly key-value pair table of lower floor is stored in the orderly key-value pair table in upper strata of range of key values overlap；According to daily record
Merge the balance degree on each orderly key-value pair top layer in tree, choose corresponding key-value pair table in order as combining objects；Merge
Metadata in the orderly key-value pair table chosen, and the side by the metadata after merging and corresponding key-value pair data to add
Formula writes the orderly key-value pair table of lower floor of key assignments overlapping ranges；Adjusting the range of key values of orderly key-value pair table makes the orderly of same level
The load of key-value pair table reaches equilibrium state；Using segmentation binary search mode, key-value pair data is rapidly and efficiently searched.
It is Dtable mathematical logics schematic layout pattern in the embodiment of the present invention shown in Fig. 1.Data in Dtable can draw
It is divided three classes：It is respectively
The metadata of lower floor Overlap Dtables；
The key-value pair data block of Dtable；
The metadata of Dtable itself.
Wherein, the corresponding Overlap Dtables of Dtable may have one or more, so Overlap Dtables
Metadata of the metadata equivalent to one or more Dtables set；
The key-value pair data block of Dtable includes the data added in data block during initialization, and follow-up union operation
Block；
The metadata of Dtable itself includes the data that initialized data block is indexed and subsequently added in union operation
Block is indexed.
The lightweight merging method for being merged tree construction based on daily record that embodiment is provided, is comprised the following steps：
(1) use size dynamic change and upper limit threshold has been the orderly key-value pair table of 4MB, be designated as Dtable；Dtable
Comprising two kinds of data, including metadata and key-value pair data.Wherein metadata can be divided into two parts again, it is assumed that the Dtable
Merge the Li layers of tree in daily record, at the same by range of key values is Chong Die with the Dtable in Li+1 layers one or more
Dtable is designated as overlap Dtables, then the metadata of Dtable will include the metadata of Dtable itself, and
The metadata of overlap Dtables；The range of key values of the Dtable of Li layers can be with one or more of Li+1 layers
The range of key values of Dtable overlaps, and the range of key values of the Dtable of Li+1 layers only can be with Li layers Dtable's
Range of key values overlaps；Storage order of the metadata and key-value pair data in Dtable in table is followed successively by, overlap
The metadata of Dtables, key-value pair data, the metadata of Dtable；
(2) merge the balance degree of each Dtable layer in tree according to daily record, choose corresponding Dtable right as merging
As being designated as victim Dtable；
(3) in the merging process of victim Dtable and corresponding overlap Dtables, merge be stored in first
The metadata of the metadata of the victim Dtable in victim Dtable and corresponding overlap Dtables, then will
Metadata and corresponding key-value pair data after merging is write in corresponding overlap Dtables in the way of adding；
It is that Victim Dtable select flow chart in the embodiment of the present invention shown in Fig. 2.Lightweight union operation is selected
First according to the balance degree of each Dtable layer come selection combining layer during Victim Dtable, equilibrium degree is poorer, and representing should
Dtable layers get over needs and merge operation.
In embodiment, step (2) is including following sub-step：
(21) equilibrium degree that daily record merges each layer of tree is calculated, the equilibrium degree of Li layers is equal to the size of the Dtable of Li layers
With the ratio of quantity；
(22) the minimum Dtable layers of equilibrium degree are selected as merging layer；
(23) Dtable merged in layer is chosen in order as victim Dtable, until merging all of layer
After Dtable is selected, terminate the selection of victim Dtable.
It is lightweight merging method schematic diagram in the embodiment of the present invention shown in Fig. 3.In Fig. 3, range of key values is a-c in L1
Dtable be selected as victim Dtable and merge operation；In L2 layers, range of key values and victim Dtable phases
Mutually overlap Dtable has three, and range of key values is respectively a, b and c, the corresponding overlap of composition victim Dtable
Dtables gathers.The right and left respectively illustrates the state that each Dtable layer carries out before and after lightweight merging in Fig. 3.
In embodiment, step (3) is including following sub-step：
(31) the victim Dtable for choosing step (2) are read in internal memory；
(32) metadata of victim Dtable is merged in the metadata of each overlap Dtables；
(33) range of key values of the metadata of each overlap Dtables after being merged according to (32) step,
Key-value pair data in victim Dtable is divided into corresponding data block；
(34) number that will be divided in the metadata and (33) step of each overlap Dtables after the merging of (32) step
It is respectively written in each overlap Dtables in the form of additional writing according to block；
(35) metadata of each overlap Dtables after the merging of (32) step is write back into victim again
Dtable, terminates union operation.
Wherein, step (32) also includes following sub-step：
(321) merge the metadata of victim Dtable, delete the data rope of range of key values lap in metadata
Draw；
(322) by the metadata of the victim Dtable after merging in (321) step according to each overlap
Dtables range of key values, is divided into corresponding metadata section, and the unit of corresponding overlap Dtables is then appended to respectively
The end of data, terminates metadata union operation.
Wherein, step (33) also includes following sub-step：
(331) merge the key-value pair data of victim Dtable, delete invalid key-value pair data；
(332) by the key-value pair data of the victim Dtable after merging in (331) step according to each overlap
The range of key values of the metadata of Dtables, is divided into corresponding data block, terminates key-value pair data division operation.
In embodiment, the lightweight merging method there may be load uneven in merging process with layer Dtables
Weighing apparatus, therefore in the case of being additionally included in and not causing overhead, the range of key values of each Dtables is adjusted as far as possible, make same layer
The step of secondary Dtables loads reach equilibrium state (4).
It is the range of key values adjustment schematic diagram of Dtable in the embodiment of the present invention shown in Fig. 4.In order to reduce adjustment key assignments model
Caused overhead when enclosing, the range of key values adjustment of Dtable is typically carried out after lightweight union operation, now
Not comprising related key-value pair data in Dtable, the migration that range of key values is not result in key-value pair data is adjusted.In Fig. 4, Li layers
Dtable1 range of key values distribution before merging is unbalanced, so by the key assignments model of Dtable1 and Dtable2 after union operation
Enclose and be adjusted, the range of key values for making Dtable1 and Dtable2 is adjusted to a-b, c-d by a-c, the d before merging.
It is the fractionation schematic diagram of Dtable in the embodiment of the present invention shown in Fig. 5.In Fig. 5, Li layer range of key values is e-f's
Dtable sizes reach upper limit threshold values 4MB, it is impossible to carry out one-side range of key values adjustment, so needing in the Dtable
Range of key values and key-value pair data equivalent split two Dtable of layer, and range of key values is e and f in the Li layers after splitting in such as Fig. 5
Two Dtable.
In embodiment, step (4) is including following sub-step：
(41) Dtable of load imbalance can be divided into two classes, and a class is by metadata range of key values unreasonable distribution
Cause, it is another kind of to be caused by key-value pair data amount is excessive, if the range of key values of Dtable is unreasonable, into step
(42), if the key-value pair data amount of Dtable is excessive, into step (44)；
(42) in the case where key-value pair data is not caused to migration, the range of key values of Dtable is reduced, simultaneous equal expands
The range of key values of big adjacent Dtable, make each Dtable after adjustment range of key values and originally as；
(43) if the range of key values of adjacent Dtable also occurs the phenomenon of distribution unreasonable distribution because of this adjustment,
Into step (42), otherwise terminate the range of key values adjustment of Dtable；
(44) Dtable is read in internal memory, key value order is ranked up to the key-value pair data in Dtable, generates
New metadata；
(45) Dtable after sequence is divided into into by measure multiple new Dtable, the load for terminating Dtable is adjusted
It is whole.
In embodiment, the lightweight merging method in merging process, supplemental data and do not change legacy data, therefore
There are key range to cover in Dtable between data additional several times, the key-value pair inside Dtables is no longer orderly, is raising
The speed of data search in Dtable, using the order of single supplemental data, proposes segmentation binary chop method.According to first number
According to the additional order of section, back to front each metadata section is made a look up using binary chop, until obtaining final result.
Presently preferred embodiments of the present invention is the foregoing is only, not to limit the present invention, all essences in the present invention
Any modification, equivalent and improvement made within god and principle etc., should be included within the scope of the present invention.
Claims (8)
1. it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that the inventive method include following step
Suddenly：
(1) use size dynamic change and there is the orderly key-value pair table of upper limit threshold, be designated as Dtable；Dtable includes unit
Data and key-value pair data：Wherein metadata includes the metadata of Dtable and the metadata of overlap Dtables, wherein,
If set the Dtable merges tree Li layers in daily record, daily record is merged into range of key values and the Dtable weights in tree Li+1 layers
One or more folded Dtable are designated as overlap Dtables；
(2) merge the balance degree of each Dtable layer in tree according to daily record, choose corresponding Dtable as combining objects, note
For victim Dtable；
(3) merge victim Dtable and corresponding overlap Dtables, merge be stored in victim Dtable first
Victim Dtable metadata and the metadata of corresponding overlap Dtables, then by the metadata after merging with
And corresponding key-value pair data is write in corresponding overlap Dtables in the way of adding.
2. it is according to claim 1 it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that institute
Lightweight merging method is stated in merging process, with layer Dtables load imbalance is there may be, do not causing overhead
In the case of, the range of key values of each Dtables is adjusted as far as possible, the Dtables loads for making same level reach equilibrium state.
3. it is according to claim 1 it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that institute
Lightweight merging method is stated in merging process, supplemental data and do not change legacy data, according to the additional order of metadata section, by
Using binary chop each metadata section is made a look up forward afterwards, until finding or not finding key-value pair data.
4. a kind of lightweight merging method for merging tree construction based on daily record according to claim 1 and 2, its feature exists
In the step (2) is including following sub-step：
(21) equilibrium degree that daily record merges each Dtable layers of tree is calculated, each Dtable of the equilibrium degree of Li layers according to Li layers
Size and number ratio calculation；
(22) the minimum Dtable layers of equilibrium degree are selected as merging layer；
(23) Dtable merged in layer is chosen in order as victim Dtable, until merging all of layer
After Dtable is selected, terminate the selection of victim Dtable.
5. a kind of lightweight merging method for merging tree construction based on daily record according to claim 1 and 2, its feature exists
In the step (3) is including following sub-step：
(31) combining objects victim Dtable are read in internal memory；
(32) metadata of victim Dtable is merged in the metadata of each overlap Dtables；
(33) according to the range of key values of the metadata of each overlap Dtables after merging, in victim Dtable
Key-value pair data is divided into corresponding data block；
(34) data block after the metadata of each overlap Dtables after merging and division is divided in the form of additional writing
In not writing each overlap Dtables；
(35) metadata of each overlap Dtables after merging is write back into victim Dtable again, terminates to merge behaviour
Make.
6. it is according to claim 5 it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that institute
Sub-step (32) is stated including following sub-step：
(321) merge the metadata of victim Dtable, delete the data directory of range of key values lap in metadata；
(322) by the metadata of the victim Dtable after merging according to each overlap Dtables range of key values, divide
Into corresponding metadata section, the end of the metadata of corresponding overlap Dtables is then appended to respectively, terminate metadata
Union operation.
7. it is according to claim 5 it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that institute
Sub-step (33) is stated including following sub-step：
(331) merge the key-value pair data of victim Dtable, delete invalid key-value pair data；
(332) by the key-value pair data of the victim Dtable after merging according to the metadata of each overlap Dtables
Range of key values, is divided into corresponding data block, terminates key-value pair data division operation.
8. it is according to claim 2 it is a kind of based on daily record merge tree construction lightweight merging method, it is characterised in that
The adjustment of load of Dtable includes following sub-step：
(41) the reason for judging Dtable load imbalances, if caused by metadata range of key values unreasonable distribution, then enter
Enter step (42)；If being caused by key-value pair data amount is excessive, then into step (44)；
(42) in the case where key-value pair data is not caused to migration, the range of key values of Dtable is reduced, simultaneous equal expands phase
The range of key values of adjacent Dtable, make each Dtable after adjustment range of key values and originally as；
(43) if the range of key values of adjacent Dtable also occurs the phenomenon of unreasonable distribution because of this adjustment, into step
(42)；Otherwise terminate the adjustment of load of Dtable；
(44) Dtable is read in internal memory, key value order is ranked up to the key-value pair data in Dtable, generates new
Metadata；
(45) Dtable after sequence is divided into into by measure multiple new Dtable, terminates the adjustment of load of Dtable.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611246968.8A CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611246968.8A CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106682184A true CN106682184A (en) | 2017-05-17 |
| CN106682184B CN106682184B (en) | 2019-12-20 |
Family
ID=58872790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611246968.8A Active CN106682184B (en) | 2016-12-29 | 2016-12-29 | Lightweight merging method based on log merging tree structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106682184B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
| CN110377227A (en) * | 2019-06-13 | 2019-10-25 | 阿里巴巴集团控股有限公司 | A kind of data block storage method, apparatus and electronic equipment |
| US10983975B2 (en) | 2019-06-13 | 2021-04-20 | Ant Financial (Hang Zhou) Network Technology Co., Ltd. | Data block storage method and apparatus, and electronic device |
| CN113688096A (en) * | 2021-07-15 | 2021-11-23 | 三星（中国）半导体有限公司 | Storage method, storage device and storage system |
| CN116595015A (en) * | 2023-07-18 | 2023-08-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN105045891A (en) * | 2015-07-31 | 2015-11-11 | 中国科学院计算技术研究所 | Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus |
| CN105159915A (en) * | 2015-07-16 | 2015-12-16 | 中国科学院计算技术研究所 | Dynamically adaptive LSM (Log-structured merge) tree combination method and system |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
| US20160179865A1 (en) * | 2014-12-17 | 2016-06-23 | Yahoo! Inc. | Method and system for concurrency control in log-structured merge data stores |
-
2016
- 2016-12-29 CN CN201611246968.8A patent/CN106682184B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN105447059A (en) * | 2014-09-29 | 2016-03-30 | 华为技术有限公司 | Data processing method and device |
| US20160179865A1 (en) * | 2014-12-17 | 2016-06-23 | Yahoo! Inc. | Method and system for concurrency control in log-structured merge data stores |
| CN105159915A (en) * | 2015-07-16 | 2015-12-16 | 中国科学院计算技术研究所 | Dynamically adaptive LSM (Log-structured merge) tree combination method and system |
| CN105045891A (en) * | 2015-07-31 | 2015-11-11 | 中国科学院计算技术研究所 | Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
| CN109558084B (en) * | 2018-11-29 | 2021-08-17 | 文华学院 | Data processing method and related equipment |
| CN110377227A (en) * | 2019-06-13 | 2019-10-25 | 阿里巴巴集团控股有限公司 | A kind of data block storage method, apparatus and electronic equipment |
| CN110377227B (en) * | 2019-06-13 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data block storage method and device and electronic equipment |
| US10983975B2 (en) | 2019-06-13 | 2021-04-20 | Ant Financial (Hang Zhou) Network Technology Co., Ltd. | Data block storage method and apparatus, and electronic device |
| CN113688096A (en) * | 2021-07-15 | 2021-11-23 | 三星（中国）半导体有限公司 | Storage method, storage device and storage system |
| CN116595015A (en) * | 2023-07-18 | 2023-08-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN116595015B (en) * | 2023-07-18 | 2023-12-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106682184B (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106682184A (en) | Light-weight combination method based on log combination tree structure | |
| CN109196459B (en) | Decentralized distributed heterogeneous storage system data distribution method | |
| CN104133661B (en) | Multi-core parallel hash partitioning optimizing method based on column storage | |
| CN105117415B (en) | A kind of SSD data-updating methods of optimization | |
| CN104361113B (en) | A kind of OLAP query optimization method under internal memory flash memory mixing memory module | |
| CN102200892B (en) | Capacity expansion method based on dynamic redundant array of independent disks (RAID) system | |
| JP6243045B2 (en) | Graph data query method and apparatus | |
| CN105468542B (en) | Address distribution method and device | |
| CN109558084A (en) | A kind of data processing method and relevant device | |
| CN104331253B (en) | Calculation method for object migration in object storage system | |
| CN104484233B (en) | A kind of resource allocation methods | |
| CN106445677A (en) | Load balancing method and device | |
| CN108536692A (en) | A kind of generation method of executive plan, device and database server | |
| WO2016191995A1 (en) | Method and device for partitioning association table in distributed database | |
| CN108647155A (en) | A kind of method and apparatus that the multistage cache based on deep learning is shared | |
| CN110532200A (en) | A kind of memory system based on mixing memory architecture | |
| CN103049574A (en) | Key value system and key value method for implementation of dynamic duplicates of documents | |
| CN108519856A (en) | Based on the data block copy laying method under isomery Hadoop cluster environment | |
| CN102158533B (en) | Distributed web service selection method based on QoS (Quality of Service) | |
| CN105681414A (en) | Method and system for avoiding data hotspot of Hbase | |
| CN100377118C (en) | Built-in file system realization based on SRAM | |
| CN107426315A (en) | A kind of improved method of the distributed cache system Memcached based on BP neural network | |
| CN108427648A (en) | Storage system page internal organ data index method and device | |
| CN105893466A (en) | Data processing method and system applied to database | |
| CN107153680A (en) | A kind of method and system of the online expanding node of distributed memory database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |