CN102971732B - The system architecture of the integrated classification query processing of key/value storer - Google Patents
The system architecture of the integrated classification query processing of key/value storer Download PDFInfo
- Publication number
- CN102971732B CN102971732B CN201180033303.8A CN201180033303A CN102971732B CN 102971732 B CN102971732 B CN 102971732B CN 201180033303 A CN201180033303 A CN 201180033303A CN 102971732 B CN102971732 B CN 102971732B
- Authority
- CN
- China
- Prior art keywords
- key
- value
- memory storage
- layer
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims description 81
- 230000005055 memory storage Effects 0.000 claims abstract description 103
- 238000000034 method Methods 0.000 claims abstract description 52
- 238000013507 mapping Methods 0.000 claims abstract description 9
- 238000003860 storage Methods 0.000 claims description 120
- 230000008569 process Effects 0.000 claims description 28
- 230000015654 memory Effects 0.000 claims description 25
- 230000008520 organization Effects 0.000 description 33
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000003672 processing method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of key/value storer, described key/value storer comprises: ground floor memory storage, and it is configured to store the information about multiple keys of multiple value, and does not store described value; And second layer memory storage, it is couple to described ground floor memory storage, and be configured to store the value associated with described key, and do not store described key, wherein compared with described second layer memory storage, described ground floor memory storage has shorter stand-by period and larger handling capacity, and wherein compared with described ground floor memory storage, described second layer memory storage has larger capacity.The invention also discloses a kind of method, described method comprises: receive key/value operation requests at ground floor memory storage; If described key/value operation requests is effective, the key in described key/value operation requests is mapped to the steady arm be stored in second layer memory storage; And if described key has corresponding steady arm, by described pointer mapping to the value in third layer memory storage.
Description
Application claims on July 2nd, 2010 by the refined denomination of invention submitted of A Fulan Kapp be " the double-deck indexing means that stand-by period is short, persistence is very high using storer (RAM) and SSD " the 61/361st, No. 256 U.S. Provisional Patent Application cases, and denomination of invention that people submits is " system architecture of the integrated classification query processing of key/value storer " the 13/098th by A Fulan Kapp is refined etc. on April 29th, 2011, the earlier application right of priority of No. 177 U.S. patent application case, the content of this earlier application is incorporated in Ben Wenben in the mode introduced.
Technical field
The present invention relates generally to network service, exactly, relates to the integrated classification query processing of key/value storer.
background of invention
Network aggregation computing machine, device and other resources, interconnected by communication channel.The communication of network reduction between network Computer user and device, and allow user can computing machine, device and other resources in share web.Network can be used for various uses by enterprise, comprises, such as data management and file and storage system.
Such as, some current network comprises the content network providing the contents such as the content such as exported as required to client.In the network of content-based route, content router be responsible for by user request and contents construction to correct recipient.In the network of content-based route, each entity transmitting a part for framework as content is assigned title unique within the scope of territory.These entities can comprise the such as data content such as video clipping or webpage, and/or the infrastructure element such as such as router, converter or server.Content router uses title prefix (can be the correct prefix of complete content name or content name) to carry out route to content packet in content network.Therefore, compared with limited Internet protocol (IP) prefix space, the name space that routing decision spatial spread becomes essence larger.Data content also can be stored in by the network of content-based route, such as, in one or more content server.
Summary of the invention
In one embodiment, the present invention includes a kind of key/value storer, described key/value storer comprises: ground floor memory storage, and it is configured to store the information about multiple keys of multiple value, but does not store described value; And second layer memory storage, it is couple to described ground floor memory storage, and be configured to store the value associated with described key, but do not store described key, wherein compared with described second layer memory storage, described ground floor memory storage has shorter stand-by period and larger handling capacity, and wherein compared with described ground floor memory storage, described second layer memory storage has larger capacity.
In another embodiment, the present invention includes a kind of network components, described network components comprises: receiver, and it is configured to receive in multiple hit inquiry and unnatural death and inquires about; Ground floor storage medium, it is configured to the multiple keys storing multiple value; Second layer storage medium, it is configured to storing value; And logical circuit, it is configured to process described hit inquiry on described ground floor storage medium and described second layer storage medium, and only processes in described unnatural death on described ground floor storage medium and inquire about, and does not access described second storage medium.
In a third aspect, the present invention includes a kind of method, described method comprises: receive key/value operation requests at ground floor memory storage; If described key/value operation requests is effective, the key in described key/value operation requests is mapped to the steady arm be stored in second layer memory storage; And if described key has corresponding steady arm, by described pointer mapping to the value in third layer memory storage.
The following detailed description of carrying out with appended claims in conjunction with the drawings will more clearly understand these and other feature.
accompanying drawing is sketched
In order to more thoroughly understand the present invention, now by reference to the accompanying drawings with detailed description with reference to following brief description, in drawings and detailed description, same reference numerals represents same section.
Fig. 1 is the schematic diagram of an embodiment of classification query processing scheme.
Fig. 2 is the schematic diagram of an embodiment of key/value storage organization.
Fig. 3 is the schematic diagram of another embodiment of key/value storage organization.
Fig. 4 is the schematic diagram of another embodiment of key/value storage organization.
Fig. 5 is the process flow diagram of an embodiment of classification inquiry processing method.
Fig. 6 is the schematic diagram of an embodiment of emitter/receiver unit.
Fig. 7 is the schematic diagram of an embodiment of general-purpose computing system.
Embodiment
Although first should be understood that the illustrative embodiment that provided hereinafter one or more embodiment, any amount of technology can be used, no matter be current known, unknown or existing, implement disclosed system and/or method.The present invention never should be limited to hereafter illustrated illustrative embodiment, accompanying drawing and technology, comprises illustrated and the exemplary design described and embodiment herein, but can revise in the full breadth of the scope of appended claims and its equipollent.
Data content can be used as value and stores in a network, such as, is stored in value storer.Can be used for the value produce index (such as, after adding value to value storer) in value storer with the interior key associated that is perhaps worth, or from value storer, retrieve the value of association.Key also can store in a network, such as, is stored in key memory.In this article, the key memory combined and value storer are called key/value storer, can be used for storing and maintain key/value entry.Key/value storer can be configured to support the multiple operation for key/value entry, comprises insertion, deletes and inquiry.
When designing key/value storer, performance and capacity are two important Considerations.Because query manipulation may be use in key/value storer to operate more frequently, therefore performance can be described as the time span that can complete inquiry, also referred to as the stand-by period.Performance also can be described as the inquiry quantity that can complete in special time period, is called handling capacity.The capacity of key/value storer is measured by the quantity of the entry that can store in key/value storer or process.Such as, based on the restriction of hardware and cost aspect, improve performance and capacity may comparatively difficulty simultaneously.Such as, the capacity (such as, specifically depending on entry size) of 1,000,000 comparatively undesirable entry magnitudes may be had for the typical key/value storer of performance optimization.Typical key/value storer for capacity optimization may have the stand-by period of comparatively undesirable millisecond magnitude, and the handling capacity of per second thousand inquiry magnitudes.
Disclosed herein is a kind of system and method, for providing compared to current system, and the key/value storer that performance and capacity can be improved.Described key/value storer can be embodied as, such as stand-alone service, or is embodied as such as data management system, file and storage system, or a part for the larger system such as the network of content-based route or network.Described system can based on key/value storage organization, and it can support that the stand-by period is relatively short, handling capacity is comparatively large, and the key/value storer that capacity is larger.Described key/value storage organization can support classification query processing and integrated hardware and software function, thus improves capacity and performance simultaneously.Classification query processing realizes by the position of the key by storage and the value disassociation of corresponding storage.
It is an embodiment of the classification query processing scheme 100 that can implement in a network shown in Fig. 1.Classification query processing scheme 100 can process the inquiry request of two types: inquire about in hit inquiry and unnatural death.Hit inquiry can comprise being stored in in value storer the inquiry request be perhaps worth, therefore can successful execution.In unnatural death, inquiry can comprise being stored in in value storer the invalid inquiry and/or inquiry request that are perhaps worth, therefore cannot perform.Classification query processing scheme 100 can comprise about three processing layers, i.e. the first query processing layer 110 at top, the second middle query processing layer 112, and the 3rd inquiry processing layer 114 of bottom.
First query processing layer 110 can be configured to process derivative key, and described derivative key is mapped to the actual key in the second query processing layer 112.Derivative key can be the compressed data structure of key, instead of actual key, therefore needs the less of resource and the speed of query processing can be made faster.Such as, the first query processing layer 110 can comprise Bloom filter (bloom filter), and it is used as the compressed data structure of key, and compared with the actual key of process, the speed of membership query can be made faster.Or the first query processing layer 110 can process received hashed key, instead of actual key, to accelerate query processing.In another embodiment, the first query processing layer 110 can use other compression bonding processing schemes.Second query processing layer 112 can be configured to process actual key, and the value steady arm described actual key is mapped in the 3rd inquiry processing layer 114 or index.Value steady arm can indicate the position of the value corresponding with key.3rd inquiry processing layer 114 can be configured to the steady arm of processing costs, thus visits the respective value that may comprise asked content with minimum process and communication resource cost.
Hit inquiry can at the first query processing layer 110, second query processing layer 112, and the 3rd follow-up inquiry processing layer 114 processes, to access the value of asking corresponding to indicated key.Therefore, all hits inquiry received can process by the every one deck in these three layers, and this represents with the arrow of formed objects between three layers, and wherein arrow size represents the quantity of inquiry request.In unnatural death, inquiry can process at the first query processing layer 110, and inquires about and can process at the second query processing layer 112 in unnatural death at least partially.Such as, only have effective query, the inquiry such as comprising effective key or compression bonding can process at the second query processing layer 112, and invalid inquiry can be given up at the first query processing layer 110 (after process).In another example, the most of inquiry request for non-existent value can filter out at the first query processing layer 110.This represents with arrow less between the first query processing layer 110 and the second query processing layer 112.In addition, to arrive in all unnatural death of the second query processing layer 112 inquiry and can give up (after process), and inquire about in any unnatural death and all can not arrive the 3rd inquiry processing layer 114, inquire about corresponding value because may not have in value storer with this type of.This inquires about lacking of arrow between processing layer 114 with the second query processing layer 112 and the 3rd and represents.
By by the segmentation of query processing step or three layers of being divided in classification query processing scheme 100, can avoid as in unnatural death, inquiry distributes unnecessary or original system resource (such as, processing time, cost and/or power), thus improve performance (such as, stand-by period and capacity).Use classification query processing scheme 100 stop in advance to inquire about in unnatural death also can limit or service-denial (DoS) in minimizing system or distributed DoS(DDoS) attack, because this type of attack may comprise invalid inquiry and/or may not have respective value in value storer.
In some systems, storing value/key can be stored in one or more memory storage, comprises dynamic RAM (DRAM), solid state drive (SSD), and hard disk drive (HDD).In these three kinds of memory storages, the performance of DRAM may be the highest but capacity is minimum, and the capacity of SSD and performance may be medium, and the capacity of HDD may be maximum but performance is minimum.Such as, the stand-by period of DRAM can be a nanosecond magnitude (such as, about 10 nanoseconds), and the handling capacity of DRAM can be GB per second (GB/s) magnitude or about per second 100,000 inquiry.The capacity of DRAM in typical server can be about 100GB.Comparatively speaking, SSD can have the stand-by period of musec order, and the handling capacity of megabytes per second (MB/s) magnitude.The capacity of SSD in typical server is about 10 times of the capacity of DRAM, such as, and about one terabyte (TB).HDD can have the stand-by period of millisecond magnitude, the handling capacity of MB/s magnitude, and is about the capacity of 10 times of capacity of SSD, such as, and 1,000,000,000 entry magnitudes.
The attribute of DRAM, SSD and HDD is summed up in Table 1.Memory storage is classified into three layers, and wherein compared to the memory storage on lower level, the memory storage in higher level has higher performance and higher cost.The ratio of Performance and Cost Modeling can represent the memory technology advanced level of memory storage.Such as, compared with the memory storage that performance/benefit cost ratio is lower, may based on more advanced technology with the memory storage that superior performance/benefit cost ratio manufactures.Additional layer can be added, because the new memory storage with different qualities and attribute may be there is in the list of table 1.
| Layer | Memory storage | Stand-by period magnitude | Handling capacity magnitude | Unit price | The max cap. of every server |
| 1 | DRAM | Nanosecond | About GB/s | About 20 dollars/GB | About 100GB |
| 2 | SSD | Microsecond | >200MB/s | About 2 dollars/GB | About 1TB |
| 3 | HDD | Millisecond | <100MB/s | About 0.05 dollar/GB | About 10TB |
Table 1. memory storage attribute
In view of attribute and the cost of memory storage, design has high-performance simultaneously and jumbo key/value storage system may be very difficult.Under normal circumstances, for obtaining high-performance, multiple key/value entry can be stored on comparatively faster memory storage, such as DRAM.But, the useful capacity of the hardware of DRAM and cost possibility limitation entry.Such as, if the mean size of entry equals several kilobyte (KB), such as, be less than about 10KB, then the total quantity of entry may be limited in 1,000,000 magnitudes by hardware and cost.The unit price of DRAM is about 10 times of the unit price of SSD, and is about 200 times of the unit price of HDD.Therefore, being placed on by key/value bar destination aggregation (mda) may than other memory storages of use costly on DRAM.
Under normal circumstances, for obtaining Large Copacity, multiple key/value entry can be stored on the larger memory storage of capacity, such as HDD.HDD can have in fact lower than the unit price of other memory storages (such as DRAM and SSD).Such as, HDD can support that size stores in the key/value of TB magnitude.If the mean size of entry equals several kilobyte (KB), then the quantity of storable key/value entry can in 1,000,000,000 magnitudes.But the performance of HDD may beyond acceptable level, such as, lower than the system requirements for stand-by period and handling capacity.Such as, the stand-by period of HDD is about 1 of the stand-by period of DRAM, 000,000 times, and the handling capacity of HDD is about about 0.1% of the handling capacity of DRAM.
Be an embodiment of key/value storage organization 200 shown in Fig. 2, it can be embodied as separate payment or be implemented in network.Key/value storage organization 200 can support to have high-performance and jumbo key/value storer simultaneously, and can based on classification query processing scheme 100.Key/value storage organization 200 such as, based on three layers of memory storage, DRAM, SSD and HDD, can provide three layers of key/value storer.Key/value storage organization 200 can comprise ground floor storage system 210, at least one second layer storage system 220, at least one third layer storage system 230, at least one central processing unit (CPU) core 240, at least one corresponding network interface 250, and one or more client 260.As shown in Figure 2, the parts of key/value storage organization 200 can be coupled to each other.Ground floor storage system 210, second layer storage system 220, third layer storage system 230, and CPU core 240 can be arranged in computer system or server, and computer system or server can be couple to each client 260 via corresponding network interface 250.Computer system or server can be arranged on or be couple to the router in network, such as, and the content router in the network of content-based route.
Ground floor storage system 210 can be configured to store and manage multiple derivative or actual key pair.Such as, ground floor storage system 210 can be the DRAM with the first query processing layer 110, and derivative key is mapped to the actual key in the second query processing layer 112 by it.Second layer storage system 220 can be configured to store and manage derivative or actual key, and multiple corresponding steady arm or index.Such as, second layer storage system 220 can be the SSD with the second query processing layer 112, and actual key is mapped to the steady arm in the 3rd inquiry processing layer 114 by it.Third layer storage system 230 can be configured to store and manage the value corresponding with steady arm and key.Such as, third layer storage system 230 can be has the 3rd inquiry processing layer 114 HDD, its by pointer mapping to be worth.
CPU core 240 can be configured to the request receiving key/value operation via network interface 250 from client 260, such as, insert, delete, and inquiry request.These requests can comprise in the hit inquiry and/or unnatural death that can be forwarded to ground floor storage system 210 to be inquired about.Such as, CPU core 240 can by request forward to the first query processing layer 110 run in ground floor storage system 210.Network interface 250 can be configured to receive request, such as content requests from client 260, and by request forward to CPU core 240.Network interface 250 also can forward from CPU core 240 be returned to client 260, replys and can comprise asked interiorly perhaps to be worth.Client 260 can comprise client node and/or device, and it provides content to multiple user.Such as, client 260 can comprise modulator-demodular unit, and it is connected to and allows client to check or the customer rs premise of play content, subscriber equipment or device, or both are all connected to.
Under normal circumstances, in some systems, key/value, to can jointly be stored in identical memory storage or system, therefore can conduct interviews asking to join one the time.In other system, key/value to different groups can be divided in same storage, to improve performance.In some systems, the memory storage that performance is higher also can be used as cache memory, thus accelerates a part for operation.But in key/value storage organization 200, key and value can carry out physical separation, and as mentioned above, are stored on the different layers (such as, having heterogeneity in performance and capacity) of storage system, thus support classification query processing.Logically, key and value still can as key/value to conducting interviews.
Key/value is placed in multilayered memory system to disassociation, can provides and there is relatively high performance and jumbo flexible key/value stores.Specifically, key (derivative key and actual key) can be stored on higher level memory storage (at aspect of performance), and described memory storage corresponds to ground floor storage system 210 and second layer storage system 220, such as DRAM and SSD.Value can be stored on lower level memory storage, and described memory storage corresponds to third layer storage system 230, such as HDD.Key and value physical separation can be realized on Different Logic path, such as, in classification query processing scheme 100, inquire about in process hit inquiry and unnatural death.
In key/value storage organization 200, derivative and actual key can be stored in ground floor storage system 210 and second layer storage system 220, to improve the overall performance of key/value storage system.Because the size of key the size of ratio may want little several magnitude, the associating size being therefore stored in the key in higher level memory storage may be key/value to or the sub-fraction of total data size of entry.Such as, key length can in the scope of several byte to a hundreds of byte.If the size of key equals about 256 bits (such as about 32 bytes), then the set of about 1,000,000,000 keys can have the size of about 32GB, and it can be stored in DRAM or SSD.If the discrepancy of quantity of key closes ground floor storage system 210(such as DRAM), then less multiple derivative key can then be stored in higher level memory storage.Such as, the Bloom filter that can store multiple hash key assignments or key in DRAM represents, instead of actual key.Derivative key can be used in classification query processing scheme 100 to represent and obtain actual key (being stored in second layer storage system 220).Derivative key also can replace actual key, inquires about for distinguishing in hit inquiry and unnatural death.
Determine that in unnatural death, inquiry can improve system performance, because the process path inquired about in unnatural death can stop in advance, to reduce the unnecessary use to system resource, such as, does not access third layer low performance storage system 230(such as, HDD) locate stored value.Derivative key or actual key are stored in ground floor storage system 210 and second layer storage system 220, instead of third layer storage system 230 also can improve in the performance determining query aspects in unnatural death, because higher level memory storage may have shorter stand-by period and larger handling capacity.In addition, value be stored in the capacity that third layer storage system 230 can increase key/value storage system and reduce costs.Compared with higher level memory storage, third layer storage system 230 can have larger capacity and lower unit price.Such as, the value stored in third layer storage system 230 can comprise about 90% of the total volume of key/value storage system, such as, in computer memory system.Because value can occupy the major part of generic key/Value Data size, therefore utilize the capacity of third layer storage system 230 can increase the total volume of system.
Value is stored in the performance that third layer storage system 230 also can improve system.Value is separated the input that can simplify key/value and management with key by this way, thus can performance be improved.Query manipulation performance can improve, because the steady arm of the value of asking can use second layer storage system 220(such as, SSD) on corresponding key obtain, therefore only may need access third layer storage system 230(such as, HDD) once come to retrieve value.If key and value are all stored in third layer storage system 230, then access may be at least needed to carry out searching value twice.Key/value also can improve update performance, because third layer storage system 230 does not need the storage map information about key and value.On the contrary, value can to manage only for the form added in third layer storage system 230, and such as, in log-structured file system, this can simplify the insertion of new entry.For optical-mechanical storage device (such as, HDD), performance is write workload to improve by continuous print.
Such as system such as multilayered memory such as key/value storage organization 200 grade can realize high-performance and Large Copacity, and also can realize the larger design flexibility of key/value storer, such as, to obtain performance and the capacity requirement of various rank.Such as, multilayered memory system can be three layers of key/value storer, such as key/value storage organization 200, or two-layer key/value storer.Such as, two-layer key/value storer can comprise DRAM and SSD, DRAM and HDD, or SSD and HDD.Multilayered memory system also can comprise the memory storage of more than three layers, and these memory storages can have different performances and capacity.
In addition, multilayered memory system support data high-speed cache, thus a part for operation is accelerated.Correspondingly, multilayered memory system can comprise the multi-layered memory devices for storage key and value, and is used as the additional memory means of cache memory or the part for some existing memory storage of key and/or the value partly stored.Compared with other memory storages in system, extra cache memory can have higher performance, and can be used for shortening the stand-by period and increasing handling capacity.Cache memory may clearly for storage key and value forever or in relatively long-time, but, such as, by storing and access portion key or value within relatively short time in processing procedure, improve the performance that key/value operates.Such as, comprising in the SSD of storage key and steady arm and the two-layer key/value for the HDD of storing value, DRAM can be used as cache memory, with in relatively very fast time memory storage and access portion key.
Be another embodiment of key/value storage organization 300 shown in Fig. 3, it can provide multilayered memory system.Key/value storage organization 300 based on three layers of memory storage, can provide three layers of key/value storer, and can support classification query processing scheme 100.Key/value storage organization 300 can comprise DRAM310, at least one SSD320, at least one HDD330, at least one CPU core 340, at least one corresponding network interface 350, and one or more client 360.The parts of key/value storage organization 300 can be configured to the corresponding component substantially similarity with key/value storage organization 100.
Suppose that the mean size of key is for about 16 bytes, and the mean size of value is about 4KB, then the total volume of key/value storage organization 300 can equal about 10,000,000,000 key/value entries.Total volume may correspond to the associating capacity in each DRAM310, SSD320 and HDD330.In addition, the handling capacity of SSD320 can be about 1/10th of the handling capacity of DRAM310, and the handling capacity of HDD330 can be about 1/50 of the handling capacity of DRAM310.In figure 3, for DRAM310, capacity is numeral with X GB(X) represent, handling capacity is numeral with Y GB/S(Y) represent.For SSD320, capacity 100x X GB represents, handling capacity is integer with N x (Y/10) GB/S(N) represent.For HDD330, capacity 1000x X GB represents, handling capacity is integer with H x (Y/50) GB/S(H) represent.In addition, the quantity of CPU core 340 is integer with C(C) represent, network interface 350(or network interface unit (NIC)) quantity be integer with M(M) represent, and the quantity of client 360 is integer with M x Z(Z) represent, wherein Z is the quantity of the client of each NIC.
DRAM310 can have the Bloom filter providing derivative key.Bloom filter can process at the first query processing layer 110, derivative key to be mapped to the actual key in the second query processing layer 112.Key can reduce space (capacity) required on DRAM310 to use Bloom filter to represent, but degree of accuracy may be caused slightly to decline.The size of Bloom filter can be used for representing member's (key) quantity, and it can be determined based on required false positive rate (FPR).Such as, if need the FPR of 1%, then about 10 bits can be enough to represent each key in Bloom filter.Therefore, the DRAM of about 32GB can store about 25,600,000,000 keys.In this case, the Bloom filter in DRAM310 can fall in majority (such as, about 99%) unnatural death to inquire about at some (such as, being less than about 10) nanosecond inner filtration.
SSD320 can have original key and steady arm, and described key and steady arm can process at the second query processing layer 210, pointer mapping to be inquired about the value in processing layer 114 to the 3rd.HDD330 can have value, and described value can process at the 3rd inquiry processing layer 114.Original key and steady arm can be used to determine whether the key inquired about is arranged in the key memory of SSD320, and determine the position in the value storer of analog value in HDD330.Inquire about in unnatural death, the stand-by period of system can be about one microsecond, and handling capacity can be several ten million inquiries about per second.For hit inquiry, the stand-by period can be about 10 milliseconds, and handling capacity may correspond in reading total throughout (such as, about some MB/s) or thousands of (such as, about 10,000) per second individual inquiry that size is the HDD330 of about 4KB.Power system capacity can be about hundreds of hundred million key/value entries.
Be another embodiment of key/value storage organization 400 shown in Fig. 4, it can provide two-layer key/value to store, and supports classification query processing scheme 100.Key/value storage organization 400 can comprise DRAM410, at least one SSD420, at least one CPU core 440, at least one corresponding network interface 450, and one or more client 460.The parts of key/value storage organization 400 can be configured to the corresponding component substantially similarity with key/value storage organization 100.DRAM410 with SSD420 can have capacity identical with DRAM310 with SSD320 respectively and handling capacity.But DRAM410 can have original key or derivative key, and steady arm, described key and steady arm can carry out processing (such as, at the second query processing layer 112) with by pointer mapping to value.SSD420 can have value, and described value can carry out processing (at the 3rd inquiry processing layer 114) to provide asked content to client 460.Such as, compared with key/value storage organization 300, key/value storage organization 400 may not comprise the HDD of storing value, and this can improve the performance of system, but can reduce capacity.
The capacity of key/value storage organization 400 can be about billions key/value entries.Each entry in DRAM410 can comprise the key of about 16 bytes and the steady arm of about 16 bytes.Each steady arm can comprise the pointer pointing to relating value on SSD420, and the length of value.The size of each pointer and length can be about 8 bytes.Therefore, the dram space that about 1,000,000,000 keys can need about 32GB is stored.Value can adopt and be stored on SSD420 only for additional form.When processing inquiry, the key of association can be arranged in DRAM410, and access SSD420 may be needed subsequently once with searching value.Therefore, if employ about 10 SSD420 in key/value storage organization 400, and load balance, then the handling capacity of SSD420 can mate the handling capacity of DRAM, and this handling capacity can be hundreds thousand of inquiry magnitudes per second.In this case, the inquiry stand-by period can determine primarily of a SSD access, and for hit inquiry, the described stand-by period is about 100 microseconds, inquires about in unnatural death, and the described stand-by period is about 10 nanoseconds.For hit inquiry, throughput of system can be 100,000 inquiries about per second, inquires about in unnatural death, and throughput of system can be 1,000 ten thousand inquiries about per second.Power system capacity can be determined by the total volume of SSD420, can be several TB magnitude or 10,000,000,000 key/value entry magnitudes.
In another embodiment, the capacity of key/value storage organization 400 is by substituting SSD420 to expand with HDD.Therefore, capacity easily extensible more than 10 times, but due to the performance of HDD lower, system latency time may increase, and handling capacity may reduce.But inquire about in unnatural death, system latency time may keep identical with the DRAM stand-by period.Compared with the typical key/value storer based on HDD, all reduce owing to inquiring about required HDD access number in hit inquiry and unnatural death, the two-layer key/value storer therefore based on HDD still can have shorter stand-by period and larger handling capacity.
In addition, the dram space in key/value storage organization 400 reduces by the derivative key of storage instead of original key in DRAM410.Such as, the hashed value of about 8 bytes can be used to the original key substituting about 16 bytes.In addition, compacter locator structure can be used in DRAM410, to save space further.But the size reducing key and/or steady arm may cause efficiency to reduce.Such as, identical with the situation that use Bloom filter carrys out alternative original key, FPR will increase.This may increase inquiry in some unnatural death and arrive SSD420, but not in the possibility that DRAM410 is rejected, this can weigh between capacity and performance.
Can based on an embodiment of the classification inquiry processing method 500 of classification query processing scheme 100 shown in Fig. 5.Classification inquiry processing method 500 can be implemented in multilevel memory system, such as, based on key/value storage organization 200, key/value storage organization 300, or key/value storage organization 400.In multilevel memory system, use classification inquiry processing method 500 can the total volume of improved system and performance (such as stand-by period and handling capacity).
Method 500 can from block 510, and wherein key/value operation requests can receive at ground floor memory storage.Key assignments operation requests can comprise the key of the relevance of asking with client, such as derivative key.Key/value operation requests can may have relatively high-performance and the enterprising row relax of ground floor device compared with low capacity at such as DRAM etc.In block 520, method 500 can OK button/Value Operations request whether effective.Such as, received request may correspond to be inquired about in unnatural death, does not comprise correct key/value operation requests or effective key, such as, in described unnatural death in inquiry when DoS attack.If key/value operation requests is effective, then method 500 can enter block 520.Otherwise key/value operation requests can be given up and method 500 can terminate.To give up in invalidation request or unnatural death inquiry can delivery system to process effective request, such as hit inquiry, thus improve performance further.The ground floor memory storage that key/value operation requests also can shorten in the stand-by period processes, and this can strengthen the protection to the DoS in system or ddos attack further.
In block 530, the key in key/value operation requests can be mapped to the steady arm be stored in second layer memory storage.Steady arm can indicate the position of the value corresponding with key.Compared with ground floor memory storage, second layer memory storage, such as SSD, can have lower-performance, but have larger capacity to hold key and steady arm.In block 540, method 500 can be determined, such as, in key/steady arm item list, whether key has corresponding value.Such as, received request may correspond to be inquired about in unnatural death, and wherein key may not have corresponding value or content, does not therefore associate with the steady arm in second layer memory storage.If key associates with steady arm, such as, have corresponding storing value, then method 500 can enter block 550.Otherwise key/value operation requests can be rejected and method 500 can terminate.Give up inquiry in this type of unnatural death and can improve system performance, and prevent the unnecessary use to resource, can conduct interviews because remove non-value, otherwise value storer can not conduct interviews.
In block 550, steady arm can be mapped to the value in third layer memory storage.Compared with second layer memory storage, third layer memory storage, such as HDD, can have lower-performance, but have larger capacity to hold steady arm and value.Value can obtain in third layer memory storage, and is supplied to client.Method 500 can terminate subsequently.In other embodiments, system can only comprise two-layer memory storage, wherein derivative key or original key can carry out processing and the steady arm be mapped in ground floor memory storage (such as DRAM or SSD), and steady arm can be mapped to the value in second layer memory storage (such as SSD or DRAM).
Be an embodiment of network components 600 shown in Fig. 6, described network components 600 can be any device via network transmission data bag and/or content.Such as, network components 600 can be arranged in or correspond to node or the router of network.Such as, network components 600 can be arranged in the router of network or any node.Network components 600 can comprise: one or more inbound port or unit 610, for receiving content, packet, object from other network componentses, and/or type-length-value (TLV); Logical circuit 620, for determining the objective network parts sending packet; And one or more outbound port or unit 630, for frame is sent to other network componentses.Network components 600 also can be implemented classification inquiry processing method 500 and carry out contents processing.
Network components mentioned above may be implemented in any general-purpose network component, such as, have computing machine or the network components of the network throughput capability of the necessary workload on enough processing poweies, memory source and process network.Figure 7 shows that the typical general-purpose network component 700 of the one or more embodiment being suitable for implementing parts disclosed herein.Network components 700 comprises processor 702(and can be described as central processing unit or CPU), itself and storage communication, described memory storage comprises supplementary storage 704, ROM (read-only memory) (ROM) 706, random-access memory (ram) 708, I/O (I/O) device 710 and network connection device 712.Processor 702 can be used as one or more cpu chip and implements, or can be a part of one or more special IC (ASIC).
Supplementary storage 704 is made up of one or more disc driver or tape drive usually, and for the non-volatile memories of data, if RAM708 does not have enough spaces to preserve all operational datas, so described supplementary storage 704 can be used as overflow data memory storage.Supplementary storage 704 can in order to store this class method when selection performs the program be loaded in RAM708.The instruction of reading the term of execution that ROM706 being for being stored in program and possible data.ROM706 is Nonvolatile memory devices, its memory capacity relative to supplementary storage 704 compared with usually less for large storage capacity.RAM708 is used for stores volatile data, also may be used for storing instruction.Access ROM706 and RAM708 is usually fast than access supplementary storage 704.
The invention discloses at least one embodiment, and the feature of one of ordinary skill in the art to embodiment and/or embodiment make change, combination and/or amendment all within the scope of the invention.Because of combination, merge and/or omit alternate embodiment that the feature of embodiment produces also within the scope of the invention.When clearly stating numerical range or restriction, these expression scopes or restriction should be interpreted as that (such as, from about 1 to about 10 comprises 2,3,4 etc. for the repetition scope of the similar magnitude in the scope that comprises and belong to and clearly state or restriction or restriction; Be greater than 0.10 and comprise 0.11,0.12,0.13 etc.).For example, when announcement has the numerical range of lower limit Rl and upper limit Ru, any numeral belonging to described scope is specifically disclosed.Specifically, following numeral in described scope be clear and definite disclosed in: R=Rl+k* (Ru-Rl), wherein k is the variable increased progressively with 1% in 1% to 100% scope, that is, k be 1%, 2%, 3%, 4%, 5% ..., 70%, 71%, 72% ..., 95%, 96%, 97%, 98%, 99% or 100%.In addition, any numerical range defined by two defined above digital R be also clear and definite disclosed in.Use term " optionally " to mean that described element is needs relative to arbitrary element of claim, or described element is unwanted, two kinds of replacement schemes are all in the scope of described claim.Use such as " comprise ", the wider term such as " comprising " and " having " be interpreted as providing to such as " by ... composition ", " substantially by ... composition " and " substantially by ... form " etc. the support of narrower term.Therefore, protection domain does not limit by explanation stated above, but is defined by appended claims, and described scope comprises all equivalents of the subject matter of appended claims.Each and each claim are incorporated in instructions as further disclosure, and appended claims is embodiments of the invention.The discussion that reference in described disclosure is carried out not is admitted that it is prior art, especially there is any reference of the publication date after the earlier application right of priority date of subject application.The disclosure of all patents, patent application case and the publication quoted in the present invention is incorporated herein by reference hereby, and it provides supplementary exemplary, procedural or other details of the present invention.
Although provide some embodiments in the present invention, should be understood that without departing from the spirit or scope of the present invention, the system and method disclosed can other particular forms many be implemented.Example of the present invention should be considered as illustrative and nonrestrictive, and the invention is not restricted to details given in this article.For example, various element or parts can combine or integrated in another system, or some feature can be omitted or not implement.
In addition, without departing from the scope of the invention, described in each embodiment and discrete or single technology, system, subsystem and the method that illustrate can with other system, module, technology or Combination of Methods or merging.Show or be discussed as other project being coupled to each other or directly coupling or communicate and also electric mode, mechanical system or alternate manner indirectly can be coupled by a certain interface, device or intermediate member or communicate.Other examples changing, replace and change can be determined by those skilled in the art, and make when not departing from spirit and scope disclosed herein.
Claims (19)
1. a key/value storer, described key/value storer comprises:
Ground floor memory storage, it is configured to store the information about multiple keys of multiple value, and does not store described value; And
Second layer memory storage, it is couple to described ground floor memory storage, and is configured to store the described value associated with described key, and does not store described key,
Wherein compared with described second layer memory storage, described ground floor memory storage has shorter stand-by period and larger handling capacity, and
Wherein compared with described ground floor memory storage, described second layer memory storage has larger capacity.
2. key/value storer according to claim 1, comprises further:
Third layer memory storage, it is couple to described ground floor memory storage and described second layer memory storage, and is configured to store the multiple steady arms associated with described value with described key,
Wherein compared with described ground floor memory storage, described third layer memory storage has compared with high latency and less handling capacity, and compared with described second layer memory storage, described third layer memory storage has shorter stand-by period and larger handling capacity, and
Wherein compared with described ground floor memory storage, described third layer memory storage has larger capacity, and compared with described second layer memory storage, described third layer memory storage has comparatively low capacity.
3. key/value storer according to claim 2, wherein said ground floor memory storage is dynamic RAM (DRAM), described third layer memory storage is solid-state drive (SSD), and described second layer memory storage is hard disk drive (HDD).
4. key/value storer according to claim 2, wherein said key represents with Bloom filter in described ground floor device, wherein said steady arm is stored in the item list in described third layer memory storage together with corresponding key, and wherein said value associates to the corresponding steady arm in described second layer memory storage.
5. key/value storer according to claim 2, wherein corresponding with described key multiple hashed key are stored in described ground floor device, wherein said steady arm is stored in the item list in described third layer memory storage together with corresponding key, and wherein said value associates to the corresponding steady arm in described second layer memory storage.
6. key/value storer according to claim 1, wherein said ground floor memory storage is dynamic RAM (DRAM), and described second layer memory storage is solid-state drive (SSD).
7. key/value storer according to claim 1, wherein said key and the multiple steady arms associated with described key are stored in the item list in described ground floor memory storage, and wherein said value associates to the corresponding steady arm in described second layer memory storage.
8. a network components, described network components comprises:
Receiver, it is configured to receive in multiple hit inquiry and unnatural death and inquires about;
First storage medium, it is configured to the multiple keys storing multiple value;
Second storage medium, it is configured to store described value; And
Logical circuit, it is configured to process described hit inquiry on described first storage medium and described second storage medium, and only processes in described unnatural death on described first storage medium and inquire about, and does not access described second storage medium.
9. network components according to claim 8, wherein said hit inquiry comprises multiple effective key, it is the related value of tool in described second storage medium, and inquiry comprises multiple invalid key in wherein said unnatural death, its not related value of tool in described second storage medium, or correspond to service-denial (DoS)/Distributed Denial of Service attack (DDoS) and attack.
10. network components according to claim 9, the processing mode of wherein said hit inquiry is effective key is mapped to the multiple steady arms in described first storage medium, then by described pointer mapping to the described value in described second storage medium.
11. network componentses according to claim 9, the processing mode of inquiring about in wherein said unnatural death is, give up invalid key or by query mappings in described unnatural death to the multiple steady arms in described first storage medium, then give up in the unnatural death irrelevant with the value in described second storage medium and inquire about.
12. network componentses according to claim 9, comprise further:
3rd storage medium, it is configured to store the multiple steady arms associated with described value with described key.
13. network componentses according to claim 12, wherein said first storage medium comprises dynamic RAM (DRAM), described 3rd storage medium comprises solid-state drive (SSD), and described second storage medium comprises hard disk drive (HDD).
14. network componentses according to claim 12, wherein said key represents with Bloom filter in described first storage medium, wherein said steady arm is stored in the item list in described 3rd storage medium together with corresponding key, and wherein said value associates to the corresponding steady arm in described second storage medium.
15. network componentses according to claim 12, wherein corresponding with described key multiple hashed key are stored in described first storage medium, wherein said steady arm is stored in the item list in described 3rd storage medium together with corresponding key, and wherein said value associates to the corresponding steady arm in described second storage medium.
16. 1 kinds of key/value querying methods, described method comprises:
Key/value operation requests is received at ground floor memory storage;
If described key/value operation requests is effective, the key in described key/value operation requests is mapped to the steady arm be stored in second layer memory storage; And
If described key has corresponding steady arm, by described pointer mapping to the value in third layer memory storage;
Wherein said key/value operation requests carries out processing to obtain described key at the first query processing layer, and wherein said key is mapped to the steady arm on the second query processing layer, and wherein said pointer mapping is to the value on the 3rd inquiry processing layer.
17. methods according to claim 16, comprise further:
At the 3rd top memory storage, use Bloom filter in key operation request or hashed key by described key/value operation map to described key.
18. methods according to claim 16, comprise further:
If described key/value operation requests or described key invalid, described first query processing layer give up described key/value operation; And
If described key does not have corresponding steady arm, give up described key/value operation requests at described second query processing layer.
19. methods according to claim 16, comprise further:
At the 3rd top memory storage or in a part for higher level memory storage, to a part for described key, a part for described value, or both carries out high-speed cache, and uses the key of institute's buffer memory and value to process described key/value operation requests.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US36125610P | 2010-07-02 | 2010-07-02 | |
| US61/361,256 | 2010-07-02 | ||
| US13/098,177 US8433695B2 (en) | 2010-07-02 | 2011-04-29 | System architecture for integrated hierarchical query processing for key/value stores |
| US13/098,177 | 2011-04-29 | ||
| PCT/CN2011/076716 WO2012000448A1 (en) | 2010-07-02 | 2011-06-30 | System architecture for integrated hierarchical query processing for key/value stores |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102971732A CN102971732A (en) | 2013-03-13 |
| CN102971732B true CN102971732B (en) | 2015-09-30 |
Family
ID=45400613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201180033303.8A Active CN102971732B (en) | 2010-07-02 | 2011-06-30 | The system architecture of the integrated classification query processing of key/value storer |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US8433695B2 (en) |
| CN (1) | CN102971732B (en) |
| WO (1) | WO2012000448A1 (en) |
Families Citing this family (51)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104508647B (en) * | 2012-06-08 | 2018-01-12 | 慧与发展有限责任合伙企业 | For the method and system for the memory span for expanding ultra-large computing system |
| US8780632B2 (en) | 2012-11-09 | 2014-07-15 | Sandisk Technologies Inc. | De-duplication techniques using NAND flash based content addressable memory |
| US8817541B2 (en) | 2012-11-09 | 2014-08-26 | Sandisk Technologies Inc. | Data search using bloom filters and NAND based content addressable memory |
| US8780635B2 (en) | 2012-11-09 | 2014-07-15 | Sandisk Technologies Inc. | Use of bloom filter and improved program algorithm for increased data protection in CAM NAND memory |
| US8792279B2 (en) | 2012-11-09 | 2014-07-29 | Sandisk Technologies Inc. | Architectures for data analytics using computational NAND memory |
| US8634248B1 (en) | 2012-11-09 | 2014-01-21 | Sandisk Technologies Inc. | On-device data analytics using NAND flash based intelligent memory |
| US8773909B2 (en) | 2012-11-09 | 2014-07-08 | Sandisk Technologies Inc. | CAM NAND with or function and full chip search capability |
| US8780634B2 (en) | 2012-11-09 | 2014-07-15 | Sandisk Technologies Inc. | CAM NAND with OR function and full chip search capability |
| US8811085B2 (en) | 2012-11-09 | 2014-08-19 | Sandisk Technologies Inc. | On-device data analytics using NAND flash based intelligent memory |
| US9098403B2 (en) * | 2012-11-09 | 2015-08-04 | Sandisk Technologies Inc. | NAND flash based content addressable memory |
| US8780633B2 (en) | 2012-11-09 | 2014-07-15 | SanDisk Technologies, Inc. | De-duplication system using NAND flash based content addressable memory |
| US9075424B2 (en) | 2013-03-06 | 2015-07-07 | Sandisk Technologies Inc. | Compensation scheme to improve the stability of the operational amplifiers |
| US20150074084A1 (en) * | 2013-09-12 | 2015-03-12 | Neustar, Inc. | Method and system for performing query processing in a key-value store |
| US20150205541A1 (en) * | 2014-01-20 | 2015-07-23 | Samya Systems, Inc. | High-capacity solid state disk drives |
| US10956050B2 (en) * | 2014-03-31 | 2021-03-23 | Sandisk Enterprise Ip Llc | Methods and systems for efficient non-isolated transactions |
| US9916356B2 (en) | 2014-03-31 | 2018-03-13 | Sandisk Technologies Llc | Methods and systems for insert optimization of tiered data structures |
| US9411539B2 (en) * | 2014-09-24 | 2016-08-09 | International Business Machines Corporation | Providing access information to a storage controller to determine a storage tier for storing data |
| US9438426B2 (en) | 2014-10-03 | 2016-09-06 | Seagate Technology Llc | Key-value data storage device with hybrid architecture |
| US9639566B2 (en) | 2014-12-18 | 2017-05-02 | Here Global B.V. | Method, apparatus and computer program product for improved storage of key-value pairs |
| KR102338872B1 (en) * | 2015-03-12 | 2021-12-13 | 삼성전자주식회사 | Storage apparatus and method for processing a plurality of client data |
| US10031884B2 (en) * | 2015-02-11 | 2018-07-24 | Samsung Electronics Co., Ltd | Storage apparatus and method for processing plurality of pieces of client data |
| KR102335798B1 (en) * | 2015-02-11 | 2021-12-07 | 삼성전자주식회사 | Storage apparatus and method for processing a plurality of client data |
| KR102506135B1 (en) | 2015-03-16 | 2023-03-07 | 삼성전자주식회사 | Data storage device and data processing system having the same |
| US10133764B2 (en) | 2015-09-30 | 2018-11-20 | Sandisk Technologies Llc | Reduction of write amplification in object store |
| US9619165B1 (en) | 2015-10-30 | 2017-04-11 | Sandisk Technologies Llc | Convertible leaf memory mapping |
| US9846651B2 (en) | 2016-01-22 | 2017-12-19 | Samsung Electronics Co., Ltd. | Computing system with cache management mechanism and method of operation thereof |
| US11455097B2 (en) | 2016-01-28 | 2022-09-27 | Weka.IO Ltd. | Resource monitoring in a distributed storage system |
| US10289340B2 (en) | 2016-02-23 | 2019-05-14 | Sandisk Technologies Llc | Coalescing metadata and data writes via write serialization with device-level address remapping |
| US10185658B2 (en) | 2016-02-23 | 2019-01-22 | Sandisk Technologies Llc | Efficient implementation of optimized host-based garbage collection strategies using xcopy and multiple logical stripes |
| US10747676B2 (en) | 2016-02-23 | 2020-08-18 | Sandisk Technologies Llc | Memory-efficient object address mapping in a tiered data structure |
| US10225344B2 (en) | 2016-08-12 | 2019-03-05 | International Business Machines Corporation | High-performance key-value store using a coherent attached bus |
| US20180095720A1 (en) * | 2016-09-30 | 2018-04-05 | Intel Corporation | Storage device with fine grained search capability |
| CN106843749B (en) * | 2016-12-14 | 2020-01-21 | 华为技术有限公司 | Write request processing method, device and equipment |
| JP6825379B2 (en) * | 2017-01-19 | 2021-02-03 | 富士通株式会社 | Information processing equipment, information processing methods and programs |
| US11321402B2 (en) * | 2017-05-05 | 2022-05-03 | Microsoft Technology Licensing, Llc. | Index storage across heterogenous storage devices |
| US20180364937A1 (en) * | 2017-06-20 | 2018-12-20 | Samsung Electronics Co., Ltd. | System and method for managing memory device |
| US20190155735A1 (en) * | 2017-06-29 | 2019-05-23 | NVXL Technology, Inc. | Data Software System Assist |
| US10922239B2 (en) | 2017-12-29 | 2021-02-16 | Samsung Electronics Co., Ltd. | Device for performing iterator operation in database |
| US10949087B2 (en) * | 2018-05-15 | 2021-03-16 | Samsung Electronics Co., Ltd. | Method for rapid reference object storage format for chroma subsampled images |
| US10915546B2 (en) | 2018-10-10 | 2021-02-09 | Micron Technology, Inc. | Counter-based compaction of key-value store tree data block |
| CN109521959A (en) * | 2018-11-01 | 2019-03-26 | 西安交通大学 | One kind being based on SSD-SMR disk mixing key assignments memory system data method for organizing |
| US11048755B2 (en) * | 2018-12-14 | 2021-06-29 | Micron Technology, Inc. | Key-value store tree with selective use of key portion |
| US10852978B2 (en) | 2018-12-14 | 2020-12-01 | Micron Technology, Inc. | Key-value store using journaling with selective data storage format |
| US10936661B2 (en) | 2018-12-26 | 2021-03-02 | Micron Technology, Inc. | Data tree with order-based node traversal |
| CN109933542A (en) * | 2019-02-27 | 2019-06-25 | 记忆科技(深圳)有限公司 | Based on solid state hard disk AES key dynamic management approach and device |
| CN110083307A (en) * | 2019-03-29 | 2019-08-02 | 华为技术有限公司 | Date storage method, memory and server |
| US11727057B2 (en) | 2019-09-04 | 2023-08-15 | Samsung Electronics Co., Ltd. | Network key value indexing design |
| CN110635908B (en) * | 2019-09-29 | 2023-03-24 | 杭州尚尚签网络科技有限公司 | Management method for supporting billions of keys for electronic contract |
| US11645424B2 (en) * | 2020-04-27 | 2023-05-09 | International Business Machines Corporation | Integrity verification in cloud key-value stores |
| CN113094372A (en) | 2021-04-16 | 2021-07-09 | 三星（中国）半导体有限公司 | Data access method, data access control device and data access system |
| US11989142B2 (en) | 2021-12-10 | 2024-05-21 | Samsung Electronics Co., Ltd. | Efficient and concurrent model execution |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1139489A (en) * | 1993-11-02 | 1997-01-01 | 帕拉科姆有限公司 | Apparatus for accelerating processing of transactions on computer databases |
Family Cites Families (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5960194A (en) * | 1995-09-11 | 1999-09-28 | International Business Machines Corporation | Method for generating a multi-tiered index for partitioned data |
| JP2001331509A (en) | 2000-05-22 | 2001-11-30 | Hitachi Ltd | Relational database processor, relational database processing method, and computer-readable recording medium recorded with relational database processing program |
| US6880064B1 (en) * | 2000-06-21 | 2005-04-12 | Mosaid Technologies, Inc. | Method and apparatus for physical width expansion of a longest prefix match lookup table |
| US6763148B1 (en) * | 2000-11-13 | 2004-07-13 | Visual Key, Inc. | Image recognition methods |
| US6804677B2 (en) * | 2001-02-26 | 2004-10-12 | Ori Software Development Ltd. | Encoding semi-structured data for efficient search and browsing |
| US7836178B1 (en) * | 2001-06-20 | 2010-11-16 | Microstrategy Incorporated | Technique for limiting access to the resources of a system |
| WO2003001720A2 (en) * | 2001-06-21 | 2003-01-03 | Isc, Inc. | Database indexing method and apparatus |
| US20030074342A1 (en) * | 2001-10-11 | 2003-04-17 | Curtis Donald S. | Customer information management infrastructure and methods |
| US7185317B2 (en) * | 2002-02-14 | 2007-02-27 | Hubbard & Wells | Logical data modeling and integrated application framework |
| US7089230B2 (en) * | 2002-06-07 | 2006-08-08 | International Business Machines Corporation | Method for efficient processing of multi-state attributes |
| US20050105513A1 (en) * | 2002-10-27 | 2005-05-19 | Alan Sullivan | Systems and methods for direction of communication traffic |
| US8447743B2 (en) * | 2004-08-17 | 2013-05-21 | International Business Machines Corporation | Techniques for processing database queries including user-defined functions |
| US20060155915A1 (en) * | 2004-12-30 | 2006-07-13 | Pereira Jose P | Database query processor |
| US20060161525A1 (en) * | 2005-01-18 | 2006-07-20 | Ibm Corporation | Method and system for supporting structured aggregation operations on semi-structured data |
| US7898817B2 (en) * | 2006-10-17 | 2011-03-01 | Hewlett-Packard Development Company, L.P. | Circuit board assembly |
| US7743053B2 (en) * | 2006-10-17 | 2010-06-22 | Hewlett-Packard Development Company, L.P. | Hybrid database query caching |
| US20080319957A1 (en) * | 2007-06-19 | 2008-12-25 | Microsoft Corporation | Extensible command trees for entity data model platform |
| US7836037B2 (en) * | 2007-10-04 | 2010-11-16 | Sap Ag | Selection of rows and values from indexes with updates |
| US8013738B2 (en) * | 2007-10-04 | 2011-09-06 | Kd Secure, Llc | Hierarchical storage manager (HSM) for intelligent storage of large volumes of data |
| US7925656B2 (en) * | 2008-03-07 | 2011-04-12 | International Business Machines Corporation | Node level hash join for evaluating a query |
| CN101593196B (en) * | 2008-05-30 | 2013-09-25 | 日电（中国）有限公司 | Method, device and system for rapidly searching ciphertext |
| US9274950B2 (en) * | 2009-08-26 | 2016-03-01 | Hewlett Packard Enterprise Development Lp | Data restructuring in multi-level memory hierarchies |
| US8190631B2 (en) * | 2009-11-24 | 2012-05-29 | Business Objects Software Limited | Query-time and view-time security for semantic layer |
| US8996563B2 (en) * | 2010-04-06 | 2015-03-31 | Tokutek, Inc. | High-performance streaming dictionary |
| US8447785B2 (en) * | 2010-06-02 | 2013-05-21 | Oracle International Corporation | Providing context aware search adaptively |
| US8997056B2 (en) | 2010-12-15 | 2015-03-31 | Hewlett-Packard Development Company, L. P. | Directed graphs pertaining to read/write operations |
-
2011
- 2011-04-29 US US13/098,177 patent/US8433695B2/en active Active
- 2011-06-30 WO PCT/CN2011/076716 patent/WO2012000448A1/en active Application Filing
- 2011-06-30 CN CN201180033303.8A patent/CN102971732B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1139489A (en) * | 1993-11-02 | 1997-01-01 | 帕拉科姆有限公司 | Apparatus for accelerating processing of transactions on computer databases |
Also Published As
| Publication number | Publication date |
|---|---|
| US20120005419A1 (en) | 2012-01-05 |
| US8433695B2 (en) | 2013-04-30 |
| WO2012000448A1 (en) | 2012-01-05 |
| CN102971732A (en) | 2013-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102971732B (en) | The system architecture of the integrated classification query processing of key/value storer | |
| US10198363B2 (en) | Reducing data I/O using in-memory data structures | |
| CN101727465B (en) | Methods for establishing and inquiring index of distributed column storage database, device and system thereof | |
| EP2793436B1 (en) | Content router forwarding plane architecture | |
| CN101247417B (en) | Double-layer metadata processing system and method | |
| CN108628986A (en) | Data query method, apparatus, computer equipment and storage medium | |
| CN105550371A (en) | Big data environment oriented metadata organization method and system | |
| CN110235098A (en) | Storage system access method and device | |
| CN101236569B (en) | Highly effective dynamic path analysis method based on ContextFS | |
| CN1504912A (en) | Performance and memory bandwidth utilization for tree searches using tree fragmentation | |
| CN102122285A (en) | Data cache system and data inquiry method | |
| CN104584512A (en) | A method for collaborative caching for content-oriented networks | |
| CN104272263A (en) | Fragmentation control for performing deduplication operations | |
| CN105260429A (en) | ICN network information name searching method based on multiple Bloom filters | |
| CN107463606B (en) | Data compression engine and method for big data storage system | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN104111924A (en) | Database system | |
| Von der Weth et al. | Multiterm keyword search in NoSQL systems | |
| US20080270352A1 (en) | Modifying entry names in directory server | |
| CN110199512A (en) | The management method and device of equipment are stored in storage system | |
| CN108647266A (en) | A kind of isomeric data is quickly distributed storage, exchange method | |
| CN104915148A (en) | System and method for efficient content caching in a streaming storage | |
| CN114844846A (en) | Multi-level cache distributed key value storage system based on programmable switch | |
| JPH1097544A (en) | Database processing system | |
| CN105912696A (en) | DNS (Domain Name System) index creating method and query method based on logarithm merging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |