CN106294772A - The buffer memory management method of distributed memory columnar database - Google Patents
The buffer memory management method of distributed memory columnar database Download PDFInfo
- Publication number
- CN106294772A CN106294772A CN201610659223.8A CN201610659223A CN106294772A CN 106294772 A CN106294772 A CN 106294772A CN 201610659223 A CN201610659223 A CN 201610659223A CN 106294772 A CN106294772 A CN 106294772A
- Authority
- CN
- China
- Prior art keywords
- caching
- node
- buffer queue
- weight
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the buffer memory management method of a kind of distributed memory columnar database, including: in caching main controlled node, set up buffer queue；Cut the physics implement plan at its place with each physical tasks for root node and calculate track to obtain caching corresponding to each physical tasks；Calculate track according to the caching that each physical tasks is corresponding and build cache feature tree in caching main controlled node；When inquiry request arrives, SQL statement is resolved to physics implement plan by query execution engine；Each node level traversal physics implement plan is started, it is judged that caching corresponding to each physical tasks calculates whether track mates with cache feature tree from the root node of physics implement plan；If coupling, from node, directly read the caching real data of this physical tasks from caching, otherwise calculate this physical tasks.By efficient cache match algorithm, the buffer memory management method of the distributed memory columnar database that the present invention provides, detects rapidly whether caching hits, improves search efficiency.
Description
Technical field
The present invention relates to computer software technical field, be specifically related to the caching pipe of a kind of distributed memory columnar database
Reason method.
Background technology
Along with the development of information age, data scale presents explosive increase, how to extract from these mass datas and has
The information being worth, is the huge challenge that faces of current social.And on-line analytical processing (OLAP, On-Line Analytical
Processing) system is demonstrated by the data analysis capabilities that it is powerful, and it is widely used to bank, telecommunications, stock exchange
Deng commercial field.
The distributed memory columnar database supporting OLAP system allows user to extract from mass data with multiple dimensions
With the valuable information of analysis, these information are probably a simple form, it is also possible to a complicated analysis result.With
The lifting of query statement complexity, the time required for inquiry operation also can be the longest, and the query statement of high complexity is propping up
Hold the frequency occurred in the distributed memory columnar database of OLAP system the highest.
In data base, inquiry request has the strongest dependency in terms of semanteme, and therefore partial query result is likely to going through
History inquiry occurred.Distributed memory columnar database introduces cache management system and preserves some historical query information, from
And reduce the execution number of times repeating inquiry.Fig. 1 is the structural representation of the cache management system of distributed memory columnar database,
Described cache management system includes query execution engine (Query Engine) 11, caching main controlled node (Cache Master)
12, secondary node (Standby Cache Master) 13 and at least one caching are from node (Cache Slave) 14.Its
In, described query execution engine 11 is responsible for resolving and performing user's SQL request, returns Query Result；Described caching main controlled node
12 are responsible for all cachings from node 14, safeguard cache metadata, eliminate caching according to related algorithm, safeguard that caching is consistent
Property；All cache metadatas of main controlled node 12 are cached, when described caching master control saves described in described secondary node 13 periodic synchronization
When point 12 breaks down, substitute described caching main controlled node 12 immediately, continue to provide buffer service；Described caching is negative from node 14
Duty memory buffers real data, responds described query execution engine 11 and reads the request of caching.
Data that how cache access frequency is high as far as possible under distributed scene, data base querying speed of accelerating, just
The problem that distributed caching management system is to be solved.
Summary of the invention
To be solved by this invention is data, the quickening number how cache access frequency is high as far as possible under distributed scene
Problem according to library inquiry speed.
The present invention is achieved through the following technical solutions:
A kind of buffer memory management method of distributed memory columnar database, the caching of described distributed memory columnar database
Management system include query execution engine, caching main controlled node and at least one caching from node, described buffer memory management method
Including: setting up buffer queue in caching main controlled node, in described buffer queue, each element corresponds to a physical tasks
Cache metadata；The physics implement plan at its place is cut to obtain each physical tasks pair for root node with each physical tasks
The caching answered calculates track；Calculate track according to the caching that each physical tasks is corresponding in caching main controlled node, build caching spy
Levy tree；When inquiry request arrives, SQL statement is resolved to physics implement plan by query execution engine；From physics implement plan
Root node start each node in level traversal physics implement plan, it is judged that caching corresponding to each physical tasks calculates track
Whether mate with described cache feature tree；If coupling, from node, directly read the caching actual number of this physical tasks from caching
According to, otherwise calculate this physical tasks.
The present invention uses caching to calculate track and uniquely identifies caching, when inquiry request arrives, it is only necessary to by each physics
Whether caching corresponding to task calculates track and mates with cache feature tree, it is possible to detect rapidly to cache and hits, from certain journey
Reduce the calculating of iterative task in distributed data base on degree, save query time, improve search efficiency.
Further, in described buffer queue, each element presses weight order arrangement from big to small.
Further, in described buffer queue the weight of each element according to Wi=qi×(a×Si+b×Pi) obtain, its
In, WiIt is the weight of i-th element, qiIt is the weight factor of the physical tasks that i-th element is corresponding, SiBe i-th element time
Empty than andtiIt is that i-th corresponding caching of element calculates the time arriving root node longest path in track, kiFor
The storage strategy constant that i-th element is corresponding, miIt is the system memory space shared by i-th element reality, PiIt is i-th element
Hit frequency andniIt is the historical hit number of times of i-th element, diIt is i-th last hit of element distance
Time interval, viBeing that i-th element averagely hits time interval, a is the factor of influence of space-time comparison weight, and b is hit frequency
The rate factor of influence to weight, i is positive integer.
Further, the buffer memory management method of described distributed memory columnar database also includes when described cache management system
System receives and carries out following steps when the new storage cached is asked: step S1, updates the weight of each element in described buffer queue；
Step S2, it is judged that whether described cache management system current residual space be enough to store new caching；If described cache management system
Current residual space be enough to store new caching, performs step S3, and caching main controlled node notice caching, will from the new caching of node storage
New cache metadata puts into described buffer queue, and the caching that the new caching of record is corresponding in described cache feature tree calculates rail
Mark；If described cache management system current residual space is not enough to store new caching, perform step S4, it is judged that the weight of new caching
Whether more than the weight of last element in described buffer queue；If the weight of new caching is not more than in described buffer queue
The weight of latter element, performs step S5, and caching main controlled node notice caching is from the node new caching of refusal storage；If newly caching
Weight more than the weight of last element in described buffer queue, perform step S6, it may be judged whether just still having inquiry operation
Last element in using described buffer queue；If still there being inquiry operation to be currently in use in described buffer queue last
Element, performs step S7, is to be deleted by last rubidium marking in described buffer queue；If being currently in use without inquiry operation
Last element in described buffer queue, performs step S8, and caching main controlled node deletes in described buffer queue last
Element, reclaims the memory space that the caching that is eliminated in described cache management system takies, and deletes institute from described cache feature tree
State the caching that in buffer queue, last element is corresponding and calculate track, and forward step S2 to.
Further, the buffer memory management method of described distributed memory columnar database also includes: for described buffer queue
In each element one corresponding reference count is set, when an element initial hit, corresponding for this element is quoted meter
Number puts 1, and starts a timer；If before the timer expires, caching main controlled node receives the SQL language using an element
The feedback that sentence inquiry is complete, then subtract 1 by the reference count of this element, and reset timer；Corresponding quoting of element
When counting is equal to 0, then close timer.
Further, it may be judged whether still having inquiry operation to be currently in use last element in described buffer queue is judgement
Whether the reference count that in described buffer queue, last element is corresponding is 0.
Further, the caching that each physical tasks is corresponding calculates track and includes that the result of each node calculates time, knot
The data transmission period that really storage size and each edge represent.
Further, in described cache feature tree, table involved by all nodes is owned by version number.
The present invention compared with prior art, has such advantages as and beneficial effect:
The buffer memory management method of the distributed memory columnar database that the present invention provides, by efficient cache match algorithm
Whether detection caching hits rapidly, utilizes reasonably caching life cycle algorithm to ensure system availability and stability, to a certain degree
On decrease the calculating of iterative task in distributed data base, save query time and memory space, improve search efficiency.
Accompanying drawing explanation
Accompanying drawing described herein is used for providing being further appreciated by the embodiment of the present invention, constitutes of the application
Point, it is not intended that the restriction to the embodiment of the present invention.In the accompanying drawings:
Fig. 1 is the structural representation of the cache management system of distributed memory columnar database；
Fig. 2 is the structural representation of the physics implement plan of the embodiment of the present invention；
Fig. 3 is the structural representation of the caching calculating track of the embodiment of the present invention；
Fig. 4 is the structural representation of the cache feature tree of the embodiment of the present invention；
Fig. 5 is the structural representation of the T2-Join calculating track of the embodiment of the present invention；
Fig. 6 is the structural representation of the T3-Join calculating track of the embodiment of the present invention；
Fig. 7 is the structural representation of the cache match result of the embodiment of the present invention；
Fig. 8 is the schematic flow sheet of the superseded method of caching of the embodiment of the present invention.
Detailed description of the invention
For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with embodiment and accompanying drawing, to this
Invention is described in further detail, and the exemplary embodiment of the present invention and explanation thereof are only used for explaining the present invention, do not make
For limitation of the invention.
Embodiment
The present embodiment provides the buffer memory management method of a kind of distributed memory columnar database, described distributed memory column
The structural representation of the cache management system of data base refers to Fig. 1, including query execution engine, caching main controlled node, standby
Node and at least one caching are from node.
When inquiry request arrives, SQL statement is resolved to the physics implement plan represented by DAG by query execution engine.
One physical tasks of each node on behalf in physics implement plan, physical tasks be divided into again GetColumn, Join, Filter,
Group and BuildRow etc., each edge represents the transmission relation of result of calculation between two physical tasks.Article one, typical inquiry
The physics of statement (SELECT A.id FROM A, B WHERE A.id=B.id AND A.id≤100AND B.id≤80)
Implement plan is as shown in Figure 1.In cache management system, data cached granularity is the result of calculation of single physical task.When
When caching is the result of calculation of BuildRow, then cache the final Query Result of whole piece SQL statement.The present embodiment uses caching
Calculate track and uniquely identify caching.
First, setting up buffer queue in caching main controlled node, in described buffer queue, each element corresponds to a thing
The cache metadata of reason task.As it was previously stated, data cached granularity is the result of calculation of single physical task, initially setting up
During buffer queue, described cache management system has enough memory spaces, therefore, when receiving cache request, directly by thing
The result of calculation of reason task is stored in caching from node, using the cache metadata of physical tasks as a unit of buffer queue
Element.
The physics implement plan at its place is cut for root node corresponding to obtain each physical tasks with each physical tasks
Caching calculate track.Specifically, each physical tasks of request caching is all to there being a physics implement plan.Hold at physics
Row in the works, obtains subgraph with the physical tasks that this request caches for root node cutting physics implement plan, and this subgraph is exactly root
The caching that node is corresponding calculates track.Caching calculates track record from initial data to all information of this caching of generation, bag
The result including each node calculates the data transmission period that time, result storage size and each edge represent.Caching calculates rail
Each node of mark is a characteristic point, and all characteristic points and relation thereof constitute the feature of caching.Perform with the physics shown in Fig. 2
As a example by plan, if the result of calculation of user's T3-Join to be cached physical tasks, then with this physical tasks node as root node,
Obtain corresponding caching and calculate track as shown in Figure 3.
Calculate track according to the caching that each physical tasks is corresponding and build cache feature tree in caching main controlled node.Described
Cache feature tree have recorded characteristic point and the relation thereof of each caching, and it is represented by hierarchical structure tissue, the number of plies according to table relation
The table number that this layer relates to, the bottom pertains only to a table.One cache feature point of each node on behalf of described cache feature tree,
Representing mutual dependence by line between characteristic point, the feature of a caching is by series of features point and relation structure thereof
Become.
Data cached inconsistent, in described cache feature tree involved by all nodes with data base's initial data in order to prevent
Table be owned by version number, only just can mate on the premise of table version number is consistent.User attempts reading or writing delaying
When depositing, input all comprises the version number that the table related to is up-to-date in current distributed database, when caching main controlled node finds certain
When the version of individual table is expired, then delete all cachings relevant with this table.One typical cache feature tree as shown in Figure 4, its
In, solid box represents that the caching that this feature point describes exists, and the caching that empty wire frame representation this feature point describes does not exists, and the latter is only
It is to ensure that the feature integrity of caching belonging to it.
When inquiry request arrives, SQL statement is resolved to physics implement plan, physics implement plan by query execution engine
Represented by physical tasks DAG.Those skilled in the art know query execution engine and how SQL statement are resolved to physics and hold
Row plan, does not repeats them here.
Each node level traversal physics implement plan is started, it is judged that each physics from the root node of physics implement plan
Caching corresponding to task calculates whether track mates with described cache feature tree.Cache match includes two kinds of situations: mate completely
With part coupling.Coupling completely refers to that the caching of request calculates all characteristic points and relation thereof in track and can cache spy described
Levy in tree and find；Part coupling is substantially consistent with coupling completely, and unique difference is, it is described that caching calculates the root node of track
The subset of certain characteristic point in cache feature tree.If coupling, directly from node, read the caching of this physical tasks in fact from caching
Border data, otherwise calculate this physical tasks.
On the basis of the cache feature tree that Fig. 4 describes, SQL statement (SELECT A.id FROM A, the B described with Fig. 2
WHERE A.id=B.id AND A.id≤100AND B.id≤80) as a example by, from physics implement plan root node start layers
The each node of secondary traversal, corresponding cache match step is as follows:
Whether detection T1-BuildRow hits caching.It is exactly whole DAG figure that the caching of T1-BuildRow calculates track, from
Root node starts level and scans whether each characteristic point hits, as long as detecting that a characteristic point is miss, then this task is not ordered
Middle caching.Owing to node T1 relates to two Table A B, so searching whether to exist this feature point at the Level 2 of characteristics tree, actual
Result is nothing, so the miss caching of T1-BuildRow.
Whether detection T2-Join hits caching.The caching of T2-Join calculates track as it is shown in figure 5, node T2 coupling caches
Characteristic point 2_1 in characteristics tree, characteristic point 1_1 in node T5 coupling cache feature tree, and node T4 is the son of characteristic point 1_5
Collection, but relation (1_5,2_1) does not exists, so the miss caching of T2-Join.
Whether detection T3-Join hits caching.The caching of T3-Join calculates track as shown in Figure 6, and node T3 mates caching
Characteristic point 2_1 in characteristics tree, characteristic point 1_2 in node T6 coupling cache feature tree, and understood node T5 coupling by aforementioned
Characteristic point 1_1 in cache feature tree, relation (T5, T3) matching relationship (1_1,2_1), relation (T6, T3) matching relationship (1_2,
2_1), so T3-Join mates completely.
Node T4 is the subset of characteristic point 1_5, i.e. T4-GetColumn part coupling, node T5 matching characteristic point 1_1, therefore
T5-GetColumn mates completely.Node T6 matching characteristic point 1_2, i.e. T6-GetColumn mates completely.
After above-mentioned steps terminates, the cache match result of this SQL statement can be obtained as shown in Figure 7.Can from Fig. 7
Going out, query execution engine has only to calculate T1 and T2 physical tasks, it is possible to completes the inquiry of whole piece SQL statement, greatly carries
High search efficiency.
Owing to the memory space of cache management system is limited, after memory space is the most occupied, receive new caching
Storage request time need existing caching is eliminated.It is one of the core of cache management system that caching is eliminated, and determines slow
The swapping in and out deposited, and then affect hit rate and the stability of buffer service of caching.In the present embodiment, described buffer queue
In each element by weight order arrangement from big to small, tail of the queue weight is minimum.When eliminating caching, all eject unit from tail of the queue every time
Element.
Assume tiIt is that i-th corresponding caching of element calculates the time (unit: ms) arriving root node longest path in track,
miIt is the system memory space (unit: Byte) shared by i-th element reality, kiIt is that the storage strategy that i-th element is corresponding is normal
Amount, different storage strategies, mode and the time of its reading cache data are the most different.The space-time of i-th element compares computing formula
For:
Assume diIt is the time interval (unit: ms) of i-th last hit of element distance, niIt is the history of i-th element
Hit-count, viIt is that i-th element averagely hits time interval (unit: ms).The hit frequency computing formula of i-th element
For:If i-th element is to add enqueue, then P firsti=1.
In sum, in described buffer queue, the weight calculation formula of i-th element is: Wi=qi×(a×Si+b×
Pi).Wherein, WiIt is the weight of i-th element, qiIt it is the weight factor of the physical tasks that i-th element is corresponding.I-th element pair
The physical tasks answered is the most complicated, weight factor q of the physical tasks that i-th element is correspondingiValue the biggest.Such as Join task pair
The value of the weight factor that the value of the weight factor answered is more corresponding than GetColumn task is big.Under default situations, i-th element pair
Weight factor q of the physical tasks answerediEqual to 1.0.A and b is all constant, represents space-time ratio and hit frequency respectively to weight
Factor of influence.If system memory space is smaller, then can tune up the value of a, increase the impact of space-time comparison weight, otherwise then
Turn a down.
In the present embodiment, one corresponding reference count is set for each element in described buffer queue, is representing
Use the inquiry operation number of this caching.When an element initial hit, the reference count that this element is corresponding puts 1, and
Starting a timer, default value can be configured according to practical situation, such as 30 seconds.If it is before the timer expires, slow
Deposit main controlled node and receive certain the SQL statement complete feedback of inquiry using an element, then corresponding for this element is quoted
Counting subtracts 1, and resets timer.When reference count corresponding to element is equal to 0, then close timer.The work of timer
Prevent query execution engine from breaking down with being, cause caching main controlled node and cannot receive the feedback that inquiry is complete.
When caching is eliminated, if the reference count of the caching correspondence that is eliminated not is 0, illustrate still have inquiry operation using
This caching, if deleting this caching immediately, may cause query execution engine to obtain cache failure, expending over head time and carrying out appointing
Business fault recovery.The caching being eliminated only is marked as to be deleted, is the most really deleted until reference count is 0.
Fig. 8 is the schematic flow sheet of the superseded method of caching of the embodiment of the present invention, when system receives depositing of a new caching
During storage request, caching main controlled node carries out following steps:
Step S1, updates the weight of each element in described buffer queue.Due in buffer queue each element apart from upper
The time interval of hit at first time can elapse increase over time, so needing first to update the weight of each element in buffer queue.
Step S2, it is judged that whether described cache management system current residual space be enough to store new caching.
If described cache management system current residual space be enough to store new caching, perform step S3, cache main controlled node
New cache metadata, from the new caching of node storage, is put into described buffer queue by notice caching, and in described cache feature tree
The caching that the new caching of record is corresponding calculates track.
If described cache management system current residual space is not enough to store new caching, perform step S4, it is judged that newly cache
Weight whether more than the weight of last element in described buffer queue.
If the weight of new caching is not more than the weight of last element in described buffer queue, perform step S5, caching
Main controlled node notice caching is from the node new caching of refusal storage.
If the weight of new caching is more than the weight of last element in described buffer queue, perform step S6, it is judged that be
No still have inquiry operation to be currently in use last element in described buffer queue, i.e. judges in described buffer queue last
Whether the reference count that element is corresponding is 0.
If still there being inquiry operation to be currently in use last element in described buffer queue, perform step S7, by described slow
It is to be deleted for depositing last rubidium marking in queue.Treat that the reference count that in described buffer queue, last element is corresponding is
When 0, then last element in described buffer queue is deleted.
If being currently in use last element in described buffer queue without inquiry operation, perform step S8, caching master control joint
Last element in buffer queue described in point deletion, reclaims the storage sky that the caching that is eliminated in described cache management system takies
Between, from described cache feature tree, delete the caching that in described buffer queue, last element is corresponding calculate track, and forward to
Step S2.
Above-described detailed description of the invention, has been carried out the purpose of the present invention, technical scheme and beneficial effect further
Describe in detail, be it should be understood that the detailed description of the invention that the foregoing is only the present invention, be not intended to limit the present invention
Protection domain, all within the spirit and principles in the present invention, any modification, equivalent substitution and improvement etc. done, all should comprise
Within protection scope of the present invention.
Claims (8)
1. a buffer memory management method for distributed memory columnar database, the caching pipe of described distributed memory columnar database
Reason system includes that query execution engine, caching main controlled node and at least one caching are from node, it is characterised in that described caching
Management method includes:
Setting up buffer queue in caching main controlled node, in described buffer queue, each element corresponds to the slow of a physical tasks
Deposit metadata；
The physics implement plan at its place is cut to obtain corresponding the delaying of each physical tasks for root node with each physical tasks
Deposit calculating track；
Calculate track according to the caching that each physical tasks is corresponding and build cache feature tree in caching main controlled node；
When inquiry request arrives, SQL statement is resolved to physics implement plan by query execution engine；
Each node level traversal physics implement plan is started, it is judged that each physical tasks from the root node of physics implement plan
Corresponding caching calculates whether track mates with described cache feature tree；
If coupling, from node, directly read the caching real data of this physical tasks from caching, otherwise calculate this physical tasks.
The buffer memory management method of distributed memory columnar database the most according to claim 1, it is characterised in that described slow
Deposit each element in queue by weight order arrangement from big to small.
The buffer memory management method of distributed memory columnar database the most according to claim 2, it is characterised in that described slow
Deposit in queue the weight of each element according to Wi=qi×(a×Si+b×Pi) obtain, wherein, WiIt is the weight of i-th element, qi
It is the weight factor of the physical tasks that i-th element is corresponding, SiBe i-th element space-time ratio andtiIt is i-th
The caching that item element is corresponding calculates the time in track to root node longest path, kiIt it is the storage strategy that i-th element is corresponding
Constant, miIt is the system memory space shared by i-th element reality, PiBe i-th element hit frequency andni
It is the historical hit number of times of i-th element, diIt is the time interval of i-th last hit of element distance, viIt is i-th element
Averagely hitting time interval, a is the factor of influence of space-time comparison weight, and b is the hit frequency factor of influence to weight, and i is just
Integer.
The buffer memory management method of distributed memory columnar database the most according to claim 3, it is characterised in that also include
Following steps are carried out when the new storage cached is asked when described cache management system receives:
Step S1, updates the weight of each element in described buffer queue；
Step S2, it is judged that whether described cache management system current residual space be enough to store new caching；
If described cache management system current residual space be enough to store new caching, perform step S3, caching main controlled node notice
Cache from the new caching of node storage, new cache metadata is put into described buffer queue, and record in described cache feature tree
The caching that new caching is corresponding calculates track；
If described cache management system current residual space is not enough to store new caching, perform step S4, it is judged that the power of new caching
Whether weight is more than the weight of last element in described buffer queue；
If the weight of new caching is not more than the weight of last element in described buffer queue, perform step S5, cache master control
Node notice caching is from the node new caching of refusal storage；
If the weight of new caching is more than the weight of last element in described buffer queue, perform step S6, it may be judged whether still
Inquiry operation is had to be currently in use last element in described buffer queue；
If still there being inquiry operation to be currently in use last element in described buffer queue, perform step S7, by described caching team
In row, last rubidium marking is to be deleted；
If being currently in use last element in described buffer queue without inquiry operation, performing step S8, caching main controlled node is deleted
Except last element in described buffer queue, reclaim the memory space that the caching that is eliminated in described cache management system takies,
From described cache feature tree, delete the caching that in described buffer queue, last element is corresponding calculate track, and forward step to
S2。
The buffer memory management method of distributed memory columnar database the most according to claim 4, it is characterised in that also wrap
Include:
For each element in described buffer queue, one corresponding reference count is set, when an element initial hit, should
The reference count that item element is corresponding puts 1, and starts a timer；
If before the timer expires, caching main controlled node receives and uses the SQL statement of an element to inquire about complete feedback, then will
The reference count that this element is corresponding subtracts 1, and resets timer；
When reference count corresponding to element is equal to 0, then close timer.
The buffer memory management method of distributed memory columnar database the most according to claim 5, it is characterised in that judgement is
No still have inquiry operation to be currently in use in described buffer queue last element for judging in described buffer queue last
Whether the reference count that element is corresponding is 0.
The buffer memory management method of distributed memory columnar database the most according to claim 1, it is characterised in that each thing
Corresponding the caching of reason task calculates track and includes that the result of each node calculates time, result storage size and each edge and represents
Data transmission period.
The buffer memory management method of distributed memory columnar database the most according to claim 1, it is characterised in that described slow
Deposit in characteristics tree the table involved by all nodes and be owned by version number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610659223.8A CN106294772B (en) | 2016-08-11 | 2016-08-11 | The buffer memory management method of distributed memory columnar database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610659223.8A CN106294772B (en) | 2016-08-11 | 2016-08-11 | The buffer memory management method of distributed memory columnar database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106294772A true CN106294772A (en) | 2017-01-04 |
| CN106294772B CN106294772B (en) | 2019-03-19 |
Family
ID=57669593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610659223.8A Active CN106294772B (en) | 2016-08-11 | 2016-08-11 | The buffer memory management method of distributed memory columnar database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106294772B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107329814A (en) * | 2017-06-16 | 2017-11-07 | 电子科技大学 | A kind of distributed memory database query engine system based on RDMA |
| CN107784103A (en) * | 2017-10-27 | 2018-03-09 | 北京人大金仓信息技术股份有限公司 | A kind of standard interface of access HDFS distributed memory systems |
| WO2019021124A1 (en) * | 2017-07-27 | 2019-01-31 | International Business Machines Corporation | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| CN109376014A (en) * | 2018-10-19 | 2019-02-22 | 郑州云海信息技术有限公司 | A kind of Distributed Lock Manager method and system |
| CN109492005A (en) * | 2018-11-07 | 2019-03-19 | 郑州云海信息技术有限公司 | A kind of B+ tree read buffer method and relevant apparatus |
| CN110119275A (en) * | 2019-05-13 | 2019-08-13 | 电子科技大学 | A kind of distributed memory columnar database Complied executing device framework |
| CN110162272A (en) * | 2019-05-23 | 2019-08-23 | 北京邮电大学 | A kind of memory calculates buffer memory management method and device |
| CN110389965A (en) * | 2018-11-30 | 2019-10-29 | 上海德拓信息技术股份有限公司 | A kind of optimization method of multidimensional data query and caching |
| US10579532B2 (en) | 2017-08-09 | 2020-03-03 | International Business Machines Corporation | Invalidating track format information for tracks in cache |
| CN112528081A (en) * | 2020-12-24 | 2021-03-19 | 长沙翔宇信息科技有限公司 | Space situation relative motion trajectory multilevel loading display method and device |
| US11036641B2 (en) | 2017-08-09 | 2021-06-15 | International Business Machines Corporation | Invalidating track format information for tracks demoted from cache |
| CN114817262A (en) * | 2022-04-27 | 2022-07-29 | 电子科技大学 | Graph traversal algorithm based on distributed graph database |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177057A (en) * | 2011-12-20 | 2013-06-26 | Sap股份公司 | Many core algorithms for in-memory column store databases |
| CN103970870A (en) * | 2014-05-12 | 2014-08-06 | 华为技术有限公司 | Database query method and server |
-
2016
- 2016-08-11 CN CN201610659223.8A patent/CN106294772B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177057A (en) * | 2011-12-20 | 2013-06-26 | Sap股份公司 | Many core algorithms for in-memory column store databases |
| CN103970870A (en) * | 2014-05-12 | 2014-08-06 | 华为技术有限公司 | Database query method and server |
Non-Patent Citations (3)
| Title |
|---|
| 张琦: "DWMS列存储中执行引擎的优化与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 徐芳芳: "基于语义缓存的RDF数据查询优化", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 王竹峰: "达梦嵌入式数据库的执行计划缓存研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107329814A (en) * | 2017-06-16 | 2017-11-07 | 电子科技大学 | A kind of distributed memory database query engine system based on RDMA |
| CN107329814B (en) * | 2017-06-16 | 2020-05-26 | 电子科技大学 | RDMA (remote direct memory Access) -based distributed memory database query engine system |
| GB2578080A (en) * | 2017-07-27 | 2020-04-15 | Ibm | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| US11704209B2 (en) | 2017-07-27 | 2023-07-18 | International Business Machines Corporation | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| WO2019021124A1 (en) * | 2017-07-27 | 2019-01-31 | International Business Machines Corporation | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| US11263097B2 (en) | 2017-07-27 | 2022-03-01 | International Business Machines Corporation | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| GB2578080B (en) * | 2017-07-27 | 2020-09-09 | Ibm | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| US10691566B2 (en) | 2017-07-27 | 2020-06-23 | International Business Machines Corporation | Using a track format code in a cache control block for a track in a cache to process read and write requests to the track in the cache |
| US11036641B2 (en) | 2017-08-09 | 2021-06-15 | International Business Machines Corporation | Invalidating track format information for tracks demoted from cache |
| US10579532B2 (en) | 2017-08-09 | 2020-03-03 | International Business Machines Corporation | Invalidating track format information for tracks in cache |
| US11086784B2 (en) | 2017-08-09 | 2021-08-10 | International Business Machines Corporation | Invalidating track format information for tracks in cache |
| CN107784103A (en) * | 2017-10-27 | 2018-03-09 | 北京人大金仓信息技术股份有限公司 | A kind of standard interface of access HDFS distributed memory systems |
| CN109376014A (en) * | 2018-10-19 | 2019-02-22 | 郑州云海信息技术有限公司 | A kind of Distributed Lock Manager method and system |
| CN109492005A (en) * | 2018-11-07 | 2019-03-19 | 郑州云海信息技术有限公司 | A kind of B+ tree read buffer method and relevant apparatus |
| CN110389965A (en) * | 2018-11-30 | 2019-10-29 | 上海德拓信息技术股份有限公司 | A kind of optimization method of multidimensional data query and caching |
| CN110389965B (en) * | 2018-11-30 | 2023-03-14 | 上海德拓信息技术股份有限公司 | Multidimensional data query and cache optimization method |
| CN110119275A (en) * | 2019-05-13 | 2019-08-13 | 电子科技大学 | A kind of distributed memory columnar database Complied executing device framework |
| CN110119275B (en) * | 2019-05-13 | 2021-04-02 | 电子科技大学 | Distributed memory column type database compiling executor architecture |
| CN110162272A (en) * | 2019-05-23 | 2019-08-23 | 北京邮电大学 | A kind of memory calculates buffer memory management method and device |
| CN112528081A (en) * | 2020-12-24 | 2021-03-19 | 长沙翔宇信息科技有限公司 | Space situation relative motion trajectory multilevel loading display method and device |
| CN114817262A (en) * | 2022-04-27 | 2022-07-29 | 电子科技大学 | Graph traversal algorithm based on distributed graph database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106294772B (en) | 2019-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106294772A (en) | The buffer memory management method of distributed memory columnar database | |
| Huang et al. | X-Engine: An optimized storage engine for large-scale E-commerce transaction processing | |
| CN103678556B (en) | The method and processing equipment of columnar database processing | |
| CN102521406B (en) | Distributed query method and system for complex task of querying massive structured data | |
| CN102521405B (en) | Massive structured data storage and query methods and systems supporting high-speed loading | |
| Olken et al. | Random sampling from databases: a survey | |
| US9514211B2 (en) | High throughput data modifications using blind update operations | |
| US8380702B2 (en) | Loading an index with minimal effect on availability of applications using the corresponding table | |
| US8566300B2 (en) | Mechanism for efficient maintenance of XML index structures in a database system | |
| US10754854B2 (en) | Consistent query of local indexes | |
| US20110035369A1 (en) | Query Optimization with Memory I/O Awareness | |
| CN109656958B (en) | Data query method and system | |
| US9576038B1 (en) | Consistent query of local indexes | |
| CN105373541B (en) | The processing method and system of the data operation request of database | |
| US20170109386A1 (en) | Create table for exchange | |
| US20120203745A1 (en) | System and method for range search over distributive storage systems | |
| CN107247778A (en) | System and method for implementing expansible data storage service | |
| CN105574054B (en) | A kind of distributed caching range query method, apparatus and system | |
| CN107491487A (en) | A kind of full-text database framework and bitmap index establishment, data query method, server and medium | |
| CN102955792A (en) | Method for implementing transaction processing for real-time full-text search engine | |
| CN109313640A (en) | Method and system for data base optimization | |
| CN111639075A (en) | Non-relational database vector data management method based on flattened R tree | |
| CN106354732A (en) | Offline data version conflict resolution method for supporting concurrent cooperation | |
| CN106161193B (en) | Mail processing method, device and system | |
| CN106484694A (en) | Full-text search method based on distributed data base and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |