CN103207897A - Distributed storage query system, operation method thereof and operation device - Google Patents
Distributed storage query system, operation method thereof and operation device Download PDFInfo
- Publication number
- CN103207897A CN103207897A CN2013100841869A CN201310084186A CN103207897A CN 103207897 A CN103207897 A CN 103207897A CN 2013100841869 A CN2013100841869 A CN 2013100841869A CN 201310084186 A CN201310084186 A CN 201310084186A CN 103207897 A CN103207897 A CN 103207897A
- Authority
- CN
- China
- Prior art keywords
- node
- query request
- data
- data query
- distributed storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a distributed storage query system based on DMM (distributed match making), an operation method thereof and an operation device. Destruction resistance and query delay of data can be simultaneously considered, the distributed storage query system based on a DMM mode meets the condition of |P(i)PiB(j)|>=k, the i and the j indicate optional nodes in the system, the k indicates a preset integer positioned within the interval of [1, N-1], the N indicates the total of nodes of the system, the P(i) indicates a set of all nodes backing up data in the node i, B(i) meets the relationship of Q(i)<B(i), and the Q(i) indicates a set of queried nodes when query is issued from the node i if j<B(i).
Description
Technical field
The present invention relates to a kind of distributed storage inquiry system and operation method and running gear.
Background technology
In current disparate networks, inquiry system is seen everywhere, as the DNS system among the Internet, the Database Systems of bank, large enterprise etc. etc.These inquiry system majorities all are the integrated systems of data center.But the increase along with data volume, deposit become all the more difficulty of all data at a data center, and in a data center all data of the side of depositing and be unfavorable for that the anti-of data ruin, in case loss difficult to the appraisal will appear in data failure, so distributed memory system occurred.
Present distributed memory system mostly is based on distributed hashtable DHT's, and these inquiry systems all are subjected to the restriction (as Chord ring-like) of fixed topology constraint, so the performance of inquiry system affects.In being based on distributed coupling DMM(Distributed match making) the storing queries system arise at the historic moment.
The core concept of DMM technology is: preserve two node set P (i) and Q (i) in each node i.P (i) set is all set of having backed up the node of the data in the node i, and namely each node among the P (i) has all backed up the data in the node i, and for example P (6)={ 1,2,3} represents to have backed up on the node 1,2,3 data of node 6.The set that is queried node when Q (i) expression is initiated inquiry from the i node, namely the node that inquires the Q (i) of initiating from the i node is inquired about.For example Q (5)={ 2,3,4} represents if node 5 when initiating a content search request, and its can initiate the data of query requests in the hope of obtaining to want to node 2,3,4 simultaneously.Obviously, if for i and j arbitrarily, it is 1 that the union size of P (i) and Q (j) is arranged, and then all query requests can both be met.
Also having a concept in the DMM technology is the R set, and the set of the data of preserving in this set expression node i is represented with R (i) usually.With P set and Q set be node set different be that the R set is data acquisition.When node i is received a query requests, from R (i), search whether requested data are arranged, reply to source node if exist then from the database of oneself, choose corresponding data, otherwise continuation is carried out the query requests distribution to the node among the Q (i).For each node i, owing to be subject to the restriction of node storage capacity, the R set R (i) of node i should satisfy | R (i) | and≤r (i), wherein r (i) is the memory limitations of node i.
Though the DMM scheme has been utilized data backup on the one hand, just data backup is used to shorten the inquiry time delay, do not have the real anti-protection of ruining that realizes.Though this scheme can be better than classical architecture such as Chord in the inquiry time delay finishing inquiry once jumping virtual route on the other hand, an obvious problem that exists is that the memory space that needs is too big.Then can cause to a lot of nodes and propose query requests simultaneously if reduce memory space, make that the query load of each node is very heavy, also cause congested in the network simultaneously easily.
Summary of the invention
In view of this, the invention provides a kind of distributed storage inquiry system based on DMM and operation method and running gear, that can take into account data anti-ly ruins and inquires about time delay.
For achieving the above object, according to an aspect of the present invention, provide a kind of distributed storage inquiry system.
Distributed storage inquiry system of the present invention is based on distributed coupling (DMM) mode, and this system satisfies following condition: | P (i) ∩ B (j) | 〉=k; Wherein, i and j represent the arbitrary node in the described system, and k represents to preset is positioned at integer in the interval [1, N-1], and N represents the node sum of described system; P (i) represents that all have backed up the set of the node of the data in the node i; B (i) satisfies following relation:
And if j ∈ B (i), then
The set that is queried node when wherein Q (i) expression is initiated inquiry from the i node.
A kind of operation method of distributed storage inquiry system is provided according to a further aspect in the invention.
The operation method of distributed storage inquiry system of the present invention comprises: receive the data query request of request source transmission at node after, judge whether preserve requested data in this node; If then requested data are sent to the described request source; If not, then the node in the described Q set of this node sends the data query request.
Alternatively, preserve in described node under the situation of requested data, described method also comprises: the data query request that deletion described request source sends.
Alternatively, in described node, do not preserve under the situation of requested data, described method also comprises: the current life span of described data query request is subtracted 1, and reach at 0 o'clock in the current life span of described data query request and delete described data query request; Described life span is a positive integer that arranges when described data query request is sent for the first time.
Alternatively, described life span is set to the node number of described distributed storage inquiry system.
A kind of running gear of distributed storage inquiry system is provided according to another aspect of the invention.
The running gear of distributed storage inquiry system of the present invention comprises: receiver module is used for receiving the data query request that request source sends to node; Judge module is used for judging whether preserve requested data in this node after described receiver module receives described data query request; Sending module, the judged result that is used at described judge module is under the situation that is requested data to be sent to the described request source, this judged result for situation not under node in the described Q set of this node send the data query request.
Alternatively, also comprise removing module, the judged result that is used at described judge module is under the situation that is, the data query request that deletion described request source sends.
Alternatively, also comprise the existence module, be used for judged result at described judge module for situation not under, the current life span of described data query request is subtracted 1, and reaches at 0 o'clock in the current life span of described data query request and delete described data query request; Described life span is a positive integer that arranges when described data query request is sent for the first time.
According to technical scheme of the present invention, introduce the anti-requirement k that ruins of B set and data, satisfy with the data on all nodes in the assurance system by data backup | P (i) ∩ B (j) | 〉=k, can realize that data resist the compromise of ruining with the data backup amount by regulating k value and backup amount thus, thus can take into account data anti-ly ruin and inquire about time delay.
Description of drawings
Accompanying drawing is used for understanding the present invention better, does not constitute improper restriction of the present invention.Wherein:
Fig. 1 is the synoptic diagram according to the basic step of the operation method of the distributed storage inquiry system of the embodiment of the invention;
Fig. 2 is the synoptic diagram according to the basic structure of the running gear of the distributed storage inquiry system of the embodiment of the invention.
Embodiment
Below in conjunction with accompanying drawing one exemplary embodiment of the present invention is explained, to help understanding, they should be thought it only is exemplary comprising the various details of the embodiment of the invention.Therefore, those of ordinary skills will be appreciated that, can make various changes and modification to the embodiments described herein, and can not deviate from scope and spirit of the present invention.Equally, for clarity and conciseness, omitted the description to known function and structure in the following description.
Anti-ly ruin and inquire about time delay for what take into account data, in the present embodiment, based on the DMM technology, introduce B set and data and resist and ruins requirement k.The B set refers to, in the distributed storage inquiry system based on DMM, from the i node, constantly transmit to the interior node of follow-up Q set, be to continue to transmit this query requests to the Q set of self after the node of Q set is received query requests, the node that finally can arrive namely constitutes set B (i) by this way.Be that B (i) satisfies following relation:
And if j ∈ B (i), then
The anti-requirement k that ruins of data refers to that all data queries still can normally be finished under the situation that k-1 node data inefficacy occur.This just requires said system to satisfy | P (i) ∩ B (j) | and 〉=k.I and j represent the arbitrary node in the above-mentioned distributed storage inquiry system.
Before the distributed storage inquiry system operation based on DMM in the present embodiment, the data backup of setting up inquiry system is earlier satisfied the data on all nodes in this system | P (i) ∩ B (j) | and 〉=k.The k here is a predefined integer, and the actual conditions that its size can viewing system are set and adjusted.When k got smaller value, inquiry will be passed through more jumping figure, and therefore time-delay is longer, but desired data backup amount is less; Otherwise when k gets higher value, thereby inquiry can be shorter through less jumping figure time-delay, but need bigger data volume.No matter how many k gets, as long as | P (i) ∩ B (j) | 〉=k, just can realize that the anti-of data ruin.Therefore according to the technical scheme of present embodiment, can be by regulating k value and carrying out data backup with assurance | P (i) ∩ B (j) | 〉=k realizes the compromise of inquiry time delay and data backup amount, and that has namely taken into account data anti-ly ruins and inquire about time delay.
Fig. 1 is the synoptic diagram according to the basic step of the operation method of the distributed storage inquiry system of the embodiment of the invention.In the method, the execution flow process of any one node i as shown in Figure 1.
Step S11: node i receives the query requests that request source sends.
In the R set of step S12: decision node i whether requested data are arranged.If then enter step S13, otherwise enter step S14.
Step S13: requested data are sent to request source.
Step S14: the node in the Q set of node i sends the data query request.Ensuing processing mode is with similar from the flow process of step S11, and just node i is replaced by the node in its Q set, and concerning these nodes, node i is exactly their request source.
Because in the distributed storage inquiry system, can constantly inquire about, so constantly accumulation of the query requests that each node is received.Take the too much storage space of node for fear of query requests, can Delete query request under the situation of requested data be arranged at node.A life span can also be set in addition, life span is a positive integer that arranges when the data query request is sent for the first time, be included in the query requests, node is when the forwarding inquiries request, life span in this query requests is subtracted 1, and node is the request of 0 o'clock deletion data query in life span.For fear of also not finding data, life span has been reduced to 0, the life span initial value can be made as the node number of distributed storage inquiry system.
Fig. 2 is the synoptic diagram according to the basic structure of the running gear of the distributed storage inquiry system of the embodiment of the invention.As shown in Figure 2, the running gear 20 of distributed storage inquiry system mainly comprises receiver module 21, judge module 22 and sending module 23.
The running gear 20 of distributed storage inquiry system shown in Figure 2 can also comprise the removing module (not shown), and the judged result that is used at judge module 22 is under the situation that is, the data query request that the removal request source sends.
The running gear 20 of distributed storage inquiry system can also comprise existence module (not shown), the judged result that is used at judge module 22 is under the situation not, the life span that the data query request is current subtracts 1, and reaches at 0 o'clock in the current life span of data query request and delete this data query request.
According to technical scheme of the present invention, introduce the anti-requirement k that ruins of B set and data, satisfy with the data on all nodes in the assurance system by data backup | P (i) ∩ B (j) | 〉=k, can realize that data resist the compromise of ruining with the data backup amount by regulating k value and backup amount thus, thus can take into account data anti-ly ruin and inquire about time delay.
Ultimate principle of the present invention has below been described in conjunction with specific embodiments, but, it is to be noted, for those of ordinary skill in the art, can understand whole or any steps or the parts of method and apparatus of the present invention, can be in the network of any calculation element (comprising processor, storage medium etc.) or calculation element, realized that with hardware, firmware, software or their combination this is that those of ordinary skills use their basic programming skill just can realize under the situation of having read explanation of the present invention.
Therefore, purpose of the present invention can also be by realizing in any program of calculation element operation or batch processing.Described calculation element can be known fexible unit.Therefore, purpose of the present invention also can be only by providing the program product that comprises the program code of realizing described method or device to realize.That is to say that such program product also constitutes the present invention, and the storage medium that stores such program product also constitutes the present invention.Obviously, described storage medium can be any storage medium of developing in any known storage medium or future.
It is pointed out that also that in apparatus and method of the present invention obviously, each parts or each step can decompose and/or reconfigure.These decomposition and/or reconfigure and to be considered as equivalents of the present invention.And, carry out the step of above-mentioned series of processes and can order following the instructions naturally carry out in chronological order, but do not need necessarily to carry out according to time sequencing.Some step can walk abreast or carry out independently of one another.
Above-mentioned embodiment does not constitute limiting the scope of the invention.Those skilled in the art should be understood that, depend on designing requirement and other factors, and various modifications, combination, sub-portfolio and alternative can take place.Any modification of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., all should be included within the protection domain of the present invention.
Claims (8)
1. distributed storage inquiry system, described system is characterized in that based on distributed coupling (DMM) mode this system satisfies following condition:
| P (i) ∩ B (j) | 〉=k; Wherein,
I and j represent the arbitrary node in the described system, and k represents to preset is positioned at integer in the interval [1, N-1], and N represents the node sum of described system;
P (i) represents that all have backed up the set of the node of the data in the node i;
B (i) satisfies following relation:
And if j ∈ B (i), then
The set that is queried node when wherein Q (i) expression is initiated inquiry from the i node.
2. the operation method of a distributed storage inquiry system, described distributed storage inquiry system is the described distributed storage inquiry system of claim 1, it is characterized in that this method comprises:
Receive the data query request of request source transmission at node after, judge whether preserve requested data in this node;
If then requested data are sent to the described request source;
If not, then the node in the described Q set of this node sends the data query request.
3. operation method according to claim 2 is characterized in that, preserves in described node under the situation of requested data, and described method also comprises: the data query request that deletion described request source sends.
4. according to claim 2 or 3 described operation methods, it is characterized in that,
Do not preserve in described node under the situation of requested data, described method also comprises: the current life span of described data query request is subtracted 1, and reach at 0 o'clock in the current life span of described data query request and delete described data query request;
Described life span is a positive integer that arranges when described data query request is sent for the first time.
5. operation method according to claim 4 is characterized in that, described life span is set to the node number of described distributed storage inquiry system.
6. the running gear of a distributed storage inquiry system, described distributed storage inquiry system is the described distributed storage inquiry system of claim 1, it is characterized in that this device comprises:
Receiver module is used for receiving the data query request that request source sends to node;
Judge module is used for judging whether preserve requested data in this node after described receiver module receives described data query request;
Sending module, the judged result that is used at described judge module is under the situation that is requested data to be sent to the described request source, this judged result for situation not under node in the described Q set of this node send the data query request.
7. running gear according to claim 6 is characterized in that, also comprises removing module, and the judged result that is used at described judge module is under the situation that is, the data query request that deletion described request source sends.
8. according to claim 6 or 7 described running gears, it is characterized in that, also comprise the existence module, the judged result that is used at described judge module is under the situation not, the current life span of described data query request is subtracted 1, and reach at 0 o'clock in the current life span of described data query request and delete described data query request;
Described life span is a positive integer that arranges when described data query request is sent for the first time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310084186.9A CN103207897B (en) | 2013-03-15 | 2013-03-15 | A kind of distributed storage inquiry system and operation method thereof and running gear |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310084186.9A CN103207897B (en) | 2013-03-15 | 2013-03-15 | A kind of distributed storage inquiry system and operation method thereof and running gear |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103207897A true CN103207897A (en) | 2013-07-17 |
| CN103207897B CN103207897B (en) | 2016-08-17 |
Family
ID=48755119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310084186.9A Active CN103207897B (en) | 2013-03-15 | 2013-03-15 | A kind of distributed storage inquiry system and operation method thereof and running gear |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103207897B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019052075A1 (en) * | 2017-09-18 | 2019-03-21 | 中国科学院声学研究所 | Query enhancement system and method for constructing elastic field based on time delay |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158531A (en) * | 2010-02-01 | 2011-08-17 | 北京大学 | Distributed transmission method for query data stream |
| CN102170422A (en) * | 2010-02-25 | 2011-08-31 | 腾讯科技（深圳）有限公司 | Method, device and system for inquiring node |
| US20120084383A1 (en) * | 2010-04-23 | 2012-04-05 | Ilt Innovations Ab | Distributed Data Storage |
-
2013
- 2013-03-15 CN CN201310084186.9A patent/CN103207897B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158531A (en) * | 2010-02-01 | 2011-08-17 | 北京大学 | Distributed transmission method for query data stream |
| CN102170422A (en) * | 2010-02-25 | 2011-08-31 | 腾讯科技（深圳）有限公司 | Method, device and system for inquiring node |
| US20120084383A1 (en) * | 2010-04-23 | 2012-04-05 | Ilt Innovations Ab | Distributed Data Storage |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019052075A1 (en) * | 2017-09-18 | 2019-03-21 | 中国科学院声学研究所 | Query enhancement system and method for constructing elastic field based on time delay |
| JP2020533922A (en) * | 2017-09-18 | 2020-11-19 | 中国科学院声学研究所Ｉｎｓｔｉｔｕｔｅ Ｏｆ Ａｃｏｕｓｔｉｃｓ， Ｃｈｉｎｅｓｅ Ａｃａｄｅｍｙ Ｏｆ Ｓｃｉｅｎｃｅｓ | Query extension system and query extension method |
| RU2760243C2 (en) * | 2017-09-18 | 2021-11-23 | Инститьют Оф Акустикс, Чайниз Акэдэми Оф Сайнсиз | System and method for improving request for creating distributed group of resources based on time delay |
| JP6992168B2 (en) | 2017-09-18 | 2022-01-13 | 中国科学院声学研究所 | Query extension system and query extension method |
| US11386103B2 (en) | 2017-09-18 | 2022-07-12 | Institute Of Acoustics, Chinese Academy Of Sciences | Query enhancement system and method for constructing elastic field based on time delay |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103207897B (en) | 2016-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7428220B1 (en) | Method and apparatus for data aggregation in a distributed system | |
| US8495013B2 (en) | Distributed storage system and method for storing objects based on locations | |
| US10257255B2 (en) | Hierarchical organization for scale-out cluster | |
| US9330177B2 (en) | System, method and device for internet search based on peer-to-peer network | |
| EP2577517B1 (en) | Scalable policy-based database synchronization of scopes | |
| US7974218B2 (en) | Intersection-based configuration management | |
| WO2009097438A2 (en) | Query deployment plan for a distributed shared stream processing system | |
| US9830091B2 (en) | Policy-based data tiering using a cloud architecture | |
| US20150312342A1 (en) | Parallel i/o write processing for use in clustered file systems having cache storage | |
| US9081839B2 (en) | Push replication for use with a distributed data grid | |
| CN109684282A (en) | A kind of method and device constructing metadata cache | |
| CN105096174A (en) | Transaction matching method and transaction matching system | |
| CN110990448B (en) | Distributed query method and device supporting fault tolerance | |
| CN100505662C (en) | Method for setting point-to-point data channel | |
| CN101963978B (en) | Distributed database management method, device and system | |
| US10817512B2 (en) | Standing queries in memory | |
| EP2415213B1 (en) | Smart routing | |
| CN102404372A (en) | Method, system and node device for storing content in WEB cache in distributed mode | |
| US10484467B2 (en) | Dynamic swarm segmentation | |
| CN103207897A (en) | Distributed storage query system, operation method thereof and operation device | |
| CN112860799A (en) | Management method for data synchronization of distributed database | |
| EP1936497A2 (en) | Automated wide area software distribution with reduced network bandwidth requirements | |
| CN106649584A (en) | Index processing method and device in master-slave database system | |
| CN106657333B (en) | Centralized directory data exchange system and method based on cloud service mode | |
| US10348828B2 (en) | Method and apparatus for optimizing data transfers utilizing machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |