CN102572602A - Distributed index achieving method in peer-to-peer (P2P) live stream system based on distributed hash table (DHT) - Google Patents
Distributed index achieving method in peer-to-peer (P2P) live stream system based on distributed hash table (DHT) Download PDFInfo
- Publication number
- CN102572602A CN102572602A CN2012100119134A CN201210011913A CN102572602A CN 102572602 A CN102572602 A CN 102572602A CN 2012100119134 A CN2012100119134 A CN 2012100119134A CN 201210011913 A CN201210011913 A CN 201210011913A CN 102572602 A CN102572602 A CN 102572602A
- Authority
- CN
- China
- Prior art keywords
- index
- node
- data block
- distributed
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Computer And Data Communications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a distributed index achieving method in a peer-to-peer (P2P) live stream system based on a distributed hash table (DHT), which can reduce system overhead to the greatest extent without reducing inquiring efficiency and index effectiveness. By means of the distributed index achieving method in the P2P live stream system based on the DHT, resource identification keys generated by the same data block are different along with difference of subspace identifiers of source nodes. Thus, indexes of the same data block are stored on the different appointed nodes, single-point failure is avoided, and the index effectiveness is guaranteed. Furthermore, due to the fact that a duplication mechanism is not adopted, index values of the same data block stored on different nodes are different, thereby avoiding network storage and maintaining indexes with redundancy and reducing system overhead.
Description
Technical field
The present invention relates to resource index mapping techniques in the P2P stream media system, relate to the discrete Hash table technology of DHT (Distributed Hash Table) simultaneously.
Background technology
In P2P (Peer to Peer) live broadcast stream media system, play all nodes of same video program and form an overlay network (Overlay Network).The video flowing server is divided into one by one little data block (Chunk) with video source, and each data block is being represented certain-length ground stream medium data.It is the data sharing service of unit that node in the nerve of a covering can provide with the data block each other, reduces the dependence and the purpose that improve the system expandability of stream media system for server thereby reach.Present stage, the overlay network framework of its composition was pressed by most of P2P live broadcast stream media system, roughly can be divided into 2 types: based on the system of tree type (Tree) and netted (Mesh) structure.Under tree, node is organized into the network of Dan Shu (Single Tree) or many trees (Multi-tree) usually.The root node of tree is video flowing server place, and data block is to push (Push) with father node downwards to the order of child node from level to level.Under network structure, each node maintenance some ground neighbor node, these neighbor nodes logically directly link to each other with own.Learn through periodically exchanging BM (Buffer Map) information whether the data that oneself need are arranged on other node between neighbor node.According to these BM information, node selects for use the propelling movement or the mode of (Pull) of drawing to obtain data from other node again.Yet all there is certain limitation in the system under above-mentioned these two kinds of network architectures, and for example the former is very sensitive and have the problem do not make full use of Internet resources to network jitter, and there are problems such as high time delay and high information exchange expense in the latter.Given this, up to the present also exist another kind of research and take advantage of a favourable situation, be about to the DHT technology and be incorporated into the P2P stream media system.
DHT (Distributed Hash Table) is called discrete Hash table technology again; Its principle is: each node and resource obtain an ID through Hash (HASH) and identify oneself in the P2P network; The ID of node basis oneself is responsible for a part of ID space, and resource is mapped to node corresponding according to the ID of oneself; A specific size of node maintenance ground routing table is carried out selective forwarding through routing table during locating resource, can guarantee what resource the location takes office in the jumping figure of confirming.DHT is the basis of many Distributed Application, for example in the P2P stream media system, utilizes DHT to carry out the distributed earth index stores and search usually.
In the P2P stream media system of introducing the DHT technology, each user shows as a node in the P2P network, and each data block shows as a resource (consider the expense correlative factor, or show as a resource by a plurality of consecutive data block) in network.The user obtains a sign ID according to oneself IP address or out of Memory through Hash, and data block also obtains a sign ID according to numbering or out of Memory through Hash.For the ease of distinguishing, we are designated as Key in addition with the sign ID of resource (data block).One < Key, the Value>that so-called index refers to is right, and the Key here can identify Key with data block and is equal under the current application scene.When certain user received a full block of data, it just can produce an index for it.Wherein, the Key value is obtained through Hash by data block, and Value has comprised relevant informations such as IP address.Again by the mapping relations between resource and the node, the user sends index and store into by its Key value correspondingly on the node.This process is in the P2P stream media system based on the distribution index of DHT and uses.As for searching, the data block that the user searches as required calculates the Key value, finds correspondingly node through mapping relations again, is storing the index relevant with this data block on this node.Read in these index Value information at last and learn the data that this data block is arranged on which user.
In traditional index mapping and management method, identical data block will obtain identifying the Key value identically, thereby those all will be stored on the same node by mapping to the index that identical block produced.As shown in Figure 1, L
NRepresent a certain particular data block number, K20 representes that the value of data block sign Key is 20, and N20 representes that the value of node identification ID is 20; On N79, N102, three nodes of N150 data block L is arranged all
N, the resource identification of generation is K20, and three about data block L
NIndex all will be stored on the node N38.If node N38 lost efficacy, the search request of all relevant data block all can't meet with a response.Therefore, for fear of single point failure and improve index validity, node usually can be with all index copy k parts of its storage to k node subsequently, and this is replicanism (Replication Mechanism).Because the generation and the issue of index are to be unit with the data block, data block data represented length is very limited usually, even if do not adopt the network of replicanism also can have index in large quantities.After adopting replicanism to improve index validity; Become k doubly to increase index quantity again; Bring expense extraly, and, because user's play position is close in the live broadcast system; Same data block all exists at many nodes, and is also just corresponding more to the index that has redundant character that this data block produced.Actual situation is that we do not need the so many index that has redundant character of storage and maintenance.
Summary of the invention
Technical problem to be solved by this invention is, provides a kind of in P2P live TV stream system, do not reduce under the prerequisite of search efficiency, index validity, reduces the distributed index implementation method based on DHT of overhead as far as possible.
The present invention solves the problems of the technologies described above the technical scheme that is adopted to be, in P2P live TV stream system, based on the distributed index implementation method of DHT, it is characterized in that, may further comprise the steps:
Each node division in the network to corresponding subspace, and is distributed the subspace identifier SID of corresponding subspace for each node;
When node need send the index of data block as source node; This node generates resource identification Key according to the sid value of data block information and source node; And produce the index < Key, Value>of this data block, wherein Key is for generating resource identification; Value is an index value, and index value includes source node IP address; Afterwards, the mapping relations of this node resource identification Key and specified node according to a preconcerted arrangement are sent to specified node with said index and store;
When node need be searched data block; This node searches as required that all subspace identifier SID obtain different resources sign Key in data block information and the network of data block; Thereby the resource identification Key according to a preconcerted arrangement and the mapping relations of specified node; In all specified nodes, search, thereby can obtain all corresponding index of this data block in the network.
The resource identification Key that the present invention generates for same data block can be different because of the subspace identifier SID of source node.Like this; The index of same data block can be stored on the different specified nodes, can avoid single point failure, guarantees index validity; And owing to do not adopt replicanism; Therefore the index value of the same data block store of storage has nothing in common with each other on the different nodes, has avoided the network storage and has safeguarded the index that has redundant character, reduces overhead.
Further, the concrete mode with each node division in the network to corresponding subspace is: at first confirm all nodes in the network to be divided in each subspace the subspace number equably.Because the uniformity that the subspace is divided, when number of nodes is not seldom the time, can think that every kind of value of Key also is uniformly, more help improving the probability that the index of different pieces of information piece is stored on different specified nodes, guarantee index validity.
If will be referred to as similar index, then consider to let node in the process of index stores, can limit to the index entry number in every group of similar index to the index that identical block produces.Reduce the index total amount in the system through the mode of restricted part index stores, reduced storage and maintenance costs relatively.The proposition of index upgrade mode on the other hand, make the index stored in the system always effectively and performance more excellent.
Further, also comprise the life cycle of this index in the said index value, the life cycle that the index of the local storage of the node in the network is corresponding arrives after date, and node is removed the index that expires automatically.
Further, also comprise the source node scoring of sending this index in the index value; The scoring of said source node is used to assess the source node of index and the ability of service is provided for other node;
Node in the network judged earlier that whether the corresponding memory space of same asset sign Key reaches the upper limit, as not, directly added before the new index that receives of storage; In this way; Then the minimum index of source node scoring compares in the index of the source node of the new index that receives scoring earlier and current storage same asset sign Key, as greater than, then substitute the minimum index of source node scoring of current storage with the new index that receives; Otherwise, abandon the index of new reception.
Description of drawings
Fig. 1 is the sketch map of existing index mapping and index issue;
Fig. 2 is the division sketch map of Hash subspace among the embodiment;
Fig. 3 is that the distributed index mapping mechanism of embodiment is directed against data block L down
NIndex issue sketch map;
Fig. 4 is to be the influence of subspace division numbers to index validity in the system.
Embodiment
The present embodiment method comprises:
1) particular content of distributed earth index mapping mechanism comprises:
1.1 introduce a sub spaces identifier SID (Sub-space Identifier), each node is confirming also will to produce a correspondingly SID behind its sign ID;
1.2 the producing method of change existing resource (data block) sign Key makes Key not only relevant with data block information, also the sid value with data block place node is relevant;
1.3 index is issued according to new Key value, same data block possibly produce several different new Key values, the index of such data block with distributed store on a plurality of nodes of the whole network.
Above-mentioned method, wherein, total subspace number is defined as i, subspace identifier SID ∈ [1, i] then, and the account form of SID is:
ID＝Hash(IP) (1.2)
In following formula (1.1)～(1.2), the sign ID of node obtains through Hash operation through the IP address, and m is the figure place in Hash space, and whole Hash space is divided into the i sub spaces.Be subordinated to which Hash subspace through decision node ID value and obtain sid value particularly.Formula (1.1) only is the expression formula of a uniform distribution node and each subspace, and those skilled in the art can estimate concrete condition and needs, alternatively provides to carry out other formula and calculate sid value.Certainly, also can be according to the node non-uniform Distribution, other mode that meets the current network present situation is calculated sid value.
Above-mentioned method, wherein, the account form that resource identification Key is new is:
Key＝Hash(L
N*SID) (1.3)
In following formula (1.3), L
NThe numbering of expression resource (data block), SID is the subspace identifier of data block place node, and Hash () representes hash function, and existing hash function commonly used all is applicable to the present invention.Thereby for being numbered L
NIdentical block, its resource identification Key the institute might value be Key ∈ { Hash (L
N* 1) ..., Hash (L
N* i) }.This new resource identification Key account form, will bring following benefit:
(1) make identical block no longer only produce unique Key value, only Key value means that the index management node after the mapping also is identical.Under the prerequisite of not using replicanism, identical management node means to have only a node to be responsible for safeguarding this data block relative index in the whole network, this means the situation that great risk generation index is lost entirely.
(2) the sign ID when data block place node is when belonging to same subspace, and the SID of these nodes is identical, so they are to L
NThe Key value that produces is also identical.Because the uniformity that the subspace is divided, when number of nodes is not seldom the time, we can think that every kind of Key possibly value also be uniform.
2) the index implementation method based on above-mentioned new distributed earth index mapping mechanism is following:
When node is that local data block to be released generates Key value and generation index < Key, Value>afterwards, according to the Key in the index, the Key by appointment and the mapping relations of specified node are sent to specified node with the index of each data block and store.
When node need be searched data block; This node searches as required that all subspace identifier SID obtain different resources sign Key in data block information and the network of data block, is 4 such as the subspace number, and then identifier SID in subspace has 4; Just can calculate 4 different Key; Thereby the resource identification Key according to a preconcerted arrangement and the mapping relations of specified node are searched in 4 all specified nodes, obtain all corresponding index of this data block in the network.
Can find out that the enforcement of said method will have following characteristics:
(1) with index management traditionally use different, in the network to the storage and maintenance of the index of identical block with relatively being evenly distributed on maximum i node, rather than concentrate on a certain node traditionally.
(2) mode of this distributed index storage is similar but be different from replicanism, similarly is the validity that can both avoid single point failure ground problem and improve data (or index); Difference is that it is to rely in a large number extraly replication overhead and storage overhead to realize that the latter improves validity, and the former need spend expense extraly hardly.
3) present embodiment has provided a kind of index stores and update strategy:
On index stores, data block and index enormous amount are based on a distinguishing feature in the P2P stream media system of DHT.If will be referred to as similar index to the index that identical block produces, then node can limit the index entry number in every group of similar index in the process of index stores.
When generating index, meeting interpolation life cycle (TTL), node scoring information such as (Score) are used for the needs of index upgrade in the Value item.
Above-mentioned method, wherein, TTL is used to judge effective time-to-live of this index representative data piece, when the storage duration of index is then removed above TTL automatically.
Above-mentioned method, wherein, Score is used to assess the source node of index and the ability (the for example size of remaining bandwidth) of service is provided for other node.
Further, node judged whether to reach the storage cap of such index earlier before the new index of storage.If do not reach the upper limit, can directly add; If reach, then earlier judge whether the relatively poor index entry that can substitute according to the Score attribute.
Further, above-mentioned strategy design also has following characteristics: reduced the index total amount in the system through the mode of restricted part index stores, reduced storage and maintenance costs relatively.The proposition of index upgrade mode on the other hand, make the index stored in the system always effectively and performance more excellent.
Embodiment
According to the needs of system, at first to specify the value of an i to data validity.In Fig. 2, be example with i=4, suppose whole Hash space for [0,180), then 4 sub spaces scopes be respectively [0,45), [45,90), [90,135), [135,180).The node of SID=1 has N19, N38; The node of SID=2 has N79, N85; The node of SID=3 has N102, N126; The node of SID=4 has N150, N168.
With node N79 is example, and its sid value is 2.Shown in accompanying drawing 3, when N79 receives a complete L that is numbered
NData block the time, the sign Key=Hash (L of this data block
N* SID)=Hash (L
N* 2)=K35.Node N79 can produce an index (Index) for this data block simultaneously, and its content format is:
Index：<Key，Value(IP，L
N，TTL，Score，...)>
Can find out that from general DHT algorithm mapping ruler K35 should be responsible for by node N38.Can new index be sent on the N38 node from N79, decide by the routing policy in the concrete DHT algorithm as for process of transmitting.Usually in DHT, using PUT () order carrying out index sends.In like manner node N102, N150 also are numbered L
NData block, the index that their produce is sent to node N168 respectively, N85 is last.Even if it is thus clear that the index that produces to identical block, because node SID difference separately, final index stores node is also inequality.Yet when different K ey value was in same node administration scope, the index stores node can be identical in some cases.
According to the needs of system, specify the value of a quantity limiting parameter l again.Node can't directly be stored when receiving index, but the judgement that will whether store earlier.With node N38 among Fig. 3 is example, and after receiving the index that comes from N79, it can be judged to being numbered L
NSimilar index item number whether cross l.If do not reach, then can store this index; Otherwise, then find out earlier in all these index entries minimum Score value (with Score high more good more be example).If the Score value of new index is higher than this minimum value, then replace a minimum index of Score value; Otherwise, abandon storage.
Can find out that through Fig. 4 when the subspace quantity of dividing is many more, promptly the number of SID is many more, the reliability of index is high more.But consider expense when searching data block, the number of SID is difficult for being provided with too highly, can in index reliability and overhead, seek a suitable value according to actual needs.Present embodiment finds that the number of SID is at 4 o'clock, and resultant effect is optimum.
Claims (5)
- In P2P live TV stream system based on the distributed index implementation method of DHT, it is characterized in that, may further comprise the steps:Each node division in the network to corresponding subspace, and is distributed the subspace identifier SID of corresponding subspace for each node;When node need send the index of data block as source node; This node generates resource identification Key according to the sid value of data block information and source node; And produce the index < Key, Value>of this data block, wherein Key is for generating resource identification; Value is an index value, and index value includes source node IP address; Afterwards, the mapping relations of this node resource identification Key and specified node according to a preconcerted arrangement are sent to specified node with said index and store;When node need be searched data block; This node searches as required that all subspace identifier SID obtain different resources sign Key in data block information and the network of data block; Thereby the resource identification Key according to a preconcerted arrangement and the mapping relations of specified node; In all specified nodes, search, thereby can obtain all corresponding index of this data block in the network.
- According to claim 1 in P2P live TV stream system based on the distributed index implementation method of DHT; It is characterized in that; With the concrete mode of each node division in the network to corresponding subspace be: at first confirm all nodes in the network to be divided in each subspace the subspace number equably.
- According to claim 1 or claim 2 in P2P live TV stream system based on the distributed index implementation method of DHT, it is characterized in that, also comprise the life cycle of this index in the said index value;The life cycle that the index of the local storage of the node in the network is corresponding arrives after date, and node is removed the index that expires automatically.
- As claim 3 said in P2P live TV stream system based on the distributed index implementation method of DHT, it is characterized in that, also comprise the source node scoring of sending this index in the said index value; The scoring of said source node is used to assess the source node of index and the ability of service is provided for other node;Node in the network judged earlier that whether the corresponding memory space of same asset sign Key reaches the upper limit, as not, directly added before the new index that receives of storage; In this way; Then earlier the minimum index of source node scoring in the index of the source node scoring of new reception hint and current storage same asset sign Key is compared, as greater than, then substitute the minimum index of source node scoring of current storage with the new index that receives; Otherwise, abandon the index of new reception.
- As claim 3 said in P2P live TV stream system based on the distributed index implementation method of DHT, it is characterized in that, be 4 with subspace number in the network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210011913.4A CN102572602B (en) | 2012-01-16 | 2012-01-16 | In P2P live streaming system, the distributed index based on DHT realizes method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210011913.4A CN102572602B (en) | 2012-01-16 | 2012-01-16 | In P2P live streaming system, the distributed index based on DHT realizes method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102572602A true CN102572602A (en) | 2012-07-11 |
| CN102572602B CN102572602B (en) | 2016-06-29 |
Family
ID=46416860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210011913.4A Expired - Fee Related CN102572602B (en) | 2012-01-16 | 2012-01-16 | In P2P live streaming system, the distributed index based on DHT realizes method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102572602B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104199679A (en) * | 2014-08-01 | 2014-12-10 | 百度在线网络技术（北京）有限公司 | Method and device used for obtaining updating |
| CN110169008A (en) * | 2018-07-10 | 2019-08-23 | 深圳花儿数据技术有限公司 | A kind of distributed data redundancy storage method based on consistency hash algorithm |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645831A (en) * | 2009-05-08 | 2010-02-10 | 中国科学院声学研究所 | Node organization method in P2P system |
| CN101741903A (en) * | 2009-11-20 | 2010-06-16 | 北京工业大学 | Group-based trust data management method in mobile P2P network |
| CN101860474A (en) * | 2009-04-08 | 2010-10-13 | 中兴通讯股份有限公司 | Peer-to-peer network and resource information processing method based on same |
| CN101867527A (en) * | 2010-07-06 | 2010-10-20 | 重庆大学 | Layering Chord routing method based on physical position |
| WO2010148843A1 (en) * | 2009-12-31 | 2010-12-29 | 中兴通讯股份有限公司 | System and method for mobile internet service sharing |
-
2012
- 2012-01-16 CN CN201210011913.4A patent/CN102572602B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101860474A (en) * | 2009-04-08 | 2010-10-13 | 中兴通讯股份有限公司 | Peer-to-peer network and resource information processing method based on same |
| CN101645831A (en) * | 2009-05-08 | 2010-02-10 | 中国科学院声学研究所 | Node organization method in P2P system |
| CN101741903A (en) * | 2009-11-20 | 2010-06-16 | 北京工业大学 | Group-based trust data management method in mobile P2P network |
| WO2010148843A1 (en) * | 2009-12-31 | 2010-12-29 | 中兴通讯股份有限公司 | System and method for mobile internet service sharing |
| CN101867527A (en) * | 2010-07-06 | 2010-10-20 | 重庆大学 | Layering Chord routing method based on physical position |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104199679A (en) * | 2014-08-01 | 2014-12-10 | 百度在线网络技术（北京）有限公司 | Method and device used for obtaining updating |
| CN110169008A (en) * | 2018-07-10 | 2019-08-23 | 深圳花儿数据技术有限公司 | A kind of distributed data redundancy storage method based on consistency hash algorithm |
| CN110169008B (en) * | 2018-07-10 | 2022-06-03 | 深圳花儿数据技术有限公司 | Distributed data redundancy storage method based on consistent hash algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102572602B (en) | 2016-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101102250B (en) | Distributed hashing mechanism for self-organizing networks | |
| CN110166220B (en) | Slicing method for slicing according to hash value of partition key | |
| JP6967615B2 (en) | Distributed search and index update methods, systems, servers and computer equipment | |
| CN102217253B (en) | Message forwarding method, equipment and network device | |
| CN101483670B (en) | Regional P2P computation data consistency maintenance method based on distributed hash table | |
| CN101567796B (en) | Multimedia network with fragmented content and business method thereof | |
| CN104115472A (en) | A method for scalable routing in content-oriented networks | |
| CN102405623B (en) | Method and device for storing routing table entry | |
| US20110208828A1 (en) | Node apparatus and computer-readable storage medium for computer program | |
| CN101626392B (en) | Methods and systems for storing and searching data | |
| EP2856355B1 (en) | Service-aware distributed hash table routing | |
| EP2747336B1 (en) | Content processing method, device and system | |
| Shang et al. | A survey of distributed dataset synchronization in Named Data Networking | |
| CN104184812A (en) | Multi-point data transmission method based on private cloud | |
| CN102945249A (en) | Policy rule matching query tree generating method, matching method and device | |
| CN110990448B (en) | Distributed query method and device supporting fault tolerance | |
| CN103281382A (en) | Method and node for file transfer based on p2p | |
| CN102891872A (en) | Method and system for storing and searching data in peer to peer (P2P) network | |
| KR20090094313A (en) | Method and system for publishing the content, method and system for querying the content | |
| CN100536422C (en) | Peer-to-peer network and its network resource inquiring method | |
| Guo et al. | HDS: A fast hybrid data location service for hierarchical mobile edge computing | |
| CN102037711B (en) | Limiting storage messages in peer to peer network | |
| CN102572602A (en) | Distributed index achieving method in peer-to-peer (P2P) live stream system based on distributed hash table (DHT) | |
| CN114244849A (en) | Routing layer of P2P network | |
| JP2014093666A (en) | Communication control device, communication device, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160629 Termination date: 20200116 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |