CN101645922B - CDN network system based on geographical position information encoding and distribution treatment method - Google Patents
CDN network system based on geographical position information encoding and distribution treatment method Download PDFInfo
- Publication number
- CN101645922B CN101645922B CN200910082190.5A CN200910082190A CN101645922B CN 101645922 B CN101645922 B CN 101645922B CN 200910082190 A CN200910082190 A CN 200910082190A CN 101645922 B CN101645922 B CN 101645922B
- Authority
- CN
- China
- Prior art keywords
- network
- node
- management server
- cache
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/52—Network services specially adapted for the location of the user terminal
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a CDN network system based on geographical position information encoding and a distribution treatment method thereof. The network system comprises a full-network management layer, a local content central management layer and a Cache network layer. The full-network management layer is acted by a server with a backup and responsible for full-network information management and analysis and implementation of operation commands of a system administrator; the local content central management layer consists of part of nodes with stable storage ability and strong calculation ability and is responsible for related calculation of a CDN local region and storage of management information; and the Cache network layer is applied in the encoding way based on hierarchical geographical position information, and a plurality of nodes are responsible for routing and the storage of data which is required to be back up. According to the CDN network system, the CDN system has higher anti-dynamic property and scalability of the CDN system and facilitates distribution treatment and data management of media files by hierarchical storage and management of media data and management data.
Description
Technical field
The structure of the content distributing network that the present invention relates to be made up of distributed node relates in particular to a kind of CDN network system and distribution processing method thereof based on level type encode geolocation information method.
Background technology
The full name of CDN is Content Distribution/Delivery Network, i.e. content distributing network.Its objective is through in existing Internet, increasing the new network architecture of one deck; With the content release of website to making the user can obtain required content nearby near user's network edge; On the one hand, CDN can reduce delay, solves the situation of Internet network congestion; On the other hand, can also effectively realize multiple broadband application, promote the development of whole broadband industry.
CDN be set up and cover on the Internet, by the virtual network that the node server zone that is distributed in zones of different is formed, be application layer " increment " network that on existing Internet network configuration basis, makes up, also be a kind of intermediate layer that transparent service is provided; It utilizes technology such as distributed caching/duplicate, load balancing, traffic engineering and client are redirected; The function that provides diverse network distribution of content and service to send specially comprises that dynamically the on-premise network content is handled the traffic to the edge according to Web content; Access request is transmitted to the optimal service device; Thereby make that the user can be with the fastest speed,, can solve network congestion from obtaining required information near user's place; Improve response speed and service quality, particularly other technological incomparable advantage is being arranged aspect the Streaming Media transmission.
Along with the development of the Internet and P2P technology, business opportunity has wherein been seen by increasing operator and company, and how to let the user obtain service then is a key issue faster and betterly.Because the difference in area; Possibly give priority to the demand of content in different places; For the content that lets this area stress is easy to by the one's respective area user capture, system can utilize GSLB technology, and this content is distributed on the service node with the nearest operate as normal of user capture; When the user files a request, directly to its response.When the node of CDN service has been used in user capture; Take into full account the place of user's initiation request and the situation of network at that time; Decide to be directed to user's request on the lighter relatively node server of user's load simultaneously recently, guarantee that user's visit can obtain more in time response reliably.Because a large amount of user captures has all directly been responded by the CDN node server that is distributed in network edge, this has just not only improved user's visit quality, and the while has been reduced the load pressure of source server effectively.
Summary of the invention
The object of the present invention is to provide a kind of CDN network system and distribution processing method thereof based on encode geolocation information; Through layering storage and management to media data and management data; Make the CDN system have higher anti-dynamic and autgmentability, be convenient to the distribution processor and the data management of media file.
To achieve these goals; A kind of CDN network system of the present invention based on encode geolocation information; Comprise three-layer architecture: the whole network management level, local content center management level and Cache network layer, wherein; Said the whole network management level are served as by server reliable and stable, that backup is arranged, mainly are responsible for the information management of the whole network and the analysis and the execution of system manager's operational order; Said local content center management level are stable by the part storage capacity; The node that computing capability is strong is formed; Main task is to be responsible for the correlation computations of CDN regional area and the storage of management information, and management data is separated with media data, is convenient to the statistics and the calculating of system extension and user behavior; Said Cache network layer is to be organized through the mode of P2P by many nodes, main route and the storage of being responsible for required Backup Data, simultaneously; The introducing of level type geographical position coding method; Guaranteed the region clustering property of node in the Cache network layer, and the P2P The Application of Technology is convenient to the expansion of this layer; Reduce the load of the whole network management and content management server, promoted user experience.
In network system of the present invention, each server capability and processing method are following:
● said the whole network management level adopt the whole network management server (Global Manage Server:GM): be responsible for the configuration network Control Parameter, parameter is safeguarded and upgraded; Domain Properties deploy content center, base area; The global information at storage all the elements center; Distributing contents is set.According to pre-configured global parameter, help to generate object content center and Cache network; Global statistics information such as the storage total amount of regular each content center of collection and Cache network, node sum, total number of files, file access number of times are for further optimum management is prepared.The adding of expansion that the setting of this layer is convenient to serve and new service area is beneficial to renewal and the optimization of operator to control information simultaneously.
● said content center layer adopts local management server (Local Manage Server:LM): content center is to be specified by GM; Be responsible for the server that each regional Cache network global information is safeguarded and calculated; All obtain file from data source, all download to this locality by LM, and by LM it are carried out piecemeal; According to required number of copies and corresponding placement locations of each data block of information calculations such as existing file popularities, and distribute to the Cache network node.Simultaneously, the Content Management central core in same area can be responsible for by a plurality of LM jointly, all stores the global index's information in the Cache of this area network among each LM, complete interconnected (see figure 2) between the LM.Many LM can reduce the calculating pressure of LM, quicken the processing time of distribution procedure.The advantage of this layer is:
■ has concentrated the management information of file with need not manual intervention, effectively computing function among the CDN and memory function is divided on the different nodes, has reduced node load and maintenance complexity;
■ is based on the multi-zone supervision of management information and memory contents, and through the mode of local message regular update, GM can effectively collect the statistical information of file access from LM, be convenient to the analysis to user behavior of operator or system designer.
● the composition node of said Cache network layer is called Cache node (Cache Peer:CP); All CP of all regions form the Cache network of an overall situation jointly through distributed hashtable (DHT); And the CP node in same area is called the Cache network in an area; And manage by each regional LM, the node in the Cache network can be connected (see figure 3) with at least one LM maintenance.In order to make the node in the same regional Cache network have topological proximity, use coding method (being hash method) here based on level type geographical location information, guarantee that the close node in geographical position is in same zone on topological diagram.The CP node receives the dispatch messages that LM sent, the position of finding data trnascription institute to place, and backup the acceleration user capture.Each regional LM manages all CP of Cache network, comprises the number of copies of file data blocks, the copy placement location, and user access activity distributes, life cycle, information such as popularity.Each CP node storage ID and nearest data block and the corresponding data piece survival function of present node ID regularly detect the data block state, deletion stale data piece.In order to make the Cache network be convenient to expansion, upgrade, adapt to the network dynamic, all CP nodes are organized through DHT.At last, LM generates data block index entry administration-management reason of going forward side by side for each file.The advantage of this layer is:
■ P2P The Application of Technology can need not the center supervision and carry out the data location quickly and efficiently;
■ CP manages memory contents own voluntarily, has reduced the network service cost.
■ is based on the coding method of level type geographical location information, and the node that the geographic area is closed on also closes on the Hash space.
In addition; Content providing server (the Content ProvidingServer of data source is provided for network system of the present invention; CPS), be the part that is independent of outside the above-mentioned three-decker, can be separate server; Also can be server cluster, it stores the initial data of file to be distributed as data source.
In addition, the data distribution processing method of a kind of CDN network system based on encode geolocation information of the present invention is characterized in that, comprises the steps:
1) the whole network management server decision-making distributing contents (can also can distribute decision-making automatically by system through preestablishing rule through the mode of keeper's setting) comprising: filename, target area and other supplementarys;
2) the whole network management server is according to the distribution decision information; Local management server to objective network is sent message, comprises: the data volume of number of files to be distributed, title, data total amount, each program, filename, file source address, program popularity initial value and geographical location information;
3) local management server processing messages is cut into data block with each file to be distributed, calculates the size of all data blocks, backup number and the placement location and the required memory space of each piece, and calculates the requisite space summation;
4) local management server judges whether local Cache network remaining space satisfies memory requirement;
5) if satisfy, local management server is calculated the cryptographic hash blockID_i of each blocki through the coding method based on level type geographical location information, and sends branch to the Cache node and send instructions; Otherwise, then execution in step 9);
Wherein, the coding method of above-mentioned level type geographical location information, node that can be effectively that geographical space is close and data map also close point to the Hash space makes node and data have the geographic area aggregation, helps to improve the hit rate of CDN.In the coding method of level type geographical location information, the ID value that obtains is the sequence of a N position, this sequence from left to right, by big zone to the zonule, the expression geographical location information of level, each zone can be by the not numeric representation of isotopic number.Such as one 8 Serial No., wherein, front two is represented provinces and cities, 3,4 liang of bit representation area/counties and cities, 5,6 liang of bit representation streets, last two bit representation numbers or the like.There is the different coding granularity in different application environments, and from coarse to fine, orderliness is clear.
This coding has multiple generation method, for example:
A) the geographic area code name through some official's regulations makes up expression, such as postcode, administrative region code and area code etc.;
B), obtain characterizing the coding of differentiation in different regions through modes such as IP map addresses, couplings to different application.6) the local management server message that will distribute blocki is sent to the node in the Cache network layer; The node of receiving message is through distributed hash Routing Protocol (as: Chord; Pastry, CAN, Tapstry etc.); Find with blockID_i Hash space on nearest node, as the host node home node of blocki; All backup informations of blocki all are stored on the host node home node, and simultaneously, according to copy number that calculates and position, the copy to blocki in the Cache network layer is done storage;
7) Cache node notice local management server data backup success message;
8) local management server is that respective file is set up index, and index entry comprises: filename, file ID, corresponding data piece ID, data block copy number and data block copy life cycle.
9) through the calculating of local management server, optimize the Cache memory space, i.e. the block that deletion is of little use increases free space, upgrades the global information among the LM simultaneously.
In the optimization Cache of this step memory space process, the delet method step of file is following:
A) calculate through LM and treat that deleted data piece tabulation back is (through certain deletion strategy; For example add long more block of CDN network time; The principle that priority is low is more selected, and comprises the information such as ID value of each data block in the tabulation), handle block to be deleted one by one.
B) take out the ID value of block one by one.
C),, send deletion message to its home node through the DHT agreement to each block ID.
D) the home node of each data block has preserved all backup locations of this data block, takes out the backup location information list.
E) Home node each CP node in tabulation sends deletion message.
F) behind the CP knot removal block, notice home node.
G) after deletion was accomplished, home node notified LM.
H) LM upgrades index information.
10) local management server is judged current C ache network layer space size, if it is enough to optimize rear space, then execution in step 5); Otherwise, execution in step 11);
11) by local management server LM notice the whole network management server GM distribution failure.
Of the present inventionly be based on the CDN network system of level type encode geolocation information method and the beneficial effect of distribution processing method thereof: this system has guaranteed the effective division of labor of node on managerial ability through multi-tier systematic structure, is beneficial to the expansion of system; Through level type geographical location information; Management node and memory node among the CDN are effectively assembled, be convenient to Network Management and expansion, simultaneously; Through the P2P network configuration, system has good anti-dynamic characteristics.In addition, distribute and the deletion optimizing process, guaranteed the dynamically updating property of content, strengthened the service ability of system, promoted user experience.
Description of drawings
Fig. 1 is the structural representation of the CDN network system based on encode geolocation information of the present invention.
Fig. 2 is the organizational form sketch map of the content center management server LM of the CDN network system based on encode geolocation information of the present invention.
Fig. 3 is the structural representation of the Cache network of the CDN network system based on encode geolocation information of the present invention.
Fig. 4 is the flow chart of the distribution processing method of the CDN network system based on encode geolocation information of the present invention.
Fig. 5 is the data delete procedure flow chart in the distribution processing method of the CDN network system based on encode geolocation information of the present invention.
Fig. 6 is the Cache network layer sketch map in the whole nation.
Embodiment
Below in conjunction with accompanying drawing and specific embodiment CDN network system and the distribution processing method thereof based on encode geolocation information of the present invention carried out detailed explanation.
Fig. 1 is the structural representation of the CDN network system based on encode geolocation information of the present invention.As shown in Figure 1; A kind of CDN network system of the present invention based on encode geolocation information; Comprise three-layer architecture: the whole network management level, local content center management level and Cache network layer, wherein; Said the whole network management level are served as by server reliable and stable, that backup is arranged, mainly are responsible for the information management of the whole network and the analysis and the execution of system manager's operational order; Said local content center management level are stable by the part storage capacity; The node that computing capability is strong is formed; Main task is to be responsible for the correlation computations of CDN regional area and the storage of management information, and management data is separated with media data, is convenient to the statistics and the calculating of system extension and user behavior; Said Cache network layer is to be organized through the mode of P2P by many nodes, main route and the storage of being responsible for required Backup Data, simultaneously; The introducing of level type geographical position coding method; Guaranteed the region clustering property of node in the Cache network layer, and the P2P The Application of Technology is convenient to the expansion of this layer; Reduce the load of the whole network management and content management server, promoted user experience.
In network system of the present invention, each server capability and processing method are following:
● the whole network management level adopt the whole network management server (Global Manage Server:GM): be responsible for the configuration network Control Parameter, parameter is safeguarded and upgraded; Domain Properties deploy content center, base area; The global information at storage all the elements center; Distributing contents is set.According to pre-configured global parameter, help to generate object content center and Cache network; Global statistics information such as the storage total amount of regular each content center of collection and Cache network, node sum, total number of files, file access number of times are for further optimum management is prepared.The adding of expansion that the setting of this layer is convenient to serve and new service area is beneficial to renewal and the optimization of operator to control information simultaneously.
● it is the organizational form sketch map of the content center management server LM of the CDN network system based on encode geolocation information of the present invention that the content center layer adopts local management server (Local Manage Server:LM): Fig. 2.Content center is to be specified by GM; Be responsible for the server that each regional Cache network global information is safeguarded and calculated; All obtain file from data source, all download to this locality by LM, and by LM it are carried out piecemeal; According to required number of copies and corresponding placement locations of each data block of information calculations such as existing file popularities, and distribute to the Cache network node.Simultaneously, as shown in Figure 2, the Content Management central core in same area can be responsible for by a plurality of LM jointly, all stores the global index's information in the Cache of this area network among each LM, and is interconnected entirely between the LM.Many LM can reduce the calculating pressure of LM, quicken the processing time of distribution procedure.The advantage of this layer is:
■ has concentrated the management information of file with need not manual intervention, effectively computing function among the CDN and memory function is divided on the different nodes, has reduced node load and maintenance complexity;
■ is based on the multi-zone supervision of management information and memory contents, and through the mode of local message regular update, GM can effectively collect the statistical information of file access from LM, be convenient to the analysis to user behavior of operator or system designer.
● Fig. 3 is the structural representation of the Cache network of the CDN network system based on encode geolocation information of the present invention.As shown in Figure 3; The composition node of Cache network layer is called Cache node (CachePeer:CP); All CP of all regions form the Cache network of an overall situation jointly through distributed hashtable (DHT); And the CP node in same area is called the Cache network in an area, and is managed by each regional LM, and the node in the Cache network can keep being connected with at least one LM.In order to make the node in the same regional Cache network have topological proximity, use coding method (being hash method) here based on level type geographical location information, guarantee that the close node in geographical position is in same zone on topological diagram.The CP node receives the dispatch messages that LM sent, the position of finding data trnascription institute to place, and backup the acceleration user capture.Each regional LM manages all CP of Cache network, comprises the number of copies of file data blocks, the copy placement location, and user access activity distributes, life cycle, information such as popularity.Each CP node storage ID and nearest data block and the corresponding data piece survival function of present node ID regularly detect the data block state, deletion stale data piece.In order to make the Cache network be convenient to expansion, upgrade, adapt to the network dynamic, all CP nodes are organized through DHT.At last, LM generates data block index entry administration-management reason of going forward side by side for each file.The advantage of this layer is:
■ P2P The Application of Technology can need not the center supervision and carry out the data location quickly and efficiently;
■ CP manages memory contents own voluntarily, has reduced the network service cost.
■ is based on the coding method of level type geographical location information, and the node that the geographic area is closed on also closes on the Hash space.
In addition; Content providing server (the Content ProvidingServer of data source is provided for network system of the present invention; CPS), be the part that is independent of outside the above-mentioned three-decker, can be separate server; Also can be server cluster, it stores the initial data of file to be distributed as data source.
Fig. 4 is the flow chart of the distribution processing method of the CDN network system based on encode geolocation information of the present invention.As shown in Figure 4, the data distribution processing method of a kind of CDN network system based on encode geolocation information of the present invention is characterized in that, comprises the steps:
1) the whole network management server decision-making distributing contents (can also can distribute decision-making automatically by system through preestablishing rule through the mode of keeper's setting) comprising: filename, target area and other supplementarys;
2) the whole network management server is according to the distribution decision information; Local management server to objective network is sent message, comprises: the data volume of number of files to be distributed, title, data total amount, each program, filename, file source address, program popularity initial value and geographical location information;
3) local management server processing messages is cut into data block with each file to be distributed, calculates the size of all data blocks, backup number and the placement location and the required memory space of each piece, and calculates the requisite space summation;
4) local management server judges whether local Cache network remaining space satisfies memory requirement;
5) if satisfy, local management server is calculated the cryptographic hash blockID_i of each blocki through the coding method based on level type geographical location information, and sends branch to the Cache node and send instructions; Otherwise, change step 9);
Here, the coding method of level type geographical location information, node that can be effectively that geographical space is close and data map also close point to the Hash space makes node and data have the geographic area aggregation, helps to improve the hit rate of CDN.In the coding method of level type geographical location information, the ID value that obtains is the sequence of a N position, this sequence from left to right, by big zone to the zonule, the expression geographical location information of level, each zone can be by the not numeric representation of isotopic number.Such as one 8 Serial No., wherein, front two is represented provinces and cities, 3,4 liang of bit representation area/counties and cities, 5,6 liang of bit representation streets, last two bit representation numbers or the like.There is the different coding granularity in different application environments, and from coarse to fine, orderliness is clear.
This coding has multiple generation method, for example:
A) the geographic area code name through some official's regulations makes up expression, such as postcode, administrative region code, area code or the like;
B), obtain characterizing the coding of differentiation in different regions through modes such as IP map addresses, couplings to different application.
6) the local management server message that will distribute blocki is sent to the node in the Cache network layer; The node of receiving message is through distributed hash Routing Protocol (as: Chord; Pastry, CAN, Tapstry etc.); Find with blockID_i Hash space on nearest node, as the host node home node of blocki; All backup informations of blocki all are stored on the host node home node, and simultaneously, according to copy number that calculates and position, the copy to blocki in the Cache network layer is done storage;
7) Cache node notice local management server data backup success message;
8) local management server is that respective file is set up index, and index entry comprises: filename, file ID, corresponding data piece ID, data block copy number and data block copy life cycle.
9) through the calculating of local management server, optimize the Cache memory space, i.e. the block that deletion is of little use increases free space, upgrades the global information among the LM simultaneously.
Wherein, Fig. 5 is the data delete procedure flow chart in the distribution processing method of the CDN network system based on encode geolocation information of the present invention.As shown in Figure 5, in the optimization Cache of this step 9) memory space process, the delet method step of file is following:
A) calculate through LM and treat that deleted data piece tabulation back is (through certain deletion strategy; For example add long more block of CDN network time; The principle that priority is low is more selected, and comprises the information such as ID value of each data block in the tabulation), handle block to be deleted one by one.
B) take out the ID value of block one by one.
C),, send deletion message to its home node through the DHT agreement to each block ID.
D) the home node of each data block has preserved all backup locations of this data block, takes out the backup location information list.
E) Home node each CP node in tabulation sends deletion message.
F) behind the CP knot removal block, notice home node.
G) after deletion was accomplished, home node notified LM.
H) LM upgrades index information.
10) local management server is judged current C ache network layer space size, if it is enough to optimize rear space, then execution in step 5); Otherwise, execution in step 11);
11) by local management server LM notice the whole network management server GM distribution failure.
Embodiment
Below in conjunction with specific embodiment this CDN network system and function is described.Suppose that this CDN network need cover Beijing, Shanghai, three locations, Tianjin.With the Beijing area is example, supposes that server distribution is shown in below table 1; Two-server is the content center management that LM1 and LM2 are responsible for the Beijing area; In addition, a station server is as the global information of GM management CDN.Here, Pastry through the mode of its prefix matching, matees the node of corresponding area as basic Routing Protocol fast.
Table 1: Beijing area Cache network layer server distribution situation signal
According to the target of CDN network and the setting of institute's distributing contents, need be distributed on the nearest service node of target.Postcode is one 6 a Serial No., and wherein, front two is represented provinces and cities, middle two bit representation area/counties and cities, last two bit representations the subsidiary post office that will deliver.Therefore, through postcode, can be effectively with the area from big area dividing to the zonule even the street.Therefore, the postcode in required distribution area can be used as the geographical location information of this area, in order in DHT, to generate ID value, the Cache network layer sketch map in the whole nation that each node ID is as shown in Figure 6.。When file f will be distributed to the Haidian District, process was following:
1) GM sends branch to LM1 (perhaps LM2) and sends instructions, and f is distributed to the Haidian District;
2) LM1 downloads to this locality with file f from data source;
3) obtain the situation that takies of memory space in the local Cache network through LM1; Discovery remaining space deficiency thinks that new data block backups, and optimizes criterions (such as adding long more data block of CDN network time, priority is low more) according to some; Select data block tabulation to be deleted; Here, treat that the tabulation of deleted data piece is for { block 10, block11};
4) ID of taking-up block10 is like node 1000812601825135;
5) find the host node home node1000812501920145 of node 1000812601825135 through the DHT agreement;
6) from node 1000812501920145, take out all backup node tabulations of block10 1000812501920145,1000813709467800};
7) all the CP nodes in tabulation send the message of deletion block10;
8) the CP node is deleted block10 after receiving deletion message;
9) node 1000812501920145 is deleted the block10 relevant informations simultaneously;
10) after all block10 deleted end, node 1000812501920145 notice LM1 deleted successfully;
11) LM1 upgrades the concordance list that comprises block10;
12) take out next block11, continue deletion, its step is identical with the delete procedure of block10; Until treating that delete list is empty, stops; Delete procedure finishes;
13) obtain in order to quicken the user, LM1 is cut into data block (also can be non-divided) with file, and every is a block, and file can be divided into sequence of blocks of data, supposes to be divided into 3, is { block1, block2, block3};
14) according to file name; Through the coding method based on level type geographical location information is that file generates index stores ID; Suppose totally 16 of ID, preceding 6 is postcode, and back 10 is hash function generation value; That is: RID=100081 (encode geolocation information is shown with regional postcode tables here)+Hash (filename) (as 2356780132)
Be RID=1000812356780132
15) generate ID through coding method for each data block based on level type geographical location information.Data block ID generates formula:
BIDi=100081 (encode geolocation information is shown with regional postcode tables here)+Hash (sequence number of filename+block)
Suppose block1, block2, the ID of block3} be 1000810003215200,100081013567810,100084816213269}.
16) information such as comprehensive popularity; Through certain copy number computational methods; For each data block calculate required copy number and corresponding placement location (such as; Block1 need place 2 parts in node 1000812501920145 and 1000813709467800, and block2 need place 2 parts in node 1000812501920145 and 1000813709467800, and block3 need place 1 part at node 100084816213269).
17) size of supposing 3 data blocks respectively is 100M, 200M and 300M, then step 16) required storage size is 900M.LM1 calculates in the Cache network, and the residue free space of Haidian District node is if satisfy the copy backup request, then according to step 16) result of calculation generate the resource copy, and place.Placing step to each data block is following, like block1:
A) LM1 selects a node as Ingress node at random, and the dispatch messages notice of block1 is somebody's turn to do inlet;
B) Ingress node finds the nearest node 10008137094678 of ID of ID00block1, and this node is called the home node of block_1, stores the copy of a block1;
C) according to number of copies and the placement location of the block_1 that calculates; Through the Pastry Routing Protocol; To remain copy and place, simultaneously, each Backup Data all adds the timestamp of an expression residue life span; As 100 hours, and with 24 hours be the cycle regularly the oneself detect the residue life span.After backup is accomplished, notice home node;
D) after home node receives the message of backup completion, the notice Ingress node.
E) after block all adds completion, ID and the corresponding copy information of all block are formed index entry,, be stored on the nearest node 1000812501920145 of RID (1000812356780132) according to the Pastry routing algorithm;
18) node 1000812501920145 content of announcement center LM1 data trnascriptions back up successfully, generate global index's item, are stored among the LM1, and resource adds to be accomplished.
When reading of data, at first find the index node 1000812501920145 of file f, therefrom take out the ID tabulation of sequence of blocks of data, each block content is taken out from network through the Pastry method for routing again.
Explain that the other guide in the document is directed against the those of ordinary skill in this professional domain, all can carry out technology and realize, repeat no more here.
Claims (5)
1. the CDN network system based on encode geolocation information is characterized in that, comprises three-layer architecture: the whole network management level, and local content center management level and Cache network layer, wherein,
Said the whole network management level are served as by the server that backup is arranged, and are responsible for the information management of the whole network and the analysis and the execution of system manager's operational order;
Said local content center management level are made up of the node that the part storage capacity is stable, computing capability is strong; Be responsible for the correlation computations of CDN regional area and the storage of management information; Management data is separated with media data, be convenient to the statistics and the calculating of system extension and user behavior;
Said Cache network layer is used the coded system based on level type geographical location information, is organized through the mode of P2P by many nodes, is responsible for route and storage to required Backup Data, specifically,
Said the whole network management level adopt the whole network management server, and this server is responsible for: the configuration network Control Parameter, parameter is safeguarded and upgraded; Domain Properties deploy content center, base area; The global information at storage all the elements center; Distributing contents is set; According to pre-configured global parameter, help to generate object content center and Cache network; Regularly collect the global statistics information of each content center and Cache network, comprising: storage total amount, node sum, total number of files and file access number of times;
Said content center layer adopts local management server, and content center is to be specified by the whole network management server, is responsible for the server that each regional Cache network global information is safeguarded and calculated; All obtain file from data source; All download to this locality, and it is carried out piecemeal, according to required number of copies and the corresponding placement location of existing each data block of file popularity information calculations by local management server by local management server; And distribute to the Cache network node; Simultaneously, the Content Management central core in same area is responsible for by a plurality of local management server jointly; All store the global index's information in the Cache of this area network in each local management server, interconnected entirely between the local management server;
The composition node of said Cache network layer is called the Cache node; All Cache nodes of all regions are through the common Cache network of forming an overall situation of distributed hashtable; And the Cache node in same area has been formed the Cache network in an area; And manage by each regional local management server, the node in the Cache network can be connected with at least one local management server maintenance.
2. the data distribution processing method based on the CDN network system of encode geolocation information is characterized in that, comprises the steps:
1) the whole network management server is perhaps distributed the mode decision-making distributing contents of decision-making through preestablishing rule by system automatically through the mode that the keeper sets, and comprising: filename, target area and other supplementarys;
2) the whole network management server is according to the distribution decision information; Local management server to objective network is sent message, comprises: the data volume of number of files to be distributed, title, data total amount, each program, filename, file source address, program popularity initial value and geographical location information;
3) local management server processing messages is cut into data block with each file to be distributed, calculates the size of all data block block, backup number and the placement location and the required memory space of each piece, and calculates the requisite space summation;
4) local management server judges whether local Cache network remaining space satisfies memory requirement;
5) if satisfy, local management server is calculated the cryptographic hash blockID_i of each data block blocki through the coding method based on level type geographical location information, and sends branch to the Cache node and send instructions; Otherwise, then execution in step 9);
6) local management server is sent to the node in the Cache network layer with the message of distributing data piece blocki; The node of receiving message is through the distributed hash Routing Protocol, find with blockID_i Hash space on nearest node as the host node home node of data block blocki; All backup informations of data block blocki all are stored on the host node home node, simultaneously, in the Cache network layer, the copy of data block blocki are done storage according to copy number that calculates and position;
7) Cache node notice local management server data backup success message;
8) local management server is that respective file is set up index, and index entry comprises: filename, file ID, corresponding data piece block ID, data block block number of copies and data block block copy life cycle;
9) through the calculating of local management server, optimize the Cache memory space, the data block block by the Cache knot removal is of little use increases free space, upgrades the global information in the local management server simultaneously;
10) local management server is judged current C ache network layer space size, if it is enough to optimize rear space, then execution in step 5); Otherwise, execution in step 11);
11) by local management server notice the whole network management server distribution failure.
3. the data distribution processing method of the CDN network system based on encode geolocation information as claimed in claim 2; It is characterized in that the coding method of the level type geographical location information described in the said step 5) is the ID value that obtains a N bit sequence; This sequence from left to right; By big zone to the zonule, the expression geographical location information of level, each zone is by the not numeric representation of isotopic number, the generation method of coding comprises:
A) the geographic area code name through some official's regulations makes up expression, comprising: postcode, administrative region code and area code;
B), obtain characterizing the coding of differentiation in different regions through IP map addresses, matching way to different application.
4. the data distribution processing method of the CDN network system based on encode geolocation information as claimed in claim 2 is characterized in that the concrete steps of said step 9) are following:
A) utilize certain deletion strategy to calculate to treat deleted data piece block tabulation through local management server after, handle one by one and treat deleted data piece block;
B) take out the ID value of data block block one by one;
C),, send deletion message to its host node home node through the DHT agreement to each data block block ID;
D) the home node of each data block blocki has preserved all backup locations of this data block, takes out the backup location information list;
E) host node Home node each Cache node in tabulation sends deletion message;
F) behind the Cache knot removal data block block, notice host node home node;
G) after deletion was accomplished, host node home node notified local management server;
H) local management server is upgraded index information.
5. the data distribution processing method of the CDN network system based on encode geolocation information as claimed in claim 4; It is characterized in that said deletion strategy is selected according to adding CDN network time long more data block block or the low more principle of priority.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910082190.5A CN101645922B (en) | 2009-04-17 | 2009-04-17 | CDN network system based on geographical position information encoding and distribution treatment method |
| JP2012505033A JP5532361B2 (en) | 2009-04-17 | 2010-04-14 | CDN network system coding based on geolocation information and method of distributing data |
| PCT/CN2010/000495 WO2010118638A1 (en) | 2009-04-17 | 2010-04-14 | Content distribution/delivery network system based on geographic location information encoding and data distribution method therefor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910082190.5A CN101645922B (en) | 2009-04-17 | 2009-04-17 | CDN network system based on geographical position information encoding and distribution treatment method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101645922A CN101645922A (en) | 2010-02-10 |
| CN101645922B true CN101645922B (en) | 2012-09-05 |
Family
ID=41657642
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910082190.5A Active CN101645922B (en) | 2009-04-17 | 2009-04-17 | CDN network system based on geographical position information encoding and distribution treatment method |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP5532361B2 (en) |
| CN (1) | CN101645922B (en) |
| WO (1) | WO2010118638A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106657437A (en) * | 2016-12-02 | 2017-05-10 | 董伟国 | Network communication method |
Families Citing this family (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645922B (en) * | 2009-04-17 | 2012-09-05 | 中国科学院声学研究所 | CDN network system based on geographical position information encoding and distribution treatment method |
| US8850003B2 (en) * | 2010-04-20 | 2014-09-30 | Zte Corporation | Method and system for hierarchical tracking of content and cache for networking and distribution to wired and mobile devices |
| CN101888384B (en) * | 2010-06-30 | 2014-02-05 | 北京世纪互联宽带数据中心有限公司 | Multi-media safety signaling system |
| CN101902468B (en) * | 2010-06-30 | 2013-03-20 | 北京世纪互联宽带数据中心有限公司 | Multimedia safety signaling system |
| CN101877709A (en) * | 2010-06-30 | 2010-11-03 | 北京世纪互联宽带数据中心有限公司 | Multi-media safety signaling system |
| CN101888385B (en) * | 2010-06-30 | 2013-04-24 | 北京世纪互联宽带数据中心有限公司 | Multi-media safety signaling system |
| CN101867527B (en) * | 2010-07-06 | 2012-08-08 | 重庆大学 | Layering Chord routing method based on physical position |
| CN102055798B (en) * | 2010-12-09 | 2013-01-02 | 南京远古科技有限公司 | Method for collecting programs in basic Chord ring and regional Chord rings |
| CN102546711B (en) * | 2010-12-20 | 2014-12-31 | 中国移动通信集团公司 | Storage adjustment method, device and system for contents in streaming media system |
| CN102164309B (en) * | 2011-03-21 | 2014-09-03 | 华为技术有限公司 | Program processing method, system and device |
| CN102891869B (en) * | 2011-07-20 | 2015-07-15 | 郑州威科姆科技股份有限公司 | CDN (Content Delivery Network) and content distribution and scheduling method |
| CN102368776B (en) * | 2011-11-25 | 2015-02-04 | 中国科学技术大学 | Optimization function module of node list in content distribution/delivery network (CDN) |
| CN103188294A (en) * | 2011-12-28 | 2013-07-03 | 百度在线网络技术（北京）有限公司 | Deleting method and deleting system for distributed cache |
| WO2013189038A1 (en) * | 2012-06-20 | 2013-12-27 | 华为技术有限公司 | Content processing method and network side device |
| CN103701619B (en) * | 2013-10-22 | 2016-09-07 | 上海交通大学 | The internet content distribution method dynamically selected based on CDN and data center |
| CN104717258B (en) * | 2013-12-16 | 2018-09-28 | 华为技术有限公司 | A kind of data transmission method for uplink and system |
| CN104202407B (en) * | 2014-09-10 | 2018-04-13 | 北京奇艺世纪科技有限公司 | A kind of video file synchronous method and device |
| CN104506647B (en) * | 2014-12-31 | 2017-12-08 | 北京奇艺世纪科技有限公司 | A kind of data-storage system and storage and access method |
| CN105516372B (en) * | 2016-02-25 | 2019-05-14 | 深圳天珑无线科技有限公司 | Processing method, device and the server of filename |
| CN106372163B (en) * | 2016-08-31 | 2020-12-04 | 天津南大通用数据技术股份有限公司 | Data distribution method and device suitable for distributed database |
| CN106850581B (en) * | 2017-01-04 | 2020-06-23 | 网宿科技股份有限公司 | Distribution backup method, system and server for interactive live broadcast streaming media data |
| CN107295061B (en) * | 2017-05-05 | 2019-08-30 | 中广热点云科技有限公司 | A kind of content distribution method based on content distributing network |
| CN109639758B (en) * | 2018-10-31 | 2020-05-12 | 中国科学院信息工程研究所 | Method and device for protecting user behavior privacy in content-centric network |
| CN110113398A (en) * | 2019-04-23 | 2019-08-09 | 安徽云融信息技术有限公司 | A kind of large-scale wireless sensing network data-storage system based on HBase |
| CN110188006B (en) * | 2019-06-03 | 2022-04-29 | 深圳市网心科技有限公司 | Data maintenance method, device and equipment |
| CN111181801B (en) * | 2019-12-04 | 2021-11-09 | 腾讯云计算（北京）有限责任公司 | Node cluster testing method and device, electronic equipment and storage medium |
| CN111857979B (en) * | 2020-06-28 | 2023-08-15 | 厦门极致互动网络技术股份有限公司 | Information management method, system, storage medium and equipment of distributed system |
| CN112328583A (en) * | 2020-10-29 | 2021-02-05 | 北京东方耀阳信息技术有限公司 | Spatio-temporal data management method |
| CN113993122B (en) * | 2021-09-10 | 2023-12-01 | 广东南方通信建设有限公司 | NB-IOT terminal, communication method and communication system |
| CN114172912A (en) * | 2021-11-01 | 2022-03-11 | 北京中合谷投资有限公司 | Networking method of hybrid distributed network |
| CN114173167A (en) * | 2021-12-17 | 2022-03-11 | 青岛聚看云科技有限公司 | Display device, content distribution network system, and server |
| CN115174955B (en) * | 2022-05-25 | 2023-11-17 | 楼培德 | Digital cinema nationwide high-speed distribution system based on future network |
| CN116527756B (en) * | 2023-05-04 | 2023-10-20 | 杭州元媒科技有限公司 | News content distribution system based on wireless mobile communication network |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101222424A (en) * | 2007-12-24 | 2008-07-16 | 中国电信股份有限公司 | Content distribution network and scheduling method based on content in the network |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4821500B2 (en) * | 2006-08-21 | 2011-11-24 | ブラザー工業株式会社 | Content distributed storage system, frame acquisition method, node device, etc. |
| JP2008059351A (en) * | 2006-08-31 | 2008-03-13 | Brother Ind Ltd | Node device, processing program, information distribution system, and server processing program |
| CA2667696A1 (en) * | 2006-09-06 | 2008-05-15 | Akamai Technologies, Inc. | Hybrid content delivery network (cdn) and peer-to-peer (p2p) network |
| FR2917259B1 (en) * | 2007-06-08 | 2009-08-21 | Alcatel Lucent Sas | USE OF A PREFIXED HASH TREE (PHT) FOR LOCATION OF SERVICES WITHIN A POST-TO-POST COMMUNICATION NETWORK |
| CN101399743B (en) * | 2007-09-28 | 2011-09-14 | 华为技术有限公司 | Method and system for searching data in P2P network base on distributed Hash table |
| CN100591028C (en) * | 2007-10-15 | 2010-02-17 | 北京交通大学 | Centralized service based distributed peer-to-peer network implementing method and system |
| CN101645922B (en) * | 2009-04-17 | 2012-09-05 | 中国科学院声学研究所 | CDN network system based on geographical position information encoding and distribution treatment method |
-
2009
- 2009-04-17 CN CN200910082190.5A patent/CN101645922B/en active Active
-
2010
- 2010-04-14 JP JP2012505033A patent/JP5532361B2/en active Active
- 2010-04-14 WO PCT/CN2010/000495 patent/WO2010118638A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101222424A (en) * | 2007-12-24 | 2008-07-16 | 中国电信股份有限公司 | Content distribution network and scheduling method based on content in the network |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106657437A (en) * | 2016-12-02 | 2017-05-10 | 董伟国 | Network communication method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2010118638A1 (en) | 2010-10-21 |
| CN101645922A (en) | 2010-02-10 |
| JP5532361B2 (en) | 2014-06-25 |
| JP2012524307A (en) | 2012-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101645922B (en) | CDN network system based on geographical position information encoding and distribution treatment method | |
| CN100544261C (en) | A kind of data distributing/obtaining method of information Network Based | |
| CN102368776B (en) | Optimization function module of node list in content distribution/delivery network (CDN) | |
| CN111046065B (en) | Extensible high-performance distributed query processing method and device | |
| CN101257396B (en) | System for distributing multi-field content based on P2P technique as well as corresponding method | |
| Li et al. | Efficient and scalable consistency maintenance for heterogeneous peer-to-peer systems | |
| CN110866046B (en) | Extensible distributed query method and device | |
| CN101645919B (en) | Popularity-based duplicate rating calculation method and duplicate placement method | |
| WO2010127618A1 (en) | System and method for implementing streaming media content service | |
| CN110990448B (en) | Distributed query method and device supporting fault tolerance | |
| Hefeeda et al. | On-demand media streaming over the internet | |
| Liu et al. | A novel cache replacement scheme against cache pollution attack in content-centric networks | |
| Thar et al. | Consistent hashing based cooperative caching and forwarding in content centric network | |
| Shen et al. | A lightweight and cooperative multifactor considered file replication method in structured P2P systems | |
| Yu et al. | Granary: A sharing oriented distributed storage system | |
| Jin et al. | Content routing and lookup schemes using global bloom filter for content-delivery-as-a-service | |
| Xia et al. | Algorithms and performance of load-balancing with multiple hash functions in massive content distribution | |
| Cao et al. | Cost-effective replication schemes for query load balancing in DHT-based peer-to-peer file searches | |
| JP4923115B2 (en) | Method, computer program and node for distributing references to objects in a self-organizing distributed overlay network, and self-organizing distributed overlay network | |
| Xun-Yi et al. | P2P Replica-tree based Consistency Maintenance Strategy | |
| Ma et al. | Resource discovery algorithm based on small-world cluster in hierarchical grid computing environment | |
| Li et al. | An improved design of P4P based on distributed tracker | |
| KR100592667B1 (en) | Content replication method in the content distribution management system | |
| Huang et al. | Research of replication mechanism in P2P network | |
| Shen | CORP: A COoperative file Replication Protocol for structured P2P networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210804 Address after: Room 1601, 16th floor, East Tower, Ximei building, No. 6, Changchun Road, high tech Industrial Development Zone, Zhengzhou, Henan 450001 Patentee after: Zhengzhou xinrand Network Technology Co.,Ltd. Address before: 100190 Institute of acoustics, Chinese Academy of Sciences, No. 21 West Fourth Ring Road, Haidian District, Beijing Patentee before: INSTITUTE OF ACOUSTICS, CHINESE ACADEMY OF SCIENCES |
|
| TR01 | Transfer of patent right |