CN101626401A - System and method for splitting and disposing content - Google Patents
System and method for splitting and disposing content Download PDFInfo
- Publication number
- CN101626401A CN101626401A CN200910090793A CN200910090793A CN101626401A CN 101626401 A CN101626401 A CN 101626401A CN 200910090793 A CN200910090793 A CN 200910090793A CN 200910090793 A CN200910090793 A CN 200910090793A CN 101626401 A CN101626401 A CN 101626401A
- Authority
- CN
- China
- Prior art keywords
- file
- server
- terminal
- segment file
- segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a system and a method for splitting and disposing content. The method comprises the following steps: arranging a splitting server and splitting a complete file into more than one section file by the splitting server; respectively disposing the split section files on different super nodes by a disposing server to ensure that a plurality of super nodes commonly store a complete file so as to enable a terminal to acquire the content of the complete file from the plurality of the nodes. The invention can reduce disc occupation during content disposing and guarantee the load balance of the server.
Description
Technical field
The present invention relates to the Web content download technology, be meant the system and method for splitting and disposing content in a kind of point-to-point (P2P, Point to Point) network especially.
Background technology
The P2P download technology is good with its extensibility, the bandwidth availability ratio advantages of higher is occupied important position in the network download field, and the download tool of current trend all supports P2P to download basically, as a sudden peal of thunder, flash get etc.
Using the P2P network of P2P download technology, is a kind of resource distribution utilization and shared system, is devoted to share computer resource and service by the peer switch between the system.The P2P network mainly contains two kinds of topological structures: a kind of is complete decentralization, does not promptly have control centre in whole network, the mutual equality in the residing status of each node; Another kind is to utilize super node, node server and deployment server, promptly in whole network, be provided with super node, node server and deployment server, wherein, the quantity of super node, node server and deployment server, determine according to the network size size, increase but many more resource occupation and costs of can making are set.
Terminal is before file in download, and deployment server can be deployed in a complete file on each super node, makes all to preserve a complete file content on each super node; Terminal is initiated list request to node server when file in download, node server is tabulated from the super node that deployment server obtains the file place, and the super node tabulation of obtaining is returned to a terminal of sending out request; When perhaps the other-end beyond initiating the requesting terminal has downloaded to the required file of this terminal, meeting proactive notification node server, thus, node server can obtain the terminal list at this document place, and terminal list is returned to a terminal of sending out request; Node server with the tabulation of the super node that obtains and terminal list return to send out the terminal of asking after, this terminal just can be obtained a complete file content there from a super node or from an other-end by initiating request to above-mentioned super node or other-end.Second kind of topological structure is because its controllability obtains the generally approval of industry at present.
In the prior art, for the P2P network that adopts second kind of topological structure, deployment server all is complete at the file content of each super node deploy, on a plurality of super nodes, dispose identical file content simultaneously, will repeat to take the disk space of super node, increase the consumption of disk space; And terminal is can obtain a complete file content from certain super node when the file in download content, so, can cause other super nodes to be in idle state, does not demonstrate fully the load balancing of super node.
Summary of the invention
In view of this, main purpose of the present invention is to provide a kind of system and method for splitting and disposing content, can reduce disk space consumption, guarantees the load balancing of super node.
For achieving the above object, technical scheme of the present invention is achieved in that
The invention provides a kind of system of splitting and disposing content, this system comprises section server, deployment server, node server, super node and terminal; Wherein,
The section server, being used for each complete file cutting is an above segment file;
Deployment server, be used to manage all super nodes, be responsible for deployment and the deletion of segment file on different super nodes, preserve the configuration file that is write down when deployment strategy is set, and the deployment information of segment file on each super node, and provide the deployment information of segment file for node server;
Node server is used for the list request that receiving terminal is initiated, and obtains the super node tabulation at segment file place from deployment server, and the super node tabulation at required download segments file place is provided for the terminal that needs the download segments file;
Super node is used to preserve the segment file of deployment, accepts the management of deployment server, and provides corresponding segment file for terminal;
Terminal is used for initiating list request to node server, asks and obtain the super node tabulation at segment file place, and initiates request to segment file place super node, obtains the segment file in the super node.
In the such scheme, described section server also is used to preserve section description document and the whole file description file that is write down after section finishes.
In the such scheme, described node server also is used to receive the notice from segment file place terminal, obtains the terminal list at segment file place, and the terminal list at required download segments file place is provided for the terminal that needs the download segments file;
Described terminal also is used for initiating list request to node server, asks and obtain the terminal list at segment file place, and initiates request to the other-end at segment file place, obtains the segment file in the other-end;
Accordingly, terminal gets access to notifying section point server behind the segment file, and the other-end of downloading the correspondent section file for needs provides required segment file resource.
The present invention also provides a kind of method of splitting and disposing content, and this method comprises:
The section server is an above segment file with the complete file cutting;
Deployment server is deployed in the segment file of institute's cutting respectively on the different super nodes, and preserves the deployment information of segment file;
Terminal is obtained different segment files from different super nodes.
Wherein, this method further comprises: when every segment file cutting finished, the section server was noted the details of cutting automatically, generated section description document and whole file description file and was kept at self.
In the such scheme, described segment file comprises paragraph header and segment body;
Described section description document includes but not limited to: the size of piece in the piece number in segment length, the section, the section;
Described whole file description file includes but not limited to: the total hop count after the complete file cutting, the size of general act.
In the such scheme, deployment server is disposed before the segment file, this method further comprises: deployment server is formulated deployment strategy according to total hop count of complete file, the sum of super node, determines the super node that will dispose and the segment file that will dispose on super node; And record deployment information stores is the configuration file of deployment strategy.
In the such scheme, described deployment strategy is: a segment file is deployed on the above super node.
In the such scheme, described terminal is obtained segment file from super node and is:
Terminal is initiated list request to node server, asks the super node tabulation at required segment file place;
After node server is received list request, inquire about the deployment information of required segment file, and the super node tabulation at this segment file place is returned to node server by deployment server to deployment server;
After node server gets access to the super node tabulation at this segment file place, return the super node tabulation at this segment file place to terminal, and, obtain segment file from described super node by the super node initiation request of terminal in the super node tabulation of being received.
In the such scheme, this method further comprises: terminal is behind self download segments file, and the terminal list at the segment file place that each has been downloaded is known and stored to notifying section point server, node server;
Accordingly, terminal is to the tabulation of node server requesting terminal, and node server sends to the requesting terminal with the terminal list of self storage, and the other-end of terminal in the terminal list of being received initiated request, obtains the segment file from described other-end.
The system and method for splitting and disposing content provided by the present invention, is an above segment file by set section server with the complete file cutting, be deployed in respectively on the different super nodes by the segment file of deployment server again cutting, make common content of preserving a complete file on a plurality of super nodes, thereby make terminal obtain the content of a complete file from a plurality of super nodes.So, the taking of disk space when not only having reduced content disposition guaranteed that also server can realize load balancing better.
The present invention is when disposing segment file, can also adopt redundancy strategy, a segment file is deployed on the above super node simultaneously, even there is super node to produce fault like this, can guarantee that still terminal normally downloads to complete file, improve the reliability that file is downloaded.
Description of drawings
Fig. 1 is splitting and disposing content system configuration schematic diagram among the present invention;
Fig. 2 is the flow chart of splitting and disposing content method among the present invention;
Fig. 3 forms structural representation for stage casing of the present invention file.
Embodiment
Basic thought of the present invention is: the section server is set, is an above segment file by the section server with the complete file cutting, and the segment file that segments is deployed in respectively on the different super nodes by deployment server, make a plurality of super nodes preserve a complete file jointly, thereby make terminal obtain the content of a complete file from a plurality of super nodes.
Deployment server can be disposed a segment file or dispose several segment files simultaneously when deployment file on the super node, same segment file can be deployed on the super node, also can be deployed on a plurality of super nodes.Such as: super node 1,2,3 is arranged, and segment file 1,2,3 can be deployed in segment file 1 on the super node 1, segment file 2 is deployed on the super node 2, segment file 3 is deployed on the super node 3; Also can be at super node 1 deploy segment file 1 and 2, super node 2 deploy segment files 2 and 3, super node 3 deploy segment files 1 and 3, that is: segment file 1 is deployed on the super node 1,3, segment file 2 is deployed on the super node 1,2, and segment file 3 is deployed on the super node 2,3.
After the segment length of each segment file is determined, just can determine hop count according to complete file length, segment length can be set arbitrarily, but will rationally be provided with according to the scale of network, the number of super node and the length of complete file, if it is long that segment length is provided with, can make follow-up deployment underaction; Segment length is provided with too short, and hop count increases, and also can increase the complexity of deployment.
Splitting and disposing content of the present invention system comprises section server, deployment server, node server, super node and terminal as shown in Figure 1; Wherein,
The section server is used for complete file is carried out cutting, is an above segment file with each complete file cutting, preserves the section description document and the whole file description file that are write down after section finishes; Here, preferred each segment file size of dividing is identical;
Deployment server, be used to manage all super nodes, be responsible for deployment and the deletion of segment file on different super nodes, preserve the configuration file that is write down when deployment strategy is set, and the deployment information of segment file on each super node, and provide the deployment information of segment file for node server;
Node server is used for the list request that receiving terminal is initiated, and obtains the super node tabulation at segment file place from deployment server, and the super node tabulation at required download segments file place is provided for the terminal that needs the download segments file;
Further, node server also is used to receive the notice from segment file place terminal, obtains the terminal list at segment file place, and the terminal list at required download segments file place is provided for the terminal that needs the download segments file;
Super node is used to preserve the segment file of deployment, accepts the management of deployment server, and provides corresponding segment file for terminal;
Terminal is used for initiating list request to node server, asks and obtain the super node tabulation at segment file place, and initiates request to segment file place super node, obtains the segment file in the super node;
Here, terminal is initiated request to different super nodes respectively, obtains different segment files, finally obtains a complete file;
Further, terminal also is used for initiating list request to node server, asks and obtain the terminal list at segment file place, and initiates request to the other-end at segment file place, obtains the segment file in the other-end; Accordingly, arbitrarily get access to behind any one segment file can the notifying section point server for terminal, and the other-end of downloading the correspondent section file for needs provides required segment file resource;
Here, described other-end is meant current described terminal self terminal in addition.
Based on said system, the present invention also proposes a kind of method of splitting and disposing content, as shown in Figure 2, may further comprise the steps:
Step 201: the section server is an above segment file with the complete file cutting;
In this step, convenient during for the ease of follow-up deployment segment file, the section server is according to the length of complete file, according to predefined segment length, with the complete file cutting is the identical segment files of some sizes, the length of final stage by cutting after remaining file content decision, not enough not zero filling of part, other each segment length are all equal except that final stage;
For example, segment length is set to section of 25M, is several segment files of one section of 25M with the average cutting of complete file, and the length of final stage can not wait with other sections;
For the not special regulation of segment length, can set arbitrarily; After segment length is determined, can be according to the length of complete file, obtain total hop count of file after the cutting.Generally, segment length rationally is set according to the scale of network, the number of super node and the length of complete file, long if segment length is provided with, can influence the flexibility of follow-up deployment; Segment length is provided with too short, and hop count increases, and also can increase the complexity of deployment.
When every segment file cutting finished, the section server can be noted the details of cutting automatically, generated section description document and whole file description file and was kept at self;
Wherein, each segment file is formed by paragraph header and segment body, and as shown in Figure 3, paragraph header comprises information such as segment number, segment length; Segment body is the body part of segment file, is used to store actual file content, and the file content that sends to terminal promptly comes therefrom.Each segment file is made of several pieces again, and piece is a minimum unit of forming segment file.
The section description document is used for the association attributes of segment file is described, and comprises the piece number in segment length, the section, the information such as size of the interior piece of section; Here, segment length is consistent with segment length information in the paragraph header.
Whole file description file is used for the content of all segment files is done whole the description, comprises the total hop count after the complete file cutting, the information such as size of general act.
Step 202: deployment server is deployed to different super nodes respectively with segment file, and preserves the deployment information of segment file;
In this step, deployment server can be formulated deployment strategy according to the total hop count and the super node sum of complete file, to determine the super node that will dispose and the segment file that will dispose on super node before disposing segment file.For instance, the assumption of complete file is four sections by cutting, having three super nodes can dispose, then can segment file be deployed on each super node successively according to the order of segment file, be that segment file 1 is deployed on the super node 1, segment file 2 is deployed on the super node 2, and segment file 3 is deployed on the super node 3, and segment file 4 is deployed on the super node 1 once more; Perhaps segment file is deployed on the super node randomly, promptly takes out segment file 2 at random and be deployed to super node 1, in remaining segment file, take out a segment file then at random and be deployed on the remaining super node, by that analogy.
After the deployment strategy definition is finished, deployment server can be noted deployment information automatically, and be a file with the information stores that is write down, can be described as the configuration file of deployment strategy, this configuration file is kept in the deployment server, this configuration file generates when creating deployment strategy automatically, preserve the super node that will dispose information and will be in the information of the segment file of corresponding super node deploy.
When disposing segment file, deployment server can be according to the configuration file of deployment strategy generation, and the segment file that will dispose is deployed to different super nodes respectively, and preserves the deployment information of segment file, and so, disk space takies in the time of can reducing content disposition.
Deployment server can be disposed a segment file on the super node when disposing segment file, also can dispose several segment files simultaneously, as long as guarantee behind deployment success, the content of preserving a complete file on a plurality of super nodes jointly gets final product.And if only if all segment file deployment success, deployment task just is counted as merit; If there is segment file to dispose failure, then this time deployment task is promptly counted out.If deployment success, the deployment information of segment file on the super node will be preserved and write down to deployment server, and the deployment information of segment file is provided for node server.
Among the present invention, deployment server also provides a kind of optional redundancy strategy, same segment file is deployed on the more than one super node, so, when the part super node lost efficacy, can obtain required segment file by visiting other super nodes, thereby guarantee to have the complete file content for terminal downloads.
For instance, complete file is 1,2,3,4 totally four segment files by cutting, desire to be deployed on three super nodes, so, when deployment server is disposed segment file, segment file 1,2 can be deployed on the super node 1, segment file 2,3 is deployed on the super node 2, segment file 3,4 is deployed on the super node 3, like this, when one of them super node lost efficacy, can guarantee that still terminal obtains the content of a complete file from two other super node.
Step 203: terminal is obtained different segment files from different super nodes.
In this step, before a certain segment file of terminal downloads, need initiate list request, ask the super node tabulation at required segment file place to node server; After node server is received this list request, inquire about the deployment information of required segment file to deployment server, promptly inquire about this segment file and be deployed on which or which super node, and the super node tabulation at this segment file place is returned to node server by deployment server;
After node server gets access to the super node tabulation at this segment file place, return the super node tabulation at this segment file place to terminal, and initiate request, thereby can obtain segment file from corresponding super node by the super node of terminal in the tabulation of the super node received.By that analogy, terminal can be obtained different segment files from different super nodes, finally obtains complete file, has guaranteed the load balancing of super node;
Further, terminal can also be obtained segment file from the other-end beyond self, in this case, terminal can be tabulated to the node server requesting terminal, node server sends to the requesting terminal with the terminal list of self storage, terminal can be initiated request to the other-end in the terminal list of being received, thereby obtains the segment file from other-end.
Accordingly, terminal is after self downloading certain segment file arbitrarily, meeting proactive notification node server makes node server know which segment file is which or which terminal be loaded with down, thereby obtains and store the terminal list at each segment file place that has been downloaded.
Here, terminal can simultaneously be initiated request to super node and the other-end beyond self, the speed of the speed that when file in download, provides according to resource, and decision is finally from the still other-end file in download content beyond self of super node.
The above is preferred embodiment of the present invention only, is not to be used to limit protection scope of the present invention, all any modifications of being done within the spirit and principles in the present invention, is equal to and replaces and improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1, a kind of system of splitting and disposing content is characterized in that, this system comprises section server, deployment server, node server, super node and terminal; Wherein,
The section server, being used for each complete file cutting is an above segment file;
Deployment server, be used to manage all super nodes, be responsible for deployment and the deletion of segment file on different super nodes, preserve the configuration file that is write down when deployment strategy is set, and the deployment information of segment file on each super node, and provide the deployment information of segment file for node server;
Node server is used for the list request that receiving terminal is initiated, and obtains the super node tabulation at segment file place from deployment server, and the super node tabulation at required download segments file place is provided for the terminal that needs the download segments file;
Super node is used to preserve the segment file of deployment, accepts the management of deployment server, and provides corresponding segment file for terminal;
Terminal is used for initiating list request to node server, asks and obtain the super node tabulation at segment file place, and initiates request to segment file place super node, obtains the segment file in the super node.
2, system according to claim 1 is characterized in that, described section server also is used to preserve section description document and the whole file description file that is write down after section finishes.
3, system according to claim 1, it is characterized in that described node server also is used to receive the notice from segment file place terminal, obtain the terminal list at segment file place, and the terminal list at required download segments file place is provided for the terminal that needs the download segments file;
Described terminal also is used for initiating list request to node server, asks and obtain the terminal list at segment file place, and initiates request to the other-end at segment file place, obtains the segment file in the other-end;
Accordingly, terminal gets access to notifying section point server behind the segment file, and the other-end of downloading the correspondent section file for needs provides required segment file resource.
4, a kind of method of splitting and disposing content is characterized in that, this method comprises:
The section server is an above segment file with the complete file cutting;
Deployment server is deployed in the segment file of institute's cutting respectively on the different super nodes, and preserves the deployment information of segment file;
Terminal is obtained different segment files from different super nodes.
5, method according to claim 4 is characterized in that, this method further comprises: when every segment file cutting finished, the section server was noted the details of cutting automatically, generated section description document and whole file description file and was kept at self.
6, method according to claim 5 is characterized in that, described segment file comprises paragraph header and segment body;
Described section description document includes but not limited to: the size of piece in the piece number in segment length, the section, the section;
Described whole file description file includes but not limited to: the total hop count after the complete file cutting, the size of general act.
7, according to claim 4,5 or 6 described methods, it is characterized in that, deployment server is disposed before the segment file, this method further comprises: deployment server is formulated deployment strategy according to total hop count of complete file, the sum of super node, determines the super node that will dispose and the segment file that will dispose on super node; And record deployment information stores is the configuration file of deployment strategy.
8, method according to claim 7 is characterized in that, described deployment strategy is: a segment file is deployed on the above super node.
According to claim 4,5 or 6 described methods, it is characterized in that 9, described terminal is obtained segment file from super node and is:
Terminal is initiated list request to node server, asks the super node tabulation at required segment file place;
After node server is received list request, inquire about the deployment information of required segment file, and the super node tabulation at this segment file place is returned to node server by deployment server to deployment server;
After node server gets access to the super node tabulation at this segment file place, return the super node tabulation at this segment file place to terminal, and, obtain segment file from described super node by the super node initiation request of terminal in the super node tabulation of being received.
10, according to claim 4,5 or 6 described methods, it is characterized in that this method further comprises: terminal is behind self download segments file, and the terminal list at the segment file place that each has been downloaded is known and stored to notifying section point server, node server;
Accordingly, terminal is to the tabulation of node server requesting terminal, and node server sends to the requesting terminal with the terminal list of self storage, and the other-end of terminal in the terminal list of being received initiated request, obtains the segment file from described other-end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910090793A CN101626401A (en) | 2009-08-06 | 2009-08-06 | System and method for splitting and disposing content |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910090793A CN101626401A (en) | 2009-08-06 | 2009-08-06 | System and method for splitting and disposing content |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101626401A true CN101626401A (en) | 2010-01-13 |
Family
ID=41522082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910090793A Pending CN101626401A (en) | 2009-08-06 | 2009-08-06 | System and method for splitting and disposing content |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101626401A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102291629A (en) * | 2011-08-29 | 2011-12-21 | 中兴通讯股份有限公司 | P2P (peer-to-peer) proxy on-demand system and implementation method applied to IPTV (Internet protocol television) |
| CN103763389A (en) * | 2014-01-28 | 2014-04-30 | 从兴技术有限公司 | Automatic deployment method, service deployment module and application deployment module |
| CN106101174A (en) * | 2016-05-25 | 2016-11-09 | 广州酷狗计算机科技有限公司 | A kind of method and apparatus transmitting data |
| CN112235335A (en) * | 2020-03-04 | 2021-01-15 | 上海曼恒数字技术股份有限公司 | Content transmission method and system |
-
2009
- 2009-08-06 CN CN200910090793A patent/CN101626401A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102291629A (en) * | 2011-08-29 | 2011-12-21 | 中兴通讯股份有限公司 | P2P (peer-to-peer) proxy on-demand system and implementation method applied to IPTV (Internet protocol television) |
| CN103763389A (en) * | 2014-01-28 | 2014-04-30 | 从兴技术有限公司 | Automatic deployment method, service deployment module and application deployment module |
| CN106101174A (en) * | 2016-05-25 | 2016-11-09 | 广州酷狗计算机科技有限公司 | A kind of method and apparatus transmitting data |
| CN112235335A (en) * | 2020-03-04 | 2021-01-15 | 上海曼恒数字技术股份有限公司 | Content transmission method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102447624B (en) | Load balancing method in server cluster, as well as node server and cluster | |
| CN103457960B (en) | The method and system of load document in web game | |
| CN102611735B (en) | A kind of load-balancing method of application service and system | |
| CN103561049A (en) | Method for processing terminal scheduling request, system thereof and device thereof | |
| US9888065B2 (en) | Defragmenting clusters with reserved resources | |
| CN111880936B (en) | Resource scheduling method, device, container cluster, computer equipment and storage medium | |
| US8250171B2 (en) | Content delivery apparatus, content delivery method, and content delivery program | |
| CN103312752B (en) | Point to point network information dispensing method, download node, index server and system | |
| CN101626401A (en) | System and method for splitting and disposing content | |
| US20220137876A1 (en) | Method and device for distributed data storage | |
| CN101753404A (en) | Network resource downloading method | |
| CN101309282A (en) | Method, server and system for document transmission in local area network | |
| CN104967873A (en) | Streaming live scheduling method, system and scheduling server | |
| CN105641934A (en) | Method and device for switching player characters in one account in real time | |
| CN103841206A (en) | Thematic message subscripting and releasing method | |
| CN110035306A (en) | Dispositions method and device, the dispatching method and device of file | |
| CN108243239A (en) | A kind of method, apparatus, electronic equipment and system that web application service is provided | |
| WO2015010197A1 (en) | Method and apparatus for providing redundant data access | |
| CN106960011A (en) | Metadata of distributed type file system management system and method | |
| CN104202386A (en) | High-concurrency distributed file system and secondary load balancing method thereof | |
| CN102970349B (en) | A kind of memory load equalization methods of DHT network | |
| CN102508786B (en) | Chip design method for optimizing space utilization rate and chip thereof | |
| CN110309229A (en) | The data processing method and distributed system of distributed system | |
| CN111177160B (en) | Service updating method, device, server and medium | |
| CN113382065A (en) | File downloading method, device, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20100113 |