CN109376134A - A kind of file distribution dispositions method, device, system and storage medium - Google Patents

A kind of file distribution dispositions method, device, system and storage medium Download PDF

Info

Publication number
CN109376134A
CN109376134A CN201811150273.9A CN201811150273A CN109376134A CN 109376134 A CN109376134 A CN 109376134A CN 201811150273 A CN201811150273 A CN 201811150273A CN 109376134 A CN109376134 A CN 109376134A
Authority
CN
China
Prior art keywords
file
files
blocks
shared node
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811150273.9A
Other languages
Chinese (zh)
Other versions
CN109376134B (en
Inventor
李雪岩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Onething Technology Co Ltd
Original Assignee
Shenzhen Onething Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Onething Technology Co Ltd filed Critical Shenzhen Onething Technology Co Ltd
Priority to CN201811150273.9A priority Critical patent/CN109376134B/en
Publication of CN109376134A publication Critical patent/CN109376134A/en
Application granted granted Critical
Publication of CN109376134B publication Critical patent/CN109376134B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

This application discloses a kind of file distribution dispositions method based on unlimited node, it is different from the single data transmission path that tradition is set up with centralized server, the application can set up a plurality of data transmission path from different unlimited nodes simultaneously, the data downloading experience that user can not only effectively be promoted also eliminates the high bandwidth expense that original service provider is paid.The application further simultaneously discloses a kind of file distribution deployment system, device and computer readable storage medium, has above-mentioned beneficial effect.

Description

A kind of file distribution dispositions method, device, system and storage medium
Technical field
This application involves distributed technical field, in particular to a kind of file distribution deployment side applied to control centre Method, dispatching device, server, a kind of file distribution dispositions method, shared node device, electronics applied to shared node are set It is standby, a kind of file distribution deployment system and computer readable storage medium including control centre and multiple shared nodes.
Background technique
Traditional data downloading mode is referred to alternatively as P2S, Point to Server, is a kind of downloading side of point to server Formula, point are directed toward the request end that server issues downloading request, and server is then the centre data section for being stored with mass data Point, in this way, each user send number to identical, centralization server respectively by the access equipment of oneself It is requested according to downloading, and after the server tells file destination corresponding with each data download request, it will be every by server A file destination backtracking is to corresponding access equipment.
In actual use, since the server of centralization will be responsible for all access for sending data download request Equipment returns to the file destination needed for it, needs to occupy transmission of very more bandwidth resources to carry out data, and each access Equipment is only established single-point with unique centralized server and is connect, and data will also be transmitted by single channel, and usually single channel passes Data transmission bauds under defeated is limited to various influence factors and is also less able to reach the gross data downlink upper limit of user.
To improve this problem, CDN (content distributing network, Content Delivery Network) is used as a kind of solution Means occur, and establish on common network, have two key elements of content storage and distribution.The basic principle is that extensively Using various cache servers, these cache servers are distributed to user and are accessed in area or the network of Relatively centralized, with When family accesses website, the nearest cache server working properly of distance is directed toward in the access of user using global load technology On, it is requested by cache server directly in response to user.Therefore, the superiority and inferiority of CDN technology actual effect directly (adds with cache server Fast node) quantity it is related to Regional Distribution request.
Recently, Duo Jia technology manufacturer is proposed respectively accelerates new method, difference based on the new-type data downloading for accelerating node Big the large-scale of cost setting is spent to accelerate node by CDN producer in tradition, new-type acceleration node is dispersed in more in a manner of lightweight In extensive user group, so that node is accelerated to be built in a reasonable way (incentive mechanism) by extensive user group All parts of the country, along with ever-increasing quantity, can more refine instruct the user of region attachment apart from oneself Quick file download is realized on cache server, due to its quantative attribute, such acceleration node is also referred to as shared node.
Due to it is new-type accelerate node application mode appearance, how the huge hardware resource of quantity sufficient it is vast for user with A kind of more suitable file deployment way offer is better, cost is lower, the faster usage experience of speed of download, is this field skill Art personnel's urgent problem to be solved.
Summary of the invention
The purpose of the application is to provide a kind of file distribution dispositions method, it is intended to make full use of being total to for enormous amount It enjoys node and provides more preferably usage experience for user: control centre each file deployment task received being resolved into multiple sons Deployment task, and be issued in different shared nodes, so that each shared node is according to respective sub- deployment task to center Corresponding blocks of files is obtained in the CDN of change, is finally stored in each complete file in a manner of multiple blocks of files multiple shared In node.Based on this, when control centre receives the data download request that request end is sent, all be stored with just is returned to it The shared node of the blocks of files of target complete file is constituted, so that request end can be simultaneously from multiple shared nodes by multiple files Block fetches local.It is different from the single data transmission path that tradition is set up with centralized server, the application can simultaneously and not Same shared node sets up a plurality of data transmission path, in a parallel fashion instead of original serial mode, can not only have Effect promotes the data downloading experience of user, also eliminates the high bandwidth expense that original service provider is paid.
The another object of the application be the provision of a kind of dispatching device, server, shared node device, electronic equipment, File distribution systems and computer readable storage medium.
To achieve the above object, the application provides firstly a kind of file distribution deployment side applied to control centre Method, this document distributed deployment method include:
File deployment task is received, and the file deployment task is resolved into multiple sub- deployment task, and will be each described Sub- deployment task is handed down to shared node, so that each shared node is saved according to the sub- deployment task received from CDN respectively Point obtains corresponding blocks of files, and reports the file status for being stored in the blocks of files of itself;
The file access request that respond request end is sent is generated to request end return according to each target shared node Response message, so that the request end gets corresponding document block from multiple target shared nodes simultaneously;Wherein, described Target shared node is the node for being stored with file destination block.
Optionally, this document distributed deployment method further include:
Informational table of nodes is generated according to the file status of each blocks of files received, wherein is wrapped in the informational table of nodes Place nodal information containing membership and each blocks of files between file and each blocks of files.
Optionally, the file access request that respond request end is sent is returned to the request end according to the shared section of each target The response message that point generates, comprising:
File destination is determined according to the file access request;
Inquiry constitutes the place nodal information of each file destination block of the file destination in the informational table of nodes, obtains To each target shared node;
The response message is generated according to the access path of each target shared node, and the response message is returned To the request end.
Optionally, this document distributed deployment method further include:
The access temperature of respective file is calculated according to the quantity of the file access request received in prefixed time interval;
The shared node of each blocks of files of storage composition response file is adjusted according to the access temperature of each file Quantity.
Optionally, the shared of each blocks of files of storage composition response file is adjusted according to the access temperature of each file The quantity of node, comprising:
It is high temperature file by the file mark that the access temperature is more than preset heat, and increases storage and constitute the height The quantity of the shared node of each blocks of files of temperature file.
Optionally, increase the quantity of the shared node of each blocks of files of the storage composition high temperature file, comprising:
Never it is stored with the temperature that preset quantity is chosen in the shared node for each blocks of files for constituting the high temperature file Shared node;
The shared section for being stored with each blocks of files for constituting the high temperature file is issued to each temperature shared node The nodal information of point, so that each temperature shared node is obtained according to the nodal information constitutes each of the high temperature file Blocks of files, and it is stored in itself.
To achieve the above object, present invention also provides a kind of dispatching device, which includes:
Task is split and multinode issuance unit, divides for receiving file deployment task, and by the file deployment task Solution is handed down to shared node at multiple sub- deployment task, and by each sub- deployment task, so that each shared node difference Corresponding blocks of files is obtained from CDN node according to the sub- deployment task received, and reports the text for being stored in the blocks of files of itself Part state;
File access request response unit is returned for the file access request that respond request end is sent to the request end The response message that generates according to each target shared node is returned, so that the request end while from multiple target shared nodes Get corresponding document block;Wherein, the target shared node is the node for being stored with file destination block.
Optionally, the dispatching device further include:
Informational table of nodes generation unit, for generating informational table of nodes according to the file status of each blocks of files received, It wherein, include the institute of membership and each blocks of files between file and each blocks of files in the informational table of nodes In nodal information.
Optionally, the file access request response unit includes:
File destination determines subelement, for determining file destination according to the file access request;
Target shared node information inquires subelement, for the inquiry composition file destination in the informational table of nodes Each file destination block place nodal information, obtain each target shared node;
Response message generates and returns subelement, for according to the generation of the access path of each target shared node Response message, and the response message is back to the request end.
Optionally, the dispatching device further include:
Temperature computing unit is accessed, for calculating according to the quantity of the file access request received in prefixed time interval The access temperature of respective file;
Number of nodes temperature adjustment unit constitutes response text for adjusting storage according to the access temperature of each file The quantity of the shared node of each blocks of files of part.
Optionally, the number of nodes temperature adjustment unit includes:
Node increases subelement, for being high temperature file by the file mark that the access temperature is more than preset heat, And increase the quantity of the shared node of each blocks of files of the storage composition high temperature file.
Optionally, the node increase subelement includes:
Temperature shared node choose module, for be never stored with constitutes the high temperature file each blocks of files share The temperature shared node of preset quantity is chosen in node;
Nodal information issues module, constitutes the Gao Reduwen for issuing to be stored with to each temperature shared node The nodal information of the shared node of each blocks of files of part, so that each temperature shared node obtains structure according to the nodal information At each blocks of files of the high temperature file, and it is stored in itself.
To achieve the above object, present invention also provides a kind of server, which includes:
Memory, for storing computer program;
Processor is realized as described in above content when for executing the computer program applied to control centre File distribution dispositions method.
To achieve the above object, present invention also provides a kind of file distribution dispositions method applied to shared node, This document distributed deployment method includes:
Receive the sub- deployment task that control centre issues;Wherein, the sub- deployment task is that the control centre receives File deployment task a part;
Corresponding blocks of files is obtained from CDN node according to the sub- deployment task and is stored in itself, and is reported and be stored in The file status of the blocks of files of itself;
Blocks of files corresponding with the file access request is back to institute by the file access request that respond request end is sent State request end.
Optionally, this document distributed deployment method further include:
When the blocks of files for being stored in itself is when in an abnormal state, to control centre's reporting file block abnormality warnings, So that distribution situation of the control centre according to blocks of files of the blocks of files abnormality warnings adjustment in the abnormality.
Optionally, this document distributed deployment method further include:
The file access request that other shared nodes are sent is responded, blocks of files corresponding with the file access request is returned It is back to the shared node for issuing the file access request.
Optionally, corresponding blocks of files is obtained from CDN using the sub- deployment task and be stored in itself, comprising:
The file fragmentation that the blocks of files got from the CDN is split as multiple default sizes is stored in itself;
The unique features value of each file fragmentation is calculated, and establishes each unique features value and each file point Corresponding relationship between piece, and the corresponding relationship is saved to database.
Optionally, blocks of files corresponding with the file access request is back to the request end, comprising:
Blocks of files corresponding with the file access request is back to the request end in the form of each file fragmentation, and The additional corresponding relationship for taking out each file fragmentation from the database simultaneously, so that the request end is only when each described When the first unique features value and the second unique features value of file fragmentation are consistent, just determine that the blocks of files no data got is wrong Accidentally, the inconsistent blocks of files of existence anduniquess characteristic value is otherwise abandoned, is reacquired from the CDN, and by preset path The abnormal shared node for the blocks of files for reporting existence anduniquess characteristic value inconsistent;
Wherein, the first unique features value is that each file fragmentation determines unique from the corresponding relationship Characteristic value, the second unique features value are that the request end is recalculated when local reception is to each file fragmentation The unique features value arrived.
To achieve the above object, present invention also provides a kind of shared node devices, comprising:
Sub- deployment task receiving unit, the sub- deployment task issued for receiving control centre;Wherein, the sub-portion acting Business is a part for the file deployment task that the control centre receives;
Blocks of files obtains and file status reporting unit, corresponding for being obtained according to the sub- deployment task from CDN node Blocks of files and be stored in itself, and report the file status for being stored in the blocks of files of itself;
File access request response unit will be visited for the file access request that respond request end is sent with the file Ask that the corresponding blocks of files of request is back to the request end.
Optionally, the shared node device further include:
Abnormality warnings reporting unit, the blocks of files for that ought be stored in itself is when in an abnormal state, into the scheduling Heart reporting file block abnormality warnings, so that the control centre is according to blocks of files abnormality warnings adjustment in the abnormal shape The distribution situation of the blocks of files of state.
Optionally, the shared node device further include:
The mutual leaflet member of file between node, the file access request sent for responding other shared nodes will be with the text The corresponding blocks of files of part access request is back to the shared node for issuing the file access request.
Optionally, the blocks of files obtains and file status reporting unit includes:
Fragment storing sub-units, for the blocks of files got from the CDN to be split as to the text of multiple default sizes Part fragment is stored in itself;
Unique features value calculates and corresponding relationship is established and saving subunit, for calculating each file fragmentation only One characteristic value, and establish the corresponding relationship between each unique features value and each file fragmentation, and by the corresponding relationship It saves to database.
Optionally, the file access request response unit includes:
By file fragmentation return and errors validity subelement, for will blocks of files corresponding with the file access request with The form of each file fragmentation is back to the request end, and adds take out each file fragmentation from the database simultaneously Corresponding relationship, so that the request end is only when the first unique features value of each file fragmentation and the second unique features value are equal When consistent, the blocks of files no data mistake got is just determined, otherwise abandon the inconsistent blocks of files of existence anduniquess characteristic value, from The shared section of exception for the blocks of files for being reacquired in the CDN, and reporting existence anduniquess characteristic value inconsistent by preset path Point;
Wherein, the first unique features value is that each file fragmentation determines unique from the corresponding relationship Characteristic value, the second unique features value are that the request end is recalculated when local reception is to each file fragmentation The unique features value arrived.
To achieve the above object, present invention also provides a kind of electronic equipment, which includes:
Memory, for storing computer program;
Processor is realized as described in above content when for executing the computer program applied to shared node File distribution dispositions method.
To achieve the above object, present invention also provides a kind of file distribution systems, which includes:
Multiple sub-portion actings are resolved into for receiving file deployment task, and by the file deployment task in control centre Business, and each sub- deployment task is handed down to shared node;The file access request that respond request end is sent, to the request End returns to the response message generated according to each target shared node;
Multiple shared nodes, the sub- deployment task issued for receiving control centre;According to the sub- deployment task from CDN Node obtains corresponding blocks of files and is stored in itself, and reports the file status for being stored in the blocks of files of itself;Respond request The file access request sent is held, blocks of files corresponding with the file access request is back to the request end.
To achieve the above object, described computer-readable to deposit present invention also provides a kind of computer readable storage medium It is stored with computer program on storage media, realizes when the computer program is executed by processor and is answered as described in above content Divide for the file distribution dispositions method of control centre and/or applied to the file of shared node as described in above content Cloth dispositions method.
Obviously, file distribution dispositions method provided herein, it is intended to make full use of the shared node of enormous amount More preferably usage experience is provided for user: control centre each file deployment task received being resolved into multiple sub-portion actings Business, and be issued in different shared nodes, so that CDN of each shared node according to respective sub- deployment task to centralization It is middle to obtain corresponding blocks of files, finally each complete file is stored in multiple shared nodes in a manner of multiple blocks of files. Based on this, when control centre receives the data download request that request end is sent, is just returned to it and all be stored with composition mesh The shared node of the blocks of files of complete file is marked, so that request end can simultaneously fetch multiple blocks of files from multiple shared nodes It is local.It is different from the single data transmission path that tradition is set up with centralized server, the application can be total to from different simultaneously It enjoys node and sets up a plurality of data transmission path, in a parallel fashion instead of original serial mode, can not only effectively be promoted The data downloading experience of user also eliminates the high bandwidth expense that original service provider is paid.
The application additionally provides a kind of dispatching device, server, shared node device, electronic equipment, file distribution simultaneously Formula system and computer readable storage medium have above-mentioned beneficial effect, and details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of process of the file distribution dispositions method applied to control centre provided by the embodiments of the present application Figure;
Fig. 2 is the process of another file distribution dispositions method for being applied to control centre provided by the embodiments of the present application Figure;
Fig. 3 be the application provided on the basis of embodiment one and two it is a kind of according to adjustment be stored with being total to for same file block Enjoy the flow chart of the quantity of node;
Fig. 4 is that a kind of increase storage provided on the basis of embodiment three constitutes the shared of the blocks of files of high temperature file The flow chart of the method for number of nodes;
Fig. 5 is a kind of process of the file distribution dispositions method applied to shared node provided by the embodiments of the present application Figure;
Fig. 6 is a kind of flow chart of the method for return blocks of files for providing to request end on the basis of embodiment five;
Fig. 7 is a kind of structural block diagram of control centre provided by the embodiments of the present application;
Fig. 8 is a kind of structural block diagram of shared node provided by the embodiments of the present application;
Fig. 9 is that a kind of file distribution based on shared node provided by the embodiments of the present application disposes general frame figure;
Figure 10 is the number between a kind of CDN provided by the embodiments of the present application and each shared node, shared node and shared node According to the schematic diagram of flow direction.
Specific embodiment
The core of the application is to provide a kind of file distribution dispositions method, it is intended to make full use of the shared section of enormous amount Point provides more preferably usage experience for user: each file deployment task received being resolved into multiple sub-portions and is affixed one's name in control centre Task, and be issued in different shared nodes, so that each shared node arrives centralization according to respective sub- deployment task Corresponding blocks of files is obtained in CDN, and each complete file is finally stored in multiple shared nodes in a manner of multiple blocks of files In.Based on this, when control centre receives the data download request that request end is sent, is just returned to it and all be stored with composition The shared node of the blocks of files of target complete file, so that request end can simultaneously take multiple blocks of files from multiple shared nodes It returns local.Be different from the single data transmission path that tradition is set up with centralized server, the application can simultaneously from it is different Shared node sets up a plurality of data transmission path, in a parallel fashion instead of original serial mode, can not only effectively mention The data downloading experience for rising user, also eliminates the high bandwidth expense that original service provider is paid.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Embodiment one
Below in conjunction with Fig. 1, Fig. 1 is a kind of flow chart of file distribution dispositions method provided by the embodiments of the present application, is needed It is noted that the application is to rely on the shared node of enormous amount as bottom hardware, it is shared at these by a setting The CDN for being stored with a large amount of initial data of the control centre of the father of node and a centralization, it is intended to by provided by the present application Technical solution provides a more preferably data downloading experience, scheduling description entire side of the present embodiment station in control centre for user Case, specifically includes the following steps:
S101: receiving file deployment task, and file deployment task resolved into multiple sub- deployment task, and by each sub-portion Acting business is handed down to shared node, so that each shared node obtains phase from CDN node according to the sub- deployment task received respectively The blocks of files answered, and report the file status for being stored in the blocks of files of itself;
Control centre receives the file deployment task issued from more top first, the file deployment task received at this time It is completely that the deployment task, which is intended to allow, is able to respond the file access (downloading) that request end is initiated by the complete file of deployment Request.The complete file deployment task is resolved into multiple sub-portions first by shared node of the present embodiment based on enormous amount Acting business, and these sub- deployment task are handed down to different shared nodes.
If the deployment object of complete file deployment task is complete file, the deployment object of each sub- deployment task is answered For a part of the complete file, it is referred to alternatively as blocks of files.
Specifically, various ways can be based on when a complete file deployment task is decomposed into multiple sub- deployment task, Fixed file block size can be selected as fractionation unit is most descended, correspondingly, sub- deployment task corresponding with this blocks of files Head identification data having the same and different data-bias addresses are answered from other sub- deployment task, to use different numbers The file destination block in complete file is found in corresponding register according to offset address.Certainly, the size of each blocks of files is simultaneously Be not it is fixed, in the size of the influence factor flexible choice blocks of files according to present in concrete application scene.Further, often The size of a blocks of files be also possible in the case where also supporting in practical application scene it is different, but compared to the text of fixed size Part block management difficulty is bigger, but different size of blocks of files is better able to adapt to the remaining memory space of different sharing node.
It, can be according to the son being respectively received after each shared node receives the sub- deployment task issued by control centre Deployment task obtains corresponding blocks of files from the centralization CDN for preserving a large amount of original documents, that is, presses each sub- deployment task Deployment request corresponding document block fetched into each shared node locally store, each shared node also needs periodically to tune Degree center feedback is stored in the file status of the blocks of files of itself, corresponding to take in time when finding that file status is abnormal Reclamation activities avoids the occurrence of request end the phenomenon that the blocks of files got from this shared node can not be viewed normally.
Further, control centre can also generate a nodal information according to the file status that each shared node reports back Table includes the institute of the membership and each blocks of files between each complete file and each blocks of files in the informational table of nodes In nodal information, to receive the data download request for carrying target complete file that request end is sent in control centre When, the node where each blocks of files is determined according to the informational table of nodes, and returning to these nodal informations makes request end according to these Nodal information pulls corresponding blocks of files from corresponding shared node.
Further, when determining that the blocks of files in some shared node occurs abnormal according to the file status reported, The deployment task of identical blocks of files timely can also be issued to another available shared node, and confirm that it is complete from CDN Information after acquisition, in the informational table of nodes that timely updates, it is ensured that the availability of information.
S102: the file access request that respond request end is sent is returned to request end and is generated according to each target shared node Response message so that request end from multiple target shared nodes gets corresponding document block simultaneously.
It is completed for complete file to be split as after multiple blocks of files dispersions are deployed in the operation of different shared nodes in S101, This step description how the file access request that respond request end is sent, with realize to request end provide more preferably downloading use body Test: returning to the response message that generates according to each target shared node to request end, so as to request end according to the response message simultaneously Establish data connection with multiple shared nodes, pulled from each shared node for establishing data connection respectively its storage can For constituting the file destination block of target complete file, more preferably downloading is provided as request end in the way of multi-channel parallel and is used Experience, since each data transmission path has no connection to each other, independently of each other, therefore in the case where the hardware supported of request end It can be obviously improved using such mode and carry out data transmission bring downloading usage experience using only single channel, being easier to will be under entirety It carries speed and is pushed into the theoretical downstream rate upper limit, so that influence factor interference is smaller.
Wherein, target shared node is the node for being stored with file destination block, and each file destination block will be formed and be visited with file Ask request the corresponding consistent file of complete file.
It should be noted that can be used for storing the big file of magnanimity by the node cluster that the shared node of enormous amount forms, And each shared node would generally also be stored with the one or more blocks of files for constituting multiple big files, therefore there is also one The case where blocks of files is stored by multiple shared nodes simultaneously, can be corresponding multiple for a blocks of files when establishing informational table of nodes It in number shared nodes selects one mode to establish, can also be built according to for the pre-set priority of each shared node Vertical sequencing table, to execute the selection of nodal information according to priority and determine operation.
Further, if select multiple shared nodes to store each blocks of files for constituting a complete file, And how many shared nodes is selected to carry out the heat that the storage of plural number can be accessed within the unit time based on the complete file Degree, i.e., access temperature file-based determine whether to issue additional or delete file number of blocks.Further, in each shared node Between also establish have the case where data connection, shared node oneself can also be transferred to execute the realization process of additional issue, reduce CDN Bandwidth pressure and load.
Meanwhile being transmitted for the ease of the data for carrying out shared node and request end and between shared node, it can be with root It requires to carry out subdivided, MTU, Maximum Transmission Unit to blocks of files according to the MTU under practical application scene, most Big transmission unit refers to the maximum data packet size (as unit of byte) that can pass through above a certain layer of communication protocol a kind of, This parameter of maximum transmission unit is usually related (network interface card, serial ports etc.) with communication interface.
Why to carry out it is subdivided, be because each protocol Data Unit packet header and packet tail length be it is fixed, MTU is bigger, then the valid data of the carrying of a protocol Data Unit are longer, and communication efficiency is also higher.Transmit identical use Data packet number needed for user data is also lower.But MTU, which is also not, to be the bigger the better, because MTU is bigger, transmits a data packet It is also bigger to postpone also bigger, bit generation mistakes in data packet probability, therefore to weigh communication efficiency and transmission delay selection Suitable MTU.
By taking Ethernet transmits IPv4 message as an example: the length that MTU is indicated includes the length in the packet header IP, if IP layers or more The length for the data message that protocol layer is sent has been more than MTU, then will carry out fragment to data message in the IP layer of sender, and connect The IP layer of receipts person recombinates the fragment received.
Here illustrate the principle of IP packet fragmentation for a specific example: the MTU value of Ethernet is 1500bytes, it is assumed that The protocol high layer of sender has sent the data message that length is 3008bytes to IP layers, then the message is addition 20bytes's The total length of IP packet is 3028bytes behind the packet header IP, because 3028 > 1500, which will be fragmented, when fragment only Fragment only is carried out to the data on upper layer, is not needed to original IP stem fragment, so the data length of fragment is wanted there was only 3008.
Specific Slicing procedure is as follows:
1. calculating the length=packet header MTU-IP length=1500-20=of IP payload in maximum IP packet first 1480bytes;
2. 3008bytes then will be divided into 3,3008=1480+1480+ according to the length fragment of 1480bytes 48;
3. last sender will add respectively the packet header IP for 3 fragments, retransmited after forming 3 IP packets, the length of 3 IP packets Degree is respectively 1500bytes, 1500bytes and 68bytes.
In real network communication, need to avoid that fragment and recombination occurs as far as possible, because frament reassembling is to internetworking It can be affected, therefore select suitable MTU size necessary to communication performance is improved when data packet is sent.Due to usual The size of the blocks of files separated is larger (tens of MB), and the capacity maximum possible of single MTU also only has 1 or 2MB, and small is only several KB, therefore also need to carry out fragment processing in each blocks of files of shared node actual storage, so that each file fragmentation exists In addition the case where packet header and packet tail can transmit in single MTU without fragment with what is recombinated, it is able to promote file biography as far as possible Defeated speed.
Regardless of whether blocks of files is carried out deconsolidation process again, in order to guarantee what request end was pulled from each shared node Blocks of files is strictly a part of target complete file, with uniformity, and the side of characteristic value can also be calculated by hash algorithm Formula is discriminated whether with uniformity in a manner of double verifications.
Based on the above-mentioned technical proposal, file distribution dispositions method provided by the embodiments of the present application, it is intended to make full use of number Measure huge shared node and provide more preferably usage experience for user: control centre divides each file deployment task received Solution is issued in different shared nodes at multiple sub- deployment task, so that each shared node is affixed one's name to according to respective sub-portion Task obtains corresponding blocks of files into the CDN of centralization, finally stores each complete file in a manner of multiple blocks of files In multiple shared nodes.Based on this, when control centre receives the data download request that request end is sent, just returned to it It is all to be stored with the shared node for constituting the blocks of files of target complete file, so that request end can be simultaneously from multiple shared nodes Multiple blocks of files are fetched into local.It is different from the single data transmission path that tradition is set up with centralized server, the application From different shared nodes a plurality of data transmission path can be set up simultaneously, in a parallel fashion instead of original serial mode, The data downloading experience that user can not only effectively be promoted also eliminates the high bandwidth expense that original service provider is paid With.
Embodiment two
Below in conjunction with Fig. 2, Fig. 2 is the flow chart of another file distribution dispositions method provided by the embodiments of the present application, The present embodiment on the basis of example 1, increases the technical solution that S203 generates informational table of nodes according to file status, with Convenient for directly quickly determining target shared node corresponding with file access request according to the informational table of nodes summarized;Pass through S204,205 and S206 show in particular a kind of technical side that response message is generated using informational table of nodes provided by S203 Case, it should be noted that the technical solution of S203 can be based solely on embodiment one and form one embodiment one, not limit certain The scheme to provide according to S204, S205 and S206 carries out, and the present embodiment overall technical architecture is as follows:
S201: receiving file deployment task, and file deployment task resolved into multiple sub- deployment task, and by each sub-portion Acting business is handed down to shared node, so that each shared node obtains phase from CDN node according to the sub- deployment task received respectively The blocks of files answered, and report the file status for being stored in the blocks of files of itself;
S202: informational table of nodes is generated according to the file status of each blocks of files received;
It include the membership between each complete file and each blocks of files and each text in the informational table of nodes The place nodal information of part block, so as under the data for carrying target complete file that control centre receives that request end is sent Carry request when, the node where each blocks of files is determined according to the informational table of nodes, and return these nodal informations make request end by Corresponding blocks of files is pulled from corresponding shared node according to these nodal informations.
S203: file destination is determined according to file access request;
S204: inquiry constitutes the place nodal information of each file destination block of file destination in informational table of nodes, obtains Each target shared node;
S205: response message is generated according to the access path of each target shared node, and response message is back to request End.
This step is to generate response message according to the access path of each target shared node, so that request end can directly press Data transmission path is established according to access path and each target shared node.
Embodiment three
Below in conjunction with Fig. 3, Fig. 3 be the application provide on that basi of the above embodiments it is a kind of be stored with according to adjustment it is identical The flow chart of the quantity of the shared node of blocks of files, the present embodiment can be applied to embodiment one and embodiment two, it is desirable to provide one Kind adjusts storage according to file access temperature and constitutes the quantity for responding the shared node of each blocks of files of complete file, with more preferable Promotion user usage experience:
S301: the access heat of respective file is calculated according to the quantity of the file access request received in prefixed time interval Degree;
This step is intended to be calculated according to the quantity of the file access request of each complete file received in the unit time Obtain the access temperature of each complete file.For example, the number of identical complete file can will be accessed at 10 minutes directly as it Temperature is accessed, if being accessed 100 times in 10 minutes, access temperature is 100.
S302: the shared node of each blocks of files of storage composition response file is adjusted according to the access temperature of each file Quantity.
On the basis of S301, this step is intended to adjust storage according to the access temperature of each complete file to constitute and respond The quantity of the shared node of each blocks of files of whole file.It specifically includes three kinds of situations: not adjusting, reduces and increase, it can be right respectively Answer preset common heating range, lower than common temperature range lower limit, complete file higher than common temperature range limit.
One kind includes but is not limited to be determined as high temperature file, and increase each blocks of files of the high temperature file of storage composition The scheme of the quantity of shared node can be found in flow chart shown in Fig. 4:
S401: never it is stored with the temperature that preset quantity is chosen in the shared node for each blocks of files for constituting high temperature file Shared node;
S402: the shared node for being stored with each blocks of files for constituting high temperature file is issued to each temperature shared node Nodal information so that each temperature shared node obtains each blocks of files for constituting high temperature file according to nodal information, and is stored in Itself.
Example IV
Below in conjunction with Fig. 5, Fig. 5 is that a kind of file distribution applied to shared node provided by the embodiments of the present application is disposed The flow chart of method, is different from the embodiment one that scheme elaboration is carried out from the angle of control centre, and the present embodiment is station every The angle of a shared node carries out the elaboration of scheme, the specific steps are as follows:
S501: the sub- deployment task that control centre issues is received;
Wherein, the sub- deployment task be embodiment one in by control centre to the complete file deployment task received into The deployment task that row obtains after decomposing.
S502: corresponding blocks of files is obtained from CDN node according to sub- deployment task and is stored in itself, and reports and is stored in The file status of the blocks of files of itself;
S503: blocks of files corresponding with file access request is back to by the file access request that respond request end is sent Request end.
It should be noted that the effect for the file access request that shared node receives herein is to request certain to shared node The request of one blocks of files, rather than the request of the request complete file received in example 1 by control centre.
It further, can also be to control centre when the blocks of files for being stored in shared node itself is when in an abnormal state Reporting file block abnormality warnings, so that control centre is in the blocks of files of abnormality according to blocks of files abnormality warnings adjustment Distribution situation, to guarantee that user can get complete file destination.
Further, each shared when the blocks of files additional issue that other shared node response schedulings center issues is requested Node can also respond the file access request that other shared nodes are sent, and blocks of files corresponding with file access request is returned It reduces the bandwidth pressure of CDN to realize that the blocks of files between node mutually passes to the shared node for issuing file access request and bears It carries.
The content of each section of the present embodiment say in the above-mentioned embodiment using control centre as executing subject Bright, the present embodiment is only illustrated specific implementation flow from another angle, and the explanation content of above-mentioned steps refers to The corresponding portion of the various embodiments described above, details are not described herein again.
Embodiment five
Below in conjunction with Fig. 6, Fig. 6 be on the basis of embodiment five a kind of return blocks of files for providing to the method for request end Flow chart, it is desirable to provide a kind of mode for being more convenient for carrying out data transmission stores and transmits data, and to guarantee the correct of data Property provide a kind of verification scheme, specific step is as follows;
S601: the file fragmentation that the blocks of files got from CDN is split as multiple default sizes is stored in itself;
It is smaller to be split as again size when saving the blocks of files got from CDN by i.e. each shared node for it File fragmentation stores, and carries out storage purpose there are the mode of file fragmentation and is that each file fragmentation one can be used MTU is transmitted, without being split and being recombinated when data are transmitted, improving data transmission efficiency.
S602: calculating the unique features value of each file fragmentation, and establishes between each unique features value and each file fragmentation Corresponding relationship, and corresponding relationship is saved to database;
S603: the file access request that respond request end is sent, will blocks of files corresponding with file access request with each text The form of part fragment is back to request end, and adds the corresponding relationship that each file fragmentation is taken out from database simultaneously, so as to ask It asks end only when the first unique features value of each file fragmentation and the second unique features value are consistent, just determines the text got Otherwise part block no data mistake abandons the inconsistent blocks of files of existence anduniquess characteristic value, reacquires from CDN, and by pre- If the abnormal shared node for the blocks of files that path reports existence anduniquess characteristic value inconsistent.
Wherein, the first unique features value is that the unique features value that each file fragmentation is determined from corresponding relationship (can recognize To be true), the second unique features value is request end to be recalculated only when local reception is to each file fragmentation One characteristic value uses and algorithm identical when obtaining when recalculating.
It should be noted that the precondition that two unique features values compare is to correspond to an identical file point Piece, comparison process can have following two results: (1) determining the blocks of files no data mistake got, establish unique first On the basis of characteristic value and the second unique features value are consistent, illustrate that the file fragmentation being stored in each shared node is not being deposited It changes during storage;(2) the inconsistent blocks of files of existence anduniquess characteristic value is abandoned, is reacquired from CDN, and by pre- If the abnormal shared node for the blocks of files that path reports existence anduniquess characteristic value inconsistent, establish in the first unique features value and the On the basis of two unique features values are not quite identical, illustrate that the file fragmentation being stored in each shared node is sent out in storing process Changing.
Embodiment six
Refer to Fig. 7 below, Fig. 7 is a kind of structural block diagram of dispatching device provided by the embodiments of the present application, in the scheduling The heart is a kind of hardware device for realizing the file distribution dispositions method for being applied to control centre, a pair of with method part one It answers, identical explanation content refers to method part, and details are not described herein, which may include:
Task is split and multinode issuance unit 100, decomposes for receiving file deployment task, and by file deployment task It is handed down to shared node at multiple sub- deployment task, and by each sub- deployment task, so that basis receives each shared node respectively Sub- deployment task obtain corresponding blocks of files from CDN node, and report the file status for being stored in the blocks of files of itself;
File access request response unit 200 is returned for the file access request that respond request end is sent to request end According to the response message that each target shared node generates, so that request end is got from multiple target shared nodes accordingly simultaneously Blocks of files;Wherein, target shared node is the node for being stored with file destination block.
Further, which can also include:
Informational table of nodes generation unit, for generating informational table of nodes according to the file status of each blocks of files received, It wherein, include the place node letter of membership and each blocks of files between file and each blocks of files in informational table of nodes Breath.
Wherein, file access request response unit 200 may include:
File destination determines subelement, for determining file destination according to file access request;
Target shared node information inquires subelement, each target for the inquiry composition file destination in informational table of nodes The place nodal information of blocks of files obtains each target shared node;
Response message generates and returns subelement, for generating response letter according to the access path of each target shared node Breath, and response message is back to request end.
Further, which can also include:
Temperature computing unit is accessed, for calculating according to the quantity of the file access request received in prefixed time interval The access temperature of respective file;
Number of nodes temperature adjustment unit constitutes response file for adjusting storage according to the access temperature of each file The quantity of the shared node of each blocks of files.
Wherein, number of nodes temperature adjustment unit includes:
Node increases subelement, is high temperature file for that will access temperature more than the file mark of preset heat, and increase Storage is added to constitute the quantity of the shared node of each blocks of files of high temperature file.
Wherein, node increase subelement includes:
Temperature shared node chooses module, for being never stored with the shared node for constituting each blocks of files of high temperature file The middle temperature shared node for choosing preset quantity;
Nodal information issues module, has been stored with each text for constituting high temperature file for issuing to each temperature shared node The nodal information of the shared node of part block, so that each temperature shared node is obtained according to nodal information constitutes each of high temperature file Blocks of files, and it is stored in itself.
Embodiment seven
Fig. 8 is referred to below, and Fig. 8 is a kind of structural block diagram of shared node device provided by the embodiments of the present application, this is total Enjoying node apparatus is a kind of hardware device for realizing the file distribution dispositions method for being applied to shared node, with method portion Divide and correspond, identical explanation content refers to method part, and details are not described herein, which can wrap It includes:
Sub- deployment task receiving unit 300, the sub- deployment task issued for receiving control centre;Wherein, sub-portion acting Business is a part for the file deployment task that control centre receives;
Blocks of files obtains and file status reporting unit 400, corresponding for being obtained according to sub- deployment task from CDN node Blocks of files is simultaneously stored in itself, and reports the file status for being stored in the blocks of files of itself;
File access request response unit 500 will be with file access for the file access request that respond request end is sent Corresponding blocks of files is requested to be back to request end.
Further, which can also include:
Abnormality warnings reporting unit, the blocks of files for that ought be stored in itself is when in an abnormal state, in control centre Blocks of files abnormality warnings are reported, so that distribution of the control centre according to blocks of files of the blocks of files abnormality warnings adjustment in abnormality Situation.
Further, which can also include:
The mutual leaflet member of file, the file access request sent for responding other shared nodes will be visited with file between node It asks and corresponding blocks of files is requested to be back to the shared node for issuing file access request.
Wherein, blocks of files obtains and file status reporting unit 400 may include:
Fragment storing sub-units, for the blocks of files got from CDN to be split as to the file point of multiple default sizes Piece is stored in itself;
Unique features value calculates and corresponding relationship is established and saving subunit, for calculating unique spy of each file fragmentation Value indicative, and the corresponding relationship between each unique features value and each file fragmentation is established, and corresponding relationship is saved to database.
Wherein, file access request response unit 500 may include:
By file fragmentation return and errors validity subelement, for will blocks of files corresponding with file access request with each text The form of part fragment is back to request end, and adds the corresponding relationship that each file fragmentation is taken out from database simultaneously, so as to ask It asks end only when the first unique features value of each file fragmentation and the second unique features value are consistent, just determines the text got Otherwise part block no data mistake abandons the inconsistent blocks of files of existence anduniquess characteristic value, reacquires from CDN, and by pre- If the abnormal shared node for the blocks of files that path reports existence anduniquess characteristic value inconsistent;
Wherein, the unique features value that the first unique features value is determined from corresponding relationship for each file fragmentation, second Unique features value is the unique features value that request end is recalculated when local reception is to each file fragmentation.
Based on the above embodiment, present invention also provides a kind of file distribution deployment systems, comprising:
Multiple sub- deployment task are resolved into for receiving file deployment task, and by file deployment task in control centre, and Each sub- deployment task is handed down to shared node;The file access request that respond request end is sent is returned to request end according to each The response message that target shared node generates;
Multiple shared nodes, the sub- deployment task issued for receiving control centre;According to sub- deployment task from CDN node It obtains corresponding blocks of files and is stored in itself, and report the file status for being stored in the blocks of files of itself;Respond request end hair The file access request come, is back to request end for blocks of files corresponding with file access request.
Embodiment eight
In order to deepen the understanding to the application method, the present embodiment will provide a kind of specific text based on practical application scene Part distributed deployment mode, the file distribution deployment way provided in this embodiment based on shared node, general frame please join See Fig. 9, it is preferred that emphasis is by file from the CDN for carrying out data storage in a manner of centralization, according to certain redundancy encryption rule with text The mode of part block is distributed to be deployed in each shared node, main circulation direction are as follows:
1. control centre obtains new file deployment task, and handle complete file corresponding with each new file deployment task It is logically divided into multiple blocks of files, and issue these blocks of files are corresponding in mission dispatching to a part of shared node;
2. this partial sharing node downloads corresponding blocks of files after receiving the task of issuing from the CDN of centralization, and fixed The file status for the blocks of files being stored on itself is fed back to control centre by the phase;
3. control centre receives the file access request from request end (according to the difference of request end main body, also main point To represent the Windows at the end PC, representing the Mobile of mobile terminal and representing the Flash and HTML5 of form web page, wherein HTML5 is called H5 for short), and the information of each blocks of files place node of request end and requested complete file is returned to, with Request end is set to pull blocks of files from multiple nodes according to the information.
Further, the number for the file access request that control centre can also be sent according to the request end received in the unit time Amount judges whether to need to increase file distributing number (quantity for increasing the shared node of storage same file block);It is needed determining Increase file distributing number, the shared node that control centre's control is not stored with file destination block originally is stored with mesh from original It marks and obtains file destination block in the shared node of blocks of files, the pressure to reduce center CDN node by way of internal diffusion (0, Fig. 8 provides a kind of each user for playing objective cloud and plays connection type between objective cloud referring to Figure 1, realizes internal text with cost Part mutually passes).
It is directed to the diversity of file and the difference of node serve ability, two-stage is carried out and splits mode:
The first order: the size of piecemeal, each blocks of files will be 80MB, be divided into multiple Block (blocks of files separated), Block uses the basic unit as scheduling deployment;
The second level: fragment, each file fragmentation (Piece) can correspond to a bitfield, for describing request end institute The file content of storage.Fragment size depends on file size, refers to the following table 1:
1 piecemeal of table, fragment correspond to table
File Size(Blocks) Piece Size Max File Size
1 8KB 80MB
2~16 8KB*16=128KB 1280MB
17~256 8KB*16*16=2048KB 20480MB=20GB
The file of one 80MB, will be up toThe size of a fragment, each fragment is equal 1280KB will be occupied for 8KB, bitfield, can be transmitted in a MTU.
Each Piece can have its corresponding hash value, for by the consistency desired result mode based on hash value come Inspection data converts, specific strategy are as follows:
1. carrying out hash value when Piece is locally downloading from CDN for the first time to be calculated and stored in database;
2. Piece between each shared node can be simultaneously by the hash value of corresponding Piece from database when carrying out internal diffusion Taking-up is sent to opposite end shared node with Piece together;
3. also can be simultaneously by the Hash of corresponding Piece when the request end of user pulls Piece data from shared node Value is taken out from database and is sent to request end together with Piece;
4. often receiving a Piece for receiving end the request end of user (other shared nodes) all can be again in local The calculating of hash value is carried out, and is compared with the hash value received together with Piece, it is consistent and if only if two hash values Just think that the Piece is effective;
5. transmitting terminal can judge whether local data is accurate, will be deleted simultaneously if inconsistent according to the information that receiving end is fed back It re-downloads;
6. for the request end of user, the data of each Piece may derive from different transmitting terminals, if calculate Hash value is inconsistent, and direct discarding is simultaneously re-downloaded from CDN central node, while feeding back to and there is exception to end data, and Shi Jinhang is repaired.
The file distribution deployment way based on shared node provided through this embodiment, so that the downloading of single file It can be greatly improved speed of download, under also avoiding as caused by single-point unstability simultaneously from multiple spot parallel downloading It carries abnormal.Meanwhile by being verified to data transmission, avoids single machine data and be tampered caused downloading file not Consistent exception.Further, the safety of data can also be improved after encrypting to data.
Because situation is complicated, it can not enumerate and be illustrated, those skilled in the art should be able to recognize according to the application The basic skills principle combination actual conditions of offer may exist many examples, in the case where not paying enough creative works, It should within the scope of protection of this application.
Based on the above embodiment, present invention also provides a kind of server and electronic equipment, the server and electronic equipment It may each comprise memory and processor, wherein have computer program in the memory, which calls in the memory Computer program when, step provided by above-described embodiment may be implemented.Certainly, the server and electronic equipment can also wrap Include various necessary network interfaces, power supply and other components etc..
Present invention also provides a kind of computer readable storage mediums, have computer program thereon, the computer program Step provided by above-described embodiment may be implemented when being performed terminal or processor execution.The storage medium may include: U Disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), the various media that can store program code such as magnetic or disk.
Specific examples are used herein to illustrate the principle and implementation manner of the present application, and between each embodiment For progressive relationship, each embodiment focuses on the differences from other embodiments, identical between each embodiment Similar portion may refer to each other.For the device disclosed in the embodiment, reference can be made to corresponding method part illustration.The above reality The explanation for applying example is merely used to help understand the present processes and its core concept.For the ordinary skill people of the art Member for, under the premise of not departing from the application principle, can also to the application, some improvement and modification can also be carried out, these improve and Modification is also fallen into the protection scope of the claim of this application.
It should also be noted that, in the present specification, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning Covering non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes that A little elements, but also other elements including being not explicitly listed, or further include for this process, method, article or The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged Except there is also other identical elements in the process, method, article or equipment for including element.

Claims (17)

1. a kind of file distribution dispositions method, which is characterized in that be applied to control centre, comprising:
File deployment task is received, and the file deployment task is resolved into multiple sub- deployment task, and by each sub-portion Acting business is handed down to shared node, so that each shared node is obtained according to the sub- deployment task received from CDN node respectively Corresponding blocks of files is taken, and reports the file status for being stored in the blocks of files of itself;
The file access request that respond request end is sent returns to the response generated according to each target shared node to the request end Information, so that the request end gets corresponding document block from multiple target shared nodes simultaneously;Wherein, the target Shared node is the node for being stored with file destination block.
2. file distribution dispositions method according to claim 1, which is characterized in that further include:
Informational table of nodes is generated wherein according to the file status of each blocks of files received, includes text in the informational table of nodes The place nodal information of membership and each blocks of files between part and each blocks of files.
3. file distribution dispositions method according to claim 2, which is characterized in that the file that respond request end is sent is visited It asks request, the response message generated according to each target shared node is returned to the request end, comprising:
File destination is determined according to the file access request;
Inquiry constitutes the place nodal information of each file destination block of the file destination in the informational table of nodes, obtains each Target shared node;
The response message is generated according to the access path of each target shared node, and the response message is back to institute State request end.
4. file distribution dispositions method according to claim 1, which is characterized in that further include:
The access temperature of respective file is calculated according to the quantity of the file access request received in prefixed time interval;
The quantity of the shared node of each blocks of files of storage composition response file is adjusted according to the access temperature of each file.
5. file distribution dispositions method according to claim 4, which is characterized in that according to the access of each file Temperature adjustment storage constitutes the quantity of the shared node of each blocks of files of response file, comprising:
It is high temperature file by the file mark that the access temperature is more than preset heat, and increases storage and constitute the high temperature The quantity of the shared node of each blocks of files of file.
6. file distribution dispositions method according to claim 5, which is characterized in that increase storage and constitute the high temperature The quantity of the shared node of each blocks of files of file, comprising:
Never the temperature for being stored with selection preset quantity in the shared node for each blocks of files for constituting the high temperature file is shared Node;
The shared node for being stored with each blocks of files for constituting the high temperature file is issued to each temperature shared node Nodal information, so that each temperature shared node obtains each file for constituting the high temperature file according to the nodal information Block, and it is stored in itself.
7. a kind of dispatching device characterized by comprising
Task is split and multinode issuance unit, resolves into for receiving file deployment task, and by the file deployment task Multiple sub- deployment task, and each sub- deployment task is handed down to shared node, so that each shared node basis respectively The sub- deployment task received obtains corresponding blocks of files from CDN node, and reports the file shape for the blocks of files for being stored in itself State;
File access request response unit returns to root to the request end for the file access request that respond request end is sent According to the response message that each target shared node generates, so that the request end is obtained from multiple target shared nodes simultaneously To corresponding document block;Wherein, the target shared node is the node for being stored with file destination block.
8. a kind of server characterized by comprising
Memory, for storing computer program;
Processor is realized when for executing the computer program and is applied in scheduling as claimed in any one of claims 1 to 6 The file distribution dispositions method of the heart.
9. a kind of file distribution dispositions method, which is characterized in that be applied to shared node, comprising:
Receive the sub- deployment task that control centre issues;Wherein, the sub- deployment task is the text that the control centre receives A part of part deployment task;
Corresponding blocks of files is obtained from CDN node according to the sub- deployment task and is stored in itself, and is reported and be stored in itself Blocks of files file status;
Blocks of files corresponding with the file access request is back to described ask by the file access request that respond request end is sent Ask end.
10. file distribution dispositions method according to claim 9, which is characterized in that further include:
When the blocks of files for being stored in itself is when in an abnormal state, to control centre's reporting file block abnormality warnings, so that The control centre is in the distribution situation of the blocks of files of the abnormality according to blocks of files abnormality warnings adjustment.
11. file distribution dispositions method according to claim 9, which is characterized in that further include:
The file access request that other shared nodes are sent is responded, blocks of files corresponding with the file access request is back to Issue the shared node of the file access request.
12. according to the described in any item file distribution dispositions methods of claim 9 to 11, which is characterized in that utilize the son Deployment task obtains corresponding blocks of files from CDN and is stored in itself, comprising:
The file fragmentation that the blocks of files got from the CDN is split as multiple default sizes is stored in itself;
The unique features value of each file fragmentation is calculated, and is established between each unique features value and each file fragmentation Corresponding relationship, and the corresponding relationship is saved to database.
13. file distribution dispositions method according to claim 12, which is characterized in that will be with the file access request Corresponding blocks of files is back to the request end, comprising:
Blocks of files corresponding with the file access request is back to the request end in the form of each file fragmentation, and simultaneously The additional corresponding relationship for taking out each file fragmentation from the database, so that the request end is only when each file When the first unique features value and the second unique features value of fragment are consistent, the blocks of files no data mistake got is just determined, Otherwise the inconsistent blocks of files of existence anduniquess characteristic value is abandoned, is reacquired from the CDN, and reported and deposited by preset path In the abnormal shared node of the inconsistent blocks of files of unique features value;
Wherein, the first unique features value is the unique features that each file fragmentation is determined from the corresponding relationship Value, the second unique features value request end are recalculated to obtain when local reception is to each file fragmentation Unique features value.
14. a kind of shared node device characterized by comprising
Sub- deployment task receiving unit, the sub- deployment task issued for receiving control centre;Wherein, the sub- deployment task is The a part for the file deployment task that the control centre receives;
Blocks of files obtains and file status reporting unit, for obtaining corresponding text from CDN node according to the sub- deployment task Part block is simultaneously stored in itself, and reports the file status for being stored in the blocks of files of itself;
File access request response unit will be asked for the file access request that respond request end is sent with the file access Corresponding blocks of files is asked to be back to the request end.
15. a kind of electronic equipment characterized by comprising
Memory, for storing computer program;
Processor is realized when for executing the computer program and is applied to share as claim 9 to 13 is described in any item The file distribution dispositions method of node.
16. a kind of file distribution systems characterized by comprising
Multiple sub- deployment task are resolved into for receiving file deployment task, and by the file deployment task in control centre, and Each sub- deployment task is handed down to shared node;The file access request that respond request end is sent is returned to the request end Return the response message generated according to each target shared node;
Multiple shared nodes, the sub- deployment task issued for receiving control centre;According to the sub- deployment task from CDN node It obtains corresponding blocks of files and is stored in itself, and report the file status for being stored in the blocks of files of itself;Respond request end hair The file access request come, is back to the request end for blocks of files corresponding with the file access request.
17. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium Program is realized when the computer program is executed by processor and is applied to control centre as claimed in any one of claims 1 to 6 File distribution dispositions method and/or such as the described in any item file distributions applied to shared node of claim 9 to 13 Formula dispositions method.
CN201811150273.9A 2018-09-29 2018-09-29 File distributed deployment method, device, system and storage medium Active CN109376134B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811150273.9A CN109376134B (en) 2018-09-29 2018-09-29 File distributed deployment method, device, system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811150273.9A CN109376134B (en) 2018-09-29 2018-09-29 File distributed deployment method, device, system and storage medium

Publications (2)

Publication Number Publication Date
CN109376134A true CN109376134A (en) 2019-02-22
CN109376134B CN109376134B (en) 2022-04-01

Family

ID=65403090

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811150273.9A Active CN109376134B (en) 2018-09-29 2018-09-29 File distributed deployment method, device, system and storage medium

Country Status (1)

Country Link
CN (1) CN109376134B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109981780A (en) * 2019-03-27 2019-07-05 深圳市网心科技有限公司 A kind of dispatching method, node device, scheduling system and storage medium
CN110351374A (en) * 2019-07-16 2019-10-18 深圳市网心科技有限公司 A kind of file dispositions method, device and equipment
CN110750809A (en) * 2019-10-22 2020-02-04 陈沫 Block chain financial big data processing method and system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070288588A1 (en) * 2000-04-14 2007-12-13 Wein Joel M Content delivery network (CDN) content server request handling mechanism
CN101159580A (en) * 2007-11-29 2008-04-09 中国电信股份有限公司 Content P2P method and system in content distribution network
CN102143237A (en) * 2011-05-09 2011-08-03 宋健 Grid-based Internet content delivery method and system
CN103581245A (en) * 2012-07-30 2014-02-12 腾讯科技(深圳)有限公司 Content delivery method and system of content delivery network
CN106330521A (en) * 2015-07-02 2017-01-11 无锡市智达网络科技有限公司 CDN (Content Delivery Network) node management system
CN107609140A (en) * 2017-09-20 2018-01-19 郑州云海信息技术有限公司 A kind of method and device of distributive catalogue of document system file access
CN108023812A (en) * 2016-10-31 2018-05-11 华为技术有限公司 The content distribution method and device of cloud computing system, calculate node and system
US10049051B1 (en) * 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070288588A1 (en) * 2000-04-14 2007-12-13 Wein Joel M Content delivery network (CDN) content server request handling mechanism
CN101159580A (en) * 2007-11-29 2008-04-09 中国电信股份有限公司 Content P2P method and system in content distribution network
CN102143237A (en) * 2011-05-09 2011-08-03 宋健 Grid-based Internet content delivery method and system
CN103581245A (en) * 2012-07-30 2014-02-12 腾讯科技(深圳)有限公司 Content delivery method and system of content delivery network
CN106330521A (en) * 2015-07-02 2017-01-11 无锡市智达网络科技有限公司 CDN (Content Delivery Network) node management system
US10049051B1 (en) * 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks
CN108023812A (en) * 2016-10-31 2018-05-11 华为技术有限公司 The content distribution method and device of cloud computing system, calculate node and system
CN107609140A (en) * 2017-09-20 2018-01-19 郑州云海信息技术有限公司 A kind of method and device of distributive catalogue of document system file access

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109981780A (en) * 2019-03-27 2019-07-05 深圳市网心科技有限公司 A kind of dispatching method, node device, scheduling system and storage medium
CN110351374A (en) * 2019-07-16 2019-10-18 深圳市网心科技有限公司 A kind of file dispositions method, device and equipment
CN110351374B (en) * 2019-07-16 2022-04-01 深圳市网心科技有限公司 File deployment method, device and equipment
CN110750809A (en) * 2019-10-22 2020-02-04 陈沫 Block chain financial big data processing method and system

Also Published As

Publication number Publication date
CN109376134B (en) 2022-04-01

Similar Documents

Publication Publication Date Title
CN110336736B (en) Method for realizing shared subscription based on MQTT server cluster
CN110430274A (en) A kind of document down loading method and system based on cloud storage
CN100484069C (en) File data distributing method and relative device
CN101409706B (en) Method, system and relevant equipment for distributing data of edge network
CN106993054A (en) Document distribution method, node and system
CN109376134A (en) A kind of file distribution dispositions method, device, system and storage medium
US6954801B1 (en) Method and system for reducing data volume transferred over a wireless communications network
CN106657287A (en) Data access method and system
KR20130088774A (en) System and method for delivering segmented content
CN105847139A (en) Data transmission method, apparatus and system
CN101729425B (en) Method and equipment for flow sending in VRRP networking
US10691820B1 (en) Real-time distribution of messages via a network with multi-region replication in a hosted service environment
CN102025630A (en) Load balancing method and load balancing system
CN106790675A (en) Load-balancing method, equipment and system in a kind of cluster
CN102984278B (en) Realize the system and method that browser data synchronizes
CN102630378A (en) Individualized data communication on the basis of a descriptive file
CN103685511B (en) Data distributing method, device and system
CN103024085A (en) System and method for processing P2P (peer-to-peer) node request
US10237075B2 (en) Reconstructable content objects
CN106357539A (en) Data acquisition method and equipment
CN105656964B (en) The implementation method and device of data-pushing
CN105872856A (en) Method and system for distributing stream media files
CN110727738A (en) Global routing system based on data fragmentation, electronic equipment and storage medium
CN102857547B (en) The method and apparatus of distributed caching
CN115086330A (en) Cross-cluster load balancing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant