CN105282228A - Method and system for fast file download - Google Patents
Method and system for fast file download Download PDFInfo
- Publication number
- CN105282228A CN105282228A CN201510577329.9A CN201510577329A CN105282228A CN 105282228 A CN105282228 A CN 105282228A CN 201510577329 A CN201510577329 A CN 201510577329A CN 105282228 A CN105282228 A CN 105282228A
- Authority
- CN
- China
- Prior art keywords
- file
- sha
- cloud server
- download
- hash
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and system for fast file download. The method includes that disk files are scanned, and SHA-1 Hash database files are generated, are saved in a disk, and are uploaded to a cloud server; the cloud server compares all of the files on the cloud server one by one regularly and generates incremental packages; the cloud server stores the files in increment relations and the corresponding incremental packages; a client sends a http request to a file download server to download the files; the file download server adds corresponding SHA-1 Hashes of the files to the request header of the http response; and the client queries whether the corresponding SHA-1 Hashes of the files exist in a database file, if yes, the client downloads the files at a fast speed. The method and system can reduce the download of the local existing files, and save users' time and network resources.
Description
Technical field
The present invention relates to file processing technology field, especially relate to a kind of method and system of very fast download file.
Background technology
At present, people download thing, as the file of video, audio frequency and other document class from network or other approach in a large number.But often can there will be the situation of repeated downloads, its tangible this locality of the file namely will downloaded exists, as existed in the hard disk of computer; but due to the difference of path or filename; download tool None-identified is identical file, will again download, and wastes the much time.Also have a kind of situation, be exactly that this locality has existed similar file, the file downloaded is a delta package of this file, when obtaining this delta package, just can recover complete file.
Existing patent (application number: 201410845517.0) disclose a kind of method and the mobile terminal of avoiding repeated downloads, described method comprises: when mobile terminal has download file task, the local download folder of traversal mobile terminal; By binary system comparison method, in the local download folder of comparison, whether there is the file identical with download file; When there being identical file, whether mobile terminal prompting user continues to download.But this patent is only the prompting user whether local existing file of repeated downloads, when downloaded file and local file are only the differences of a delta package, this patent is also think different file, therefore still require that user carries out the download of complete file content, waste user time and Internet resources.
Summary of the invention
Technical problem to be solved by this invention is: when downloading, if when this locality exists existing file resource, optionally performs very fast download or increment and downloads, and reduces taking of user's download time and Internet resources.
In order to solve the problems of the technologies described above, the technical solution used in the present invention is: a kind of method providing very fast download file, comprising:
Scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
All Files on the regular comparison cloud server one by one of cloud server, and generate delta package;
There is the file of increment relation and the delta package of correspondence thereof in cloud server storage;
Client initiates http request to file download service device, with download file;
SHA-1 hash corresponding to file is added in the request header that file download service device responds at http;
Whether client is inquired about in database file exists SHA-1 hash corresponding to described file;
If exist, then the described file of the very fast download of client.
For solving the problem, the invention provides a kind of system of very fast download file, comprising:
Scanning stores upper transmission module, for scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
Comparison generation module, the All Files on the regular comparison cloud server one by one of cloud server, and generate delta package;
, there is the file of increment relation and the delta package of correspondence thereof for cloud server storage in cloud memory module;
Request initiation module, initiates http request to file download service device, with download file for client;
Head adds module, adds SHA-1 hash corresponding to file in the request header responded at http for file download service device;
Enquiry module, inquires about whether there is SHA-1 hash corresponding to described file for client in database file;
Very fast download module, for the described file of the very fast download of client.
Beneficial effect of the present invention is: be different from prior art, and the present invention, by scanning disk file, generates SHA-1 hash data library file, uploads to cloud server; Cloud server comparison file, the delta package corresponding to the file generated that there is increment relation; When client initiate to download with http request time, in request header, add corresponding SHA-1 hash, and inquire about in database file, finally optionally perform very fast download.By the way, the present invention can reduce to the already present file in this locality under, save user time and Internet resources.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of the inventive method embodiment one;
Fig. 2 is the schematic flow sheet of the inventive method embodiment two;
Fig. 3 is the structured flowchart of present system embodiment three;
Fig. 4 is the structured flowchart of present system embodiment four.
Embodiment
By describing technology contents of the present invention in detail, realized object and effect, accompanying drawing is coordinated to be explained below in conjunction with execution mode.
The design of most critical of the present invention is: scan in this locality, obtain the Hash hash of file, when download, whether be present in this locality by the hash contrasting the resource that will download, and optionally perform very fast download or increment download, to reduce file download workload.
Please refer to Fig. 1, the embodiment of the present invention one provides a kind of method of very fast download file, comprising:
S1: scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
S2: the All Files of cloud server regularly one by one on comparison cloud server, and generate delta package;
S3: cloud server storage exists the file of increment relation and the delta package of correspondence thereof;
S4: client initiates http request to file download service device, with download file;
S5: add SHA-1 hash corresponding to file in the request header that file download service device responds at http;
S6: whether client is inquired about in database file exists SHA-1 hash corresponding to described file;
S7: if exist, then the described file of the very fast download of client.
Be different from prior art, the embodiment of the present invention one, by scanning disk file, generates SHA-1 hash data library file, uploads to cloud server; Cloud server comparison file, the delta package corresponding to the file generated that there is increment relation; When client initiate to download with http request time, in request header, add corresponding SHA-1 hash, and inquire about in database file, finally optionally perform very fast download.By the way, the present invention can reduce to the already present file in this locality under, save user time and Internet resources.
As shown in Figure 2, on the basis of embodiment one, after the step S6 of the embodiment of the present invention two, also comprise:
If do not exist, then perform S70: the SHA-1 sending described file corresponding hashes to cloud server; And
S71: the SHA-1 hash corresponding according to described file, searches and whether has corresponding delta package.
Wherein, after step S71, also comprise:
If have, then perform S81: the delta package that file described in client downloads is corresponding, and merge described delta package and described file by delta algorithm;
Otherwise, then S82 is performed: client normally downloads described file.
Wherein, step S1 is specially:
S11: the file being greater than 10M in scanning disk, generates corresponding SHA-1 hash;
S12: according to SHA-1 hash, generates database file in client, and stores in a hard disk;
S13: upload described SHA-1 and hash in cloud server.
Wherein, step S3 is specially:
S31: cloud server storage exists the physical location information of SHA-1 hash corresponding to increment relation file;
S32: cloud server storage exists the physical location information of delta package corresponding to increment relation file.
For convenience of understanding, the present invention is described said method by a following specific embodiment.
On the client, the disk of user to its this locality scans, generate the SHA-1 hash of All Files, wherein SHA-1 hash is a kind of HASH hash, as set up user's hash table for store client place computer in the SHA-1 hash of All Files, be mainly used in follow-up service end coupling.
SHA-1 hash is stored in client, and upload to service end, store in the high in the clouds storage servers such as Baidu's cloud as similar cloud, service end is to the comparison between two of the file on it, delta algorithm is used to generate delta package to the file that there is increment relation, generate delta package, and store all files and delta package thereof.
Wherein, in the specific embodiment of the invention, 10M file is greater than, as file is less than 10M to local disk scanning, its time of normally downloading is inherently very short, therefore can set the size of scanning document according to user's speed of download, if network speed is faster, larger numerical value can be set, as 100M etc.
To the comparison between two of the file in service end, the file now in service end is the All Files that all users oneself have uploaded with other users to service end regular (as every day), the file that on such as Baidu's cloud, all users have uploaded.And use delta algorithm to generate delta package, generate delta package, and there is the associated with of increment relation.As needs storage file 1, the SHA-1 hash 1 of file 2 correspondence and the physical location information of SHA-1 hash 2 and two files, and its delta package is the physical location information of delta file 3; Physical location information wherein on individual PC can be C: the path of certain position; And the store path in service end may be more complicated, because file may exist in multiple service end.
User initiates http request file download service device by client and carrys out download file, adds SHA-1 hash corresponding to file in the head that file download service device responds at http.Then return its SHA-1 hash, client detects its SHA-1 hash and whether is present in the machine, if existed, then completes very fast download.
Otherwise, then continue to search the delta package whether this file exists association, be submitted to service end by SHA-1 hash, find whether there is corresponding delta package according to the SHA-1 hash of user's local file.If had, then download delta package, and by delta algorithm, delta package and local file are merged.Otherwise, then delta package service end not storing this file and this file is represented, then normal download file.
As shown in Figure 3, the embodiment of the present invention three provides a kind of system 100 of very fast download file, comprising:
Scanning stores upper transmission module 110, for scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
Comparison generation module 120, the All Files on the regular comparison cloud server one by one of cloud server, and generate delta package;
, there is the file of increment relation and the delta package of correspondence thereof for cloud server storage in cloud memory module 130;
Request initiation module 140, initiates http request to file download service device, with download file for client;
Add module 150, in the request header responded at http for file download service device, add SHA-1 hash corresponding to file;
Enquiry module 160, inquires about whether there is SHA-1 hash corresponding to described file for client in database file;
Very fast download module 170, for the described file of the very fast download of client.
As shown in Figure 4, the system 100 of the embodiment of the present invention four also comprises:
Delta package searches module 180, hashes to cloud server for the SHA-1 sending described file corresponding; And the SHA-1 hash corresponding according to described file, search and whether have corresponding delta package.
Wherein, also comprise:
Delta package download module 190, for the delta package that file described in client downloads is corresponding, and merges described delta package and described file by delta algorithm;
Normal download module 200, normally downloads described file for client.
Wherein, the upper transmission module 110 of described scanning storage specifically comprises:
Scanning element 111, for being greater than the file of 10M in scanning disk, generates corresponding SHA-1 hash;
Memory cell 112, for according to SHA-1 hash, generates database file in client, and stores in a hard disk;
Uploading unit 113, hashes in cloud server for uploading described SHA-1.
Wherein, described cloud memory module 130 comprises:
, there is the physical location information of SHA-1 hash corresponding to increment relation file for cloud server storage in hash memory cell 131;
, there is the physical location information of delta package corresponding to increment relation file for cloud server storage in delta package memory cell 132.
The foregoing is only embodiments of the invention; not thereby the scope of the claims of the present invention is limited; every equivalents utilizing specification of the present invention and accompanying drawing content to do, or be directly or indirectly used in relevant technical field, be all in like manner included in scope of patent protection of the present invention.
Claims (10)
1. a method for very fast download file, is characterized in that, comprising:
Scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
All Files on the regular comparison cloud server one by one of cloud server, and generate delta package;
There is the file of increment relation and the delta package of correspondence thereof in cloud server storage;
Client initiates http request to file download service device, with download file;
SHA-1 hash corresponding to file is added in the request header that file download service device responds at http;
Whether client is inquired about in database file exists SHA-1 hash corresponding to described file;
If exist, then the described file of the very fast download of client.
2. the method for very fast download file according to claim 1, is characterized in that, client also comprises after inquiring about in database file and whether there is SHA-1 hash corresponding to described file:
If do not exist, then the SHA-1 sending described file corresponding hashes to cloud server;
The SHA-1 hash corresponding according to described file, searches and whether has corresponding delta package.
3. the method for very fast download file according to claim 2, is characterized in that, the SHA-1 hash corresponding according to described file, searches after whether having corresponding delta package, also comprises:
If have, then the delta package that file described in client downloads is corresponding, and merge described delta package and described file by delta algorithm;
Otherwise then client normally downloads described file.
4. the method for very fast download file according to claim 1, is characterized in that, scanning disk file, generates SHA-1 hash data library file, be stored in disk, and the step uploading to cloud server is specially:
Be greater than the file of 10M in scanning disk, generate corresponding SHA-1 hash;
According to SHA-1 hash, generate database file in client, and store in a hard disk;
Upload described SHA-1 to hash in cloud server.
5. the method for very fast download file according to claim 1, it is characterized in that, the step that cloud server storage exists the file of increment relation and the delta package of correspondence thereof is specially:
There is the physical location information of SHA-1 hash corresponding to increment relation file in cloud server storage;
There is the physical location information of delta package corresponding to increment relation file in cloud server storage.
6. a system for very fast download file, is characterized in that, comprising:
Scanning stores upper transmission module, for scanning disk file, generates SHA-1 hash data library file, is stored in disk, and uploads to cloud server;
Comparison generation module, the All Files on the regular comparison cloud server one by one of cloud server, and generate delta package;
, there is the file of increment relation and the delta package of correspondence thereof for cloud server storage in cloud memory module;
Request initiation module, initiates http request to file download service device, with download file for client;
Add module, in the request header responded at http for file download service device, add SHA-1 hash corresponding to file;
Enquiry module, inquires about whether there is SHA-1 hash corresponding to described file for client in database file;
Very fast download module, for the described file of the very fast download of client.
7. the system of very fast download file according to claim 6, is characterized in that, also comprise:
Delta package searches module, hashes to cloud server for the SHA-1 sending described file corresponding; And the SHA-1 hash corresponding according to described file, search and whether have corresponding delta package.
8. the system of very fast download file according to claim 7, is characterized in that, also comprise:
Delta package download module, for the delta package that file described in client downloads is corresponding, and merges described delta package and described file by delta algorithm;
Normal download module, normally downloads described file for client.
9. the system of very fast download file according to claim 6, is characterized in that, described scanning stores upper transmission module and comprises:
Scanning element, for being greater than the file of 10M in scanning disk, generates corresponding SHA-1 hash;
Memory cell, for according to SHA-1 hash, generates database file in client, and stores in a hard disk;
Uploading unit, hashes in cloud server for uploading described SHA-1.
10. the system of very fast download file according to claim 6, it is characterized in that, described cloud memory module comprises:
, there is the physical location information of SHA-1 hash corresponding to increment relation file for cloud server storage in hash memory cell;
, there is the physical location information of delta package corresponding to increment relation file for cloud server storage in delta package memory cell.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510577329.9A CN105282228B (en) | 2015-09-11 | 2015-09-11 | The method and system of very fast downloading file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510577329.9A CN105282228B (en) | 2015-09-11 | 2015-09-11 | The method and system of very fast downloading file |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105282228A true CN105282228A (en) | 2016-01-27 |
| CN105282228B CN105282228B (en) | 2019-01-25 |
Family
ID=55150533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510577329.9A Active CN105282228B (en) | 2015-09-11 | 2015-09-11 | The method and system of very fast downloading file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105282228B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020194205A1 (en) * | 2001-03-16 | 2002-12-19 | Novell, Inc. | Server for synchronization of files |
| CN102185901A (en) * | 2011-04-19 | 2011-09-14 | 北京神州数码思特奇信息技术股份有限公司 | Client message conversion method |
| CN103067980A (en) * | 2013-01-25 | 2013-04-24 | 福建天晴数码有限公司 | Reducing transmission data traffic reducing method based on GPRS (general packet radio service) communication system |
| CN104023070A (en) * | 2014-06-16 | 2014-09-03 | 杜海洋 | File compression method based on cloud storage |
| CN104378397A (en) * | 2013-08-15 | 2015-02-25 | 世纪禾光科技发展（北京）有限公司 | Method and system for issuing incremental updating of program package |
| CN104539696A (en) * | 2014-12-26 | 2015-04-22 | 北京像素软件科技股份有限公司 | Client end incremental updating method and system |
-
2015
- 2015-09-11 CN CN201510577329.9A patent/CN105282228B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020194205A1 (en) * | 2001-03-16 | 2002-12-19 | Novell, Inc. | Server for synchronization of files |
| CN102185901A (en) * | 2011-04-19 | 2011-09-14 | 北京神州数码思特奇信息技术股份有限公司 | Client message conversion method |
| CN103067980A (en) * | 2013-01-25 | 2013-04-24 | 福建天晴数码有限公司 | Reducing transmission data traffic reducing method based on GPRS (general packet radio service) communication system |
| CN104378397A (en) * | 2013-08-15 | 2015-02-25 | 世纪禾光科技发展（北京）有限公司 | Method and system for issuing incremental updating of program package |
| CN104023070A (en) * | 2014-06-16 | 2014-09-03 | 杜海洋 | File compression method based on cloud storage |
| CN104539696A (en) * | 2014-12-26 | 2015-04-22 | 北京像素软件科技股份有限公司 | Client end incremental updating method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105282228B (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107423426B (en) | Data archiving method for block chain block data and electronic equipment | |
| CN102882985B (en) | Based on the file sharing method that cloud stores | |
| US10452484B2 (en) | Systems and methods for time-based folder restore | |
| AU2016382908B2 (en) | Short link processing method, device and server | |
| EP3097674B1 (en) | Content item synchronization by block | |
| US9268716B2 (en) | Writing data from hadoop to off grid storage | |
| US9910895B2 (en) | Push subscriptions | |
| WO2015078231A1 (en) | Method for generating webpage template and server | |
| JP6506374B2 (en) | Cache management | |
| CN103812882A (en) | Method and system for transferring files | |
| CN105302920A (en) | Optimal management method and system for cloud storage data | |
| CN102375837A (en) | Data acquiring system and method | |
| CN102857578A (en) | File uploading method and file uploading system of network drive and network drive client | |
| US9438666B2 (en) | Method and system for resource download | |
| CN103841180A (en) | Network data synchronization method and device based on operating instruction, terminal device and server | |
| CN103747032A (en) | File transmission method, device and system | |
| CN103269302A (en) | Shared file management method and system for instant messaging | |
| CN103605770A (en) | Method and server for generating web page templates | |
| Arer et al. | Efficient big data storage and retrieval in distributed architecture using blockchain and ipfs | |
| US10536524B2 (en) | Systems and methods for content origin administration | |
| CN104023070A (en) | File compression method based on cloud storage | |
| CN104361007A (en) | Browser and processing method for browser favorites | |
| CN104850386A (en) | Software system internationalization resource processing method | |
| CN103034692A (en) | Method for sharing feedback information of webpage, client end and server | |
| CN105282228A (en) | Method and system for fast file download |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |