CN105577830A - Optimal selection method and system for downloading list based on statistics - Google Patents
Optimal selection method and system for downloading list based on statistics Download PDFInfo
- Publication number
- CN105577830A CN105577830A CN201610072896.3A CN201610072896A CN105577830A CN 105577830 A CN105577830 A CN 105577830A CN 201610072896 A CN201610072896 A CN 201610072896A CN 105577830 A CN105577830 A CN 105577830A
- Authority
- CN
- China
- Prior art keywords
- download
- user
- resource
- list
- downloading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Abstract
The invention relates to an optimal selection method and system for downloading list based on statistics, and belongs to the field of network data transmission. The invention realizes an optimization scheme of resource downloading in the Internet environment. The method comprises the following steps: recording relevant information of user resource downloading and storing the relevant information in a database to serve as basic data of statistic analysis; and in a subsequent user resource downloading process, carrying out statistic analysis according to the recorded stored in the database, and calculating an optimal downloading list for the downloading of the user. The optimal selection method provided by the invention can be used for quickly matching the optimal downloading path, increasing the utilization rate of network resources, effectively improving the resource downloading speed and shortening the resource downloading time.
Description
Technical field
The present invention relates to a kind of download list method for optimizing and system of Corpus--based Method, belong to field of network data transmission, be mainly used in being integrated with in the information system of Internet resources download.
Background technology
Along with computer technology and the propelling of cybertimes, under Internet resources, be loaded in productive life process the application obtained widely.Meanwhile, Internet resources are also in swift and violent growth.In so immense network, how these resources are carried out downloads transmission fast and efficiently, thus facilitate user's down loading network resource better faster, become the challenge that of Current resource huge explosion epoch is huge.
Technology conventional is at present as provided multiple resource server, many download address etc., if application number is the Chinese invention patent application of 201180029787.9, it discloses a kind of system for automation Resourse Distribute to comprise: client computer, described client computer is provided with network interface; Multiple Resource Server system, described multiple Resource Server system is provided with network interface; And operations server system, described operations server Operation system setting has network interface.Described operations server system can be configured to practical activity and obtain client parameter from described client computer, from described multiple Resource Server system Gains resources parameter, obtain the operating parameter relevant to described practical activity, and carry out point being equipped with in described multiple Resource Server system according at least some in described client parameter, described resource parameters and described operating parameter and implement described practical activity.
Above-mentioned technology easily causes the problems such as Resourse Distribute is uneven, speed of download is slow, can not effectively help user to realize quick obtaining resource.
Summary of the invention
For overcoming the above problems, the present invention, by large data statistics, obtains dynamic resource list, promotes user resources speed of download.
Concrete, the invention provides a kind of download list method for optimizing of Corpus--based Method, described method comprises the steps:
Step one, the relevant information of recording user downloaded resources is also stored in database;
Step 2, in the process of subsequent user downloaded resources, the record according to being stored in database carries out statistical analysis;
Step 3, calculates optimum download list and downloads for user.
Further, the download list method for optimizing of Corpus--based Method as above, the concrete grammar of described step one is: when resource first time is downloaded by user, return the download list of acquiescence, invoking server interface after user's downloaded resources, is stored into the time started in downloading task, end time, speed of download, user network information, the Resource Server network information in database.
Further, the download list method for optimizing of Corpus--based Method as above, the concrete grammar of described step 2 is: add up according to the user network information recorded in database and the corresponding Resource Server network information, and packet memory is in database, for statistics download list provides basic data.
Further, the download list method for optimizing of Corpus--based Method as above, the concrete grammar of described step 3 is: when user carries out resource download request, after server receives request, according to the client network information of current request, inquire about the user's Download History that whether there is similar network information in a database; If existed, Download History is sorted, filter out the record composition list that speed, stability are higher, return to active user and download; If there is no, then inquire about the resource downloading record of other users, sort by resource transmission speed, the list after sequence is returned to active user and downloads.
Further, the download list method for optimizing of Corpus--based Method as above, describedly sorts to Download History, filters out speed, record composition list that stability is higher, returns to the concrete grammar that active user carries out downloading to be:
In the entry number found, calculate the download priority valve of certain resource, to every bar record, its weights R is
R=F (probability of success, speed of download, download time, current time)
=probability of success * 60%+ speed of download * 30%+1/ (current time-download time) * 10%
Wherein: the probability of success=this agreement is downloaded number of success/this agreement and downloaded sum
Speed of download=resource size/resource downloading time
R value is larger, downloads priority larger;
According to R value from big to small, corresponding resource address composition is selected to return to the resource downloading list of user.
Moreover the present invention also provides a kind of download list optimum decision system of Corpus--based Method, and described system comprises as lower module:
Logging modle, for recording user downloaded resources relevant information and be stored in database;
Statistical analysis module, in the process of subsequent user downloaded resources, carries out statistical analysis according to the record be stored in database;
Computing module, downloads for user for calculating optimum download list.
Further, the download list optimum decision system of Corpus--based Method as above, the method recording described relevant information of described logging modle is: when resource first time is downloaded by user, return the download list of acquiescence, invoking server interface after user's downloaded resources, is stored into the time started in downloading task, end time, speed of download, user network information, the Resource Server network information in database.
Further, the download list optimum decision system of Corpus--based Method as above, described statistical analysis module is added up according to the user network information recorded in database and the corresponding Resource Server network information, and packet memory is in database, for statistics download list provides basic data.
Further, the download list optimum decision system of Corpus--based Method as above, the computational methods of described computing module are: when user carries out resource download request, after server receives request, according to the client network information of current request, inquire about the user's Download History that whether there is similar network information in a database; If existed, Download History is sorted, filter out the record composition list that speed, stability are higher, return to active user and download; If there is no, then inquire about the resource downloading record of other users, sort by resource transmission speed, the list after sequence is returned to active user and downloads.
Further, the download list optimum decision system of Corpus--based Method as above, described computing module sorts to Download History, filters out the record composition list that speed, stability are higher, returns to the concrete grammar that active user carries out downloading to be:
In the entry number found, calculate the download priority valve of certain resource, to every bar record, its weights R is
R=F (probability of success, speed of download, download time, current time)
=probability of success * 60%+ speed of download * 30%+1/ (current time-download time) * 10%
Wherein: the probability of success=this agreement is downloaded number of success/this agreement and downloaded sum
Speed of download=resource size/resource downloading time
R value is larger, downloads priority larger;
According to R value from big to small, corresponding resource address composition is selected to return to the resource downloading list of user.
Design proposed by the invention can well information system for user provide simply, effective download list, improve the speed of user's Gains resources.
Accompanying drawing explanation
Fig. 1 the present invention is based on the download list method for optimizing of statistics and the flow chart of system.
Fig. 2 is the structure chart of the download list optimum decision system that the present invention is based on statistics.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in detail.
The present invention, by large data statistics, obtains dynamic resource list, promotes user resources speed of download.Such as, multiple user downloaded resource A, and resource A leaves server 1, server 2, server 3 in.When user's second needs downloaded resources A, system sorts according to the resource address of record data to each server of each user downloaded resources A, will for speed of download faster resource address preferentially return to user's second.
As shown in Figure 1, the download list method for optimizing of Corpus--based Method provided by the present invention comprises the steps:
Step one, when resource first time is downloaded by user, return the download list of acquiescence, invoking server interface after user's downloaded resources, the time started in downloading task, end time, speed of download, user network information, the Resource Server network information are stored in database.
In the process, set up the logical construction of resource downloading information storage medium, this structure must can reflect the relevant information of resource downloading.As: client network and the Resource Server network information downloaded, the download list information etc. of each resource.
Step 2, after resource is downloaded, by the every terms of information in resource downloading process stored in database, comprises the process of successful information and failed download retry, by the corresponding record of different download protocols, and write into Databasce.That is to say and each DownloadDetail is recorded in database, become one or more record.
Step 3, when other users download resource, by Download History in query statistic database, returns the resource downloading list relative to active user's optimum.
In the present invention, setting N represents the last record downloaded for n time of resource, and in practice, n looks for the algorithm of N bar record as follows:
Look for forward from the last item record, at least find often kind minimumly to find 1 record to 3 kinds of download protocols (ftp, http, cloud store), look at most 5, but always search number and be no more than 30.
Comprise different download protocols for needing, its principle is, the file that in general single transmission one is static, be difficult to weigh the efficiency of these three kinds of download protocols, it is generally acknowledged that FTP transmission is faster concerning single binary file, when transmitting multiple file, http, cloud store advantageously.Affect the many factors of speed of download, therefore select which kind of agreement on earth, can with reference to the download statistics data of this resource.
Download Ftp, network manager can distribute specific port for Ftp sometimes, and at some time, because these ports of reason such as safety can be turned off temporarily, therefore when downloading, also should with reference to the information of failed download.Although it is very fast that successful hourly velocity is downloaded in such as certain address, for various reasons (such as port is sealed), failed probability is higher, then when downloading, its weights are not high yet.
In the entry number found, calculate the download priority valve of certain resource, to every bar record, its weights R is
R=F (probability of success, speed of download, download time, current time)
=probability of success * 60%+ speed of download * 30%+1/ (current time-download time) * 10%
Wherein: the probability of success=this agreement is downloaded number of success/this agreement and downloaded sum
Speed of download=resource size/resource downloading time
R value is larger, downloads priority larger.
According to R value from big to small, corresponding resource address composition is selected to return to the resource downloading list of user.
User receives resource downloading list, according to the Article 1 address in download list, carries out resource downloading, and as failed download, attempt Article 2 download address, the rest may be inferred, can download successful resource address until find; As all addresses in download list all can not successfully be downloaded, then invoking server interface, to the resource information that server report cannot be downloaded, in the availability of server analysis resource, can safeguard resource according to these records.
As shown in Figure 2, accordingly, the present invention also provides a kind of download list optimum decision system of Corpus--based Method, and described system comprises as lower module:
Logging modle 1, for recording user downloaded resources relevant information and be stored in database;
Statistical analysis module 2, in the process of subsequent user downloaded resources, carries out statistical analysis according to the record be stored in database;
Computing module 3, downloads for user for calculating optimum download list.
Namely each module realizes three steps of method corresponding to this system.
To sum up, main idea of the present invention is 2 key elements.First point, resource downloading address is carried out statistics according to Download History and is drawn; Second point, no matter resource downloading success or not, all need to report relevant information in downloading process to server.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if belong within the scope of the claims in the present invention and equivalent technology thereof to these amendments of the present invention and modification, then the present invention is also intended to comprise these change and modification.
Claims (10)
1. a download list method for optimizing for Corpus--based Method, is characterized in that described method comprises the steps:
Step one, the relevant information of recording user downloaded resources is also stored in database;
Step 2, in the process of subsequent user downloaded resources, the record according to being stored in database carries out statistical analysis;
Step 3, calculates optimum download list and downloads for user.
2. the download list method for optimizing of Corpus--based Method as claimed in claim 1, is characterized in that:
The concrete grammar of described step one is: when resource first time is downloaded by user, return the download list of acquiescence, invoking server interface after user's downloaded resources, is stored into the time started in downloading task, end time, speed of download, user network information, the Resource Server network information in database.
3. the download list method for optimizing of Corpus--based Method as claimed in claim 1, is characterized in that:
The concrete grammar of described step 2 is: add up according to the user network information recorded in database and the corresponding Resource Server network information, and packet memory is in database, for statistics download list provides basic data.
4. the download list method for optimizing of Corpus--based Method as claimed in claim 1, is characterized in that:
The concrete grammar of described step 3 is: when user carries out resource download request, after server receives request, according to the client network information of current request, inquires about the user's Download History that whether there is similar network information in a database; If existed, Download History is sorted, filter out the record composition list that speed, stability are higher, return to active user and download; If there is no, then inquire about the resource downloading record of other users, sort by resource transmission speed, the list after sequence is returned to active user and downloads.
5. the download list method for optimizing of Corpus--based Method as claimed in claim 4, is characterized in that:
Describedly to sort to Download History, filter out speed, record composition list that stability is higher, returning to the concrete grammar that active user carries out downloading is:
In the entry number found, calculate the download priority valve of certain resource, to every bar record, its weights R is
R=F (probability of success, speed of download, download time, current time)
=probability of success * 60%+ speed of download * 30%+1/ (current time-download time) * 10%
Wherein: the probability of success=this agreement is downloaded number of success/this agreement and downloaded sum
Speed of download=resource size/resource downloading time
R value is larger, downloads priority larger;
According to R value from big to small, corresponding resource address composition is selected to return to the resource downloading list of user.
6. a download list optimum decision system for Corpus--based Method, is characterized in that described system comprises as lower module:
Logging modle, for recording user downloaded resources relevant information and be stored in database;
Statistical analysis module, in the process of subsequent user downloaded resources, carries out statistical analysis according to the record be stored in database;
Computing module, downloads for user for calculating optimum download list.
7. the download list optimum decision system of Corpus--based Method as claimed in claim 6, is characterized in that:
The method recording described relevant information of described logging modle is: when resource first time is downloaded by user, return the download list of acquiescence, invoking server interface after user's downloaded resources, is stored into the time started in downloading task, end time, speed of download, user network information, the Resource Server network information in database.
8. the download list optimum decision system of Corpus--based Method as claimed in claim 6, is characterized in that:
Described statistical analysis module is added up according to the user network information recorded in database and the corresponding Resource Server network information, and packet memory is in database, for statistics download list provides basic data.
9. the download list optimum decision system of Corpus--based Method as claimed in claim 6, is characterized in that:
The computational methods of described computing module are: when user carries out resource download request, after server receives request, according to the client network information of current request, inquire about the user's Download History that whether there is similar network information in a database; If existed, Download History is sorted, filter out the record composition list that speed, stability are higher, return to active user and download; If there is no, then inquire about the resource downloading record of other users, sort by resource transmission speed, the list after sequence is returned to active user and downloads.
10. the download list optimum decision system of Corpus--based Method as claimed in claim 9, is characterized in that:
Described computing module sorts to Download History, filters out the record composition list that speed, stability are higher, returns to the concrete grammar that active user carries out downloading to be:
In the entry number found, calculate the download priority valve of certain resource, to every bar record, its weights R is
R=F (probability of success, speed of download, download time, current time)
=probability of success * 60%+ speed of download * 30%+1/ (current time-download time) * 10%
Wherein: the probability of success=this agreement is downloaded number of success/this agreement and downloaded sum
Speed of download=resource size/resource downloading time
R value is larger, downloads priority larger;
According to R value from big to small, corresponding resource address composition is selected to return to the resource downloading list of user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610072896.3A CN105577830A (en) | 2016-02-02 | 2016-02-02 | Optimal selection method and system for downloading list based on statistics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610072896.3A CN105577830A (en) | 2016-02-02 | 2016-02-02 | Optimal selection method and system for downloading list based on statistics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105577830A true CN105577830A (en) | 2016-05-11 |
Family
ID=55887474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610072896.3A Pending CN105577830A (en) | 2016-02-02 | 2016-02-02 | Optimal selection method and system for downloading list based on statistics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105577830A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106850778A (en) * | 2017-01-17 | 2017-06-13 | 无锡清华信息科学与技术国家实验室物联网技术中心 | A kind of multi-source download performance optimization method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101247405A (en) * | 2008-03-27 | 2008-08-20 | 深圳市迅雷网络技术有限公司 | Method, system and device for calculating download time and resource downloading |
| US20080201488A1 (en) * | 1997-06-18 | 2008-08-21 | Brian Kenner | System and method for server-side optimization of data delivery on a distributed computer network |
| CN102685075A (en) * | 2011-03-15 | 2012-09-19 | 腾讯科技（深圳）有限公司 | Network transmission system, server and client |
| CN102855238A (en) * | 2011-06-28 | 2013-01-02 | 腾讯科技（深圳）有限公司 | Method and system for downloading resource data |
-
2016
- 2016-02-02 CN CN201610072896.3A patent/CN105577830A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080201488A1 (en) * | 1997-06-18 | 2008-08-21 | Brian Kenner | System and method for server-side optimization of data delivery on a distributed computer network |
| CN101247405A (en) * | 2008-03-27 | 2008-08-20 | 深圳市迅雷网络技术有限公司 | Method, system and device for calculating download time and resource downloading |
| CN102685075A (en) * | 2011-03-15 | 2012-09-19 | 腾讯科技（深圳）有限公司 | Network transmission system, server and client |
| CN102855238A (en) * | 2011-06-28 | 2013-01-02 | 腾讯科技（深圳）有限公司 | Method and system for downloading resource data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106850778A (en) * | 2017-01-17 | 2017-06-13 | 无锡清华信息科学与技术国家实验室物联网技术中心 | A kind of multi-source download performance optimization method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9787706B1 (en) | Modular architecture for analysis database | |
| US10402424B1 (en) | Dynamic tree determination for data processing | |
| US10922316B2 (en) | Using computing resources to perform database queries according to a dynamically determined query size | |
| US9736270B2 (en) | Automated client/server operation partitioning | |
| US8738645B1 (en) | Parallel processing framework | |
| US10824612B2 (en) | Key ticketing system with lock-free concurrency and versioning | |
| US11210211B2 (en) | Key data store garbage collection and multipart object management | |
| CN103370917A (en) | Message processing method and server | |
| EP4254187A1 (en) | Cross-organization & cross-cloud automated data pipelines | |
| US20230315727A1 (en) | Cost-based query optimization for untyped fields in database systems | |
| CN109842621A (en) | A kind of method and terminal reducing token storage quantity | |
| CN106331160A (en) | Data migration method and system | |
| US7979418B1 (en) | System, method, and computer program product for processing a prefix tree file utilizing a selected agent | |
| US11475011B2 (en) | Pruning cutoffs for database systems | |
| US11853229B2 (en) | Method and apparatus for updating cached information, device, and medium | |
| US10235420B2 (en) | Bucket skiplists | |
| CN105577830A (en) | Optimal selection method and system for downloading list based on statistics | |
| US11334623B2 (en) | Key value store using change values for data properties | |
| US20140040479A1 (en) | Method for a self organizing load balance in a cloud file server network | |
| US10067678B1 (en) | Probabilistic eviction of partial aggregation results from constrained results storage | |
| WO2023005264A1 (en) | Data processing method and apparatus | |
| US11055266B2 (en) | Efficient key data store entry traversal and result generation | |
| CN108628540A (en) | Data storage device and method | |
| US11144593B2 (en) | Indexing structure with size bucket indexes | |
| JP2013101539A (en) | Sampling device, sampling program, and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160511 |