CN101848137B - Load balancing method and system applied to three-layer network - Google Patents
Load balancing method and system applied to three-layer network Download PDFInfo
- Publication number
- CN101848137B CN101848137B CN2009101303176A CN200910130317A CN101848137B CN 101848137 B CN101848137 B CN 101848137B CN 2009101303176 A CN2009101303176 A CN 2009101303176A CN 200910130317 A CN200910130317 A CN 200910130317A CN 101848137 B CN101848137 B CN 101848137B
- Authority
- CN
- China
- Prior art keywords
- url
- layer
- server
- cache
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a load balancing method applied to a three-layer CDN network. The three-layer CDN network comprises a DNS layer, a Proxy layer and a Cache layer. The method comprises the following steps of: analyzing a domain name of a requesting party in the DNS layer; grouping URL requests in the Proxy layer according to URL contents; and orienting the grouped URL requests to a corresponding server in the Cache layer. The invention also discloses a load balancing system applied to the three-layer network. The system comprises a receiving and analyzing module, a URL grouping module and an orientating module. By adopting the load balancing method and the load balancing system of the invention, each of the URL contents is cached only once so as to save disc storage space greatly; moreover, the utilization rate of a CPU can be improved by fully utilizing a plurality of Cache instances.
Description
Technical field
The present invention relates to network communication field, relate in particular to a kind of load-balancing method that is applied to three-layer network.
Background technology
The CDN technology is a kind of not effective means of good problem of internet performance that solves that develops rapidly in recent years.The CDN system can be in real time leads user's request on the nearest service node of user according to being connected of network traffics and each node, load state and to integrated informations such as user's distance and response times again.The full name of CDN is Content Delivery Network, i.e. content distributing network.Generally speaking, content service is deployed in a plurality of places based on a lot of caching servers (Cache servers), is positioned at the edge of network, and is very near with user side.Caching server can be regarded the transparent mirror picture of content supplier's source server as, needs accessed resources thereby can make the user obtain it apace.
In the network application based on the http agreement, the user can come the various network resources on the caching server is conducted interviews through the URL (URL) of input marked network address.Internet resources are limited because visit capacity is very big usually, and network system need be distributed the processing of user's URL access request usually, promptly carries out load balancing, the URL request that receives is transferred on the different servers carried out.
But in the load balancing scheme of prior art; More be only URL to be asked to carry out allocation process according to the load state of server; The problem that this type scheme causes easily is: for a plurality of URL content requests; Possibly same URL content can be forwarded and be cached on a plurality of servers, cause the repeated storage of same content on these servers.The waste that this can cause the server zone memory space reduced the utilance of hardware resource, and hit rate is not high.
Summary of the invention
The objective of the invention is defective, a kind of load-balancing method and system that can efficiently utilize server is provided to prior art.
Load-balancing method provided by the invention comprises: the domain name of a. in reception of DNS layer and parsing user's URL request; B. said URL request is divided into groups according to the URL content at the Proxy layer; C. grouped URL requests is directed to the respective server in the Cache layer.
Preferably, at step b, said URL request is divided into groups at the initial character of Proxy layer according to the URL content.
Preferably, between step b and step c, also comprise: b1. divides into groups according to the URL among the load state set-up procedure b of server.
Preferably, step b1 comprises: the server group that load is surpassed predetermined amount of flow further is divided into one or more groups.
SiteServer LBS provided by the invention comprises: receive parsing module, URL grouping module and orientation module, wherein, receive parsing module, be arranged in the server of DNS layer, be used for receiving and resolve the domain name of terminal use's URL request; The URL grouping module is arranged in the server of Proxy layer, is used for according to the URL content said URL request being divided into groups; Orientation module is arranged in the server of Proxy layer, is used for grouped URL requests is directed to the respective server of Cache layer.
Preferably, the URL grouping module is used at the initial character of Proxy layer according to the URL content said URL request being divided into groups.
Preferably, also comprise second grouping module, said second grouping module is arranged in the server of Proxy layer, and the server group that is used for load is surpassed predetermined amount of flow further is divided into one or more groups.
Preferably, said second grouping module is arranged in the server of Proxy layer, and the server group that is used for load is surpassed predetermined amount of flow further is divided into one or more groups.
Adopt load-balancing method of the present invention and system, in whole Cache servers crowds, each URL content buffer memory has once been saved disk storage space greatly; And, can make full use of a plurality of Cache instances, thus the utilance of the CPU that improves.The present invention can carry out unique location based on URL, with same URL content only on the Cache of correspondence buffer memory once, thereby improved hit rate greatly and solved owing to repeat the big problem of disk consumption that buffer memory brings.
Description of drawings
Fig. 1 is the flow chart of the load-balancing method of the preferred embodiment for the present invention;
Fig. 2 is the structural representation of the SiteServer LBS of the preferred embodiment for the present invention;
Fig. 3 is the network topological diagram of the SiteServer LBS of the preferred embodiment for the present invention.
Embodiment
As shown in Figure 1, according to invention load-balancing method that preferred implementation provides,, at first receive and resolve the domain name in the terminal use URL request, thereby in numerous CDN nodes, select optimum CDN node at the DNS layer in step 100.Wherein, optimum CDN node is meant nearest from the terminal use of request, as to be most suited to handle this request certain CDN node or a plurality of (generally be two, can backup each other) CDN node.For example, according to the Internet ICP/IP protocol, certain user moves a browser on local host, and the user will send to the HTTP request message web server host of www.xyz.com by name, and browser must be known the IP address of www.xyz.com earlier.Basically every main frame all moves the client that DNS uses, and browser extracts host name from URL, host name is passed to the DNS applications client on the local host.So the DNS client is sent the DNS query messages to www.xyz.com to certain DNS service.And " authority " dns server of domain name www.xyz.com (in the present invention, being certain server that is positioned at CDN network DNS layer) finally resolves to corresponding IP address.Then, browser is opened a TCP connection to the http server that is positioned at this IP address.Because the DNS client is general and user's machine in the consolidated network zone.So position that can the basic fixed position user.In the CDN network platform, the selection algorithm of DNS node layer is considered following three parameters usually: 1) this DNS client's position is nearest in which node; 2) the current availability of which node is the highest; 3) the current load of which node is the lightest.
In step 102, distribute said URL request according to the URL content at the Proxy layer.In preferred implementation of the present invention, divide into groups according to the initial character of URL content, for example, with the URL of 0-9 beginning as one group, with the URL of A-G beginning as another group or the like.Can realize above-mentioned distribution through static hash algorithm, according to content URL distributed to make identical URL content be assigned to same Cache servers, thereby can avoid the repeated storage of same content on a plurality of servers.In this preferred implementation; Be to distribute, but it should be appreciated by those skilled in the art that the rule of classification or the allocation algorithm to the URL content that also can adopt other to be fit to according to the static hash algorithm of the initial character utilization of URL content; For example, can divide into groups through the URL class.
In step 104, grouped URL requests is directed to the respective server in the Cache layer, thereby on optimum CDN network node, selects best Cache servers.Each URL request that distributes is sent on the respective cache server in the Cache layer.Because mirror image stores content corresponding on said caching server, therefore, through said content delivery is returned user side, user side can be visited and obtained this content.
Preferably, load-balancing method of the present invention also comprises step 103 between step 102 and the step 104 again: divide into groups according to the URL in the load state set-up procedure 102 of server.This be because; In practical application, some URL is " warm " very, promptly has some or some URL of a lot of user captures; Therefore when these URL are asked Hash to particular server; Can cause " cold excessively " or " overheated " accessing certain server, that is, the load of some server greatly and the load of other servers is minimum.
Therefore, in step 104, preferably can dynamically adjust the classification of URL.For example: in step 102; Former URL type of comprising resolved into 0-9, A-Z access group; When the visit in the A-Z access group " heat " to the F-K access group; Can dynamically adjust F-K is split into two or more sets, for example be split as A-H, J-Z group etc., thereby realize " heat ", " cold " dynamic equalization server access.Particularly; Can adopt dynamic hash algorithm to carry out above-mentioned dynamic adjustment; For example: when to some or certain flowing of access of several groups during greater than a certain setting threshold; Through hash algorithm the key that generates is done a complementation with the number of servers in the configuration of Proxy layer and calculate, obtain Cache servers number, so that back-end server finds the Cache Server of content correspondence rapidly.Other existing algorithms that are fit to also can be used for the dynamic adjustment of step 104, for example, can adopt wheel to be dispatching algorithm (round-robin), the dispatching algorithm of having the right (weighted round-robin) etc.
In the present invention; With static Hash and dynamic hash algorithm Combination application: through configuration URLhash algorithm (being static hash algorithm) on ProxyServer; Make visit capacity according to the URL equiblibrium mass distribution to each Cache Server of Cache layer; After the URL shunting, each URL will only be present among the Cache Server, and the data of every Cache Server all can be different.
With combine with the static hash algorithm spatial cache that can reduce " cold " content and increase the memory space of " heat " content of dynamic hash algorithm.
URL hash algorithm can use the simple algorithm of the crc32 acquiescence of standard.
Simple algorithm java.util.zip.CRC32 class capable of using in java realizes that the c language of simple algorithm is realized as follows:
#define?ngx_hash(key，c)((u_int)key*31?c)
u_int?ngx_hash_key(u_char*data，size_t?len)
{
u_int?i，key；key＝0；
for(i＝0；i＜len；i){
key*＝31；
key＝data[i]；
}
return?key；
}
As shown in Figure 2; A kind of SiteServer LBS 200 that is applied to three-layer network in the preferred implementation has been shown among the figure,, can have found out in conjunction with SiteServer LBS network topological diagram shown in Figure 3; In said system; Comprise: receive parsing module 201, URL grouping module 202 and orientation module 203, in preferred execution mode, also comprise second grouping module 204.
Receive the server that parsing module 201 is arranged in the DNS layer, be used to receive user's URL request.Receive the core of parsing module formula dns server, so that the URL of user's input is resolved to the IP address.
URL grouping module 202 is arranged in the server of Proxy layer, is used for according to the URL content said URL request being divided into groups.URL grouping module 202 is realized that by static hash algorithm dividing into groups promptly is that URL is classified according to its content (for example its initial character), and inhomogeneous URL is directed on the different Cache servers.
Orientation module 203 is arranged in the server of Proxy layer, is used for grouped URL requests is directed to the respective server of Cache layer.Orientation module 203 can realize through the software module on the Proxy server.
Preferably, further also comprising second grouping module 204 in the preferred implementation, said second grouping module 204 is arranged in the server of Proxy layer, and the server group that is used for load is surpassed predetermined amount of flow further is divided into one or more groups.Second grouping module 204 not only realizes by the URLHash algorithm, and comprises that the load of server differentiates algorithm, when the flow of some or certain several server surpasses predetermined value, carries out the URL hash algorithm URL is redirected.
Above-mentioned module preferably realizes through software module, but hardware circuit that also can be fit to or control logic.
Although the present invention describes through above-mentioned preferred implementation, its way of realization is not limited to above-mentioned execution mode.Should be realized that under the situation that does not break away from purport of the present invention those skilled in the art can make different variations and modification to the present invention.
Claims (7)
1. load-balancing method that is applied to three layers of CDN network, said three layers of CDN network comprise DNS layer, Proxy layer and Cache layer, it is characterized in that said method comprises:
A. the reception parsing module that is arranged in the server of DNS layer receives and resolves the domain name of user's URL request at the DNS layer;
B. the URL grouping module that is arranged in the server of Proxy layer is divided into groups to said URL request according to the URL content at the Proxy layer;
C. the orientation module that is arranged in the server of Proxy layer is directed to the respective server of Cache layer with grouped URL requests, so that identical URL content is assigned to same Cache stratum server.
2. method according to claim 1 is characterized in that, at step b, at the initial character of Proxy layer according to the URL content said URL request is divided into groups.
3. method according to claim 1 and 2 is characterized in that, between step b and step c, also comprises:
B1. request is divided into groups according to the URL among the load state set-up procedure b of server.
4. method according to claim 3 is characterized in that step b1 comprises: the Cache stratum server group that load is surpassed predetermined amount of flow further is divided into one or more groups.
5. a SiteServer LBS that is applied to three-layer network is characterized in that, said system comprises: receive parsing module, URL grouping module and orientation module, wherein,
Receive parsing module, be arranged in the server of DNS layer, be used for receiving and resolving the domain name of terminal use URL request;
The URL grouping module is arranged in the server of Proxy layer, is used for according to the URL content said URL request being divided into groups;
Orientation module is arranged in the server of Proxy layer, is used for grouped URL requests is directed to the respective server of Cache layer, so that identical URL content is assigned to same Cache stratum server.
6. SiteServer LBS according to claim 5 is characterized in that, the URL grouping module is used at the initial character of Proxy layer according to the URL content said URL request being divided into groups.
7. according to claim 5 or 6 described SiteServer LBSs; It is characterized in that; Also comprise second grouping module, said second grouping module is arranged in the server of Proxy layer, and the Cache stratum server group that is used for load is surpassed predetermined amount of flow further is divided into one or more groups.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101303176A CN101848137B (en) | 2009-03-26 | 2009-03-26 | Load balancing method and system applied to three-layer network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009101303176A CN101848137B (en) | 2009-03-26 | 2009-03-26 | Load balancing method and system applied to three-layer network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101848137A CN101848137A (en) | 2010-09-29 |
| CN101848137B true CN101848137B (en) | 2012-11-21 |
Family
ID=42772602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009101303176A Expired - Fee Related CN101848137B (en) | 2009-03-26 | 2009-03-26 | Load balancing method and system applied to three-layer network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101848137B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102118433A (en) * | 2010-12-27 | 2011-07-06 | 网宿科技股份有限公司 | Multiple-tier distributed cluster system |
| CN102833791A (en) * | 2011-06-16 | 2012-12-19 | 中兴通讯股份有限公司 | Packet domain content caching system for radio network controller (RNC) and implementing method thereof |
| CN103067439B (en) * | 2012-10-17 | 2016-05-11 | 北京奇虎科技有限公司 | Load-balancing method and system |
| CN102932434B (en) * | 2012-10-17 | 2015-10-14 | 北京奇虎科技有限公司 | A kind of method and device for carrying out load balancing to server |
| CN103916437A (en) * | 2013-01-05 | 2014-07-09 | 中国移动通信集团公司 | File release system, device and method |
| CN104852934A (en) * | 2014-02-13 | 2015-08-19 | 阿里巴巴集团控股有限公司 | Method for realizing flow distribution based on front-end scheduling, device and system thereof |
| CN104202362B (en) * | 2014-08-14 | 2017-11-03 | 上海帝联信息科技股份有限公司 | SiteServer LBS and its content distribution method and device, load equalizer |
| CN104469392B (en) * | 2014-12-19 | 2018-04-20 | 北京奇艺世纪科技有限公司 | A kind of video file storage method and device |
| CN104935648B (en) * | 2015-06-03 | 2018-07-17 | 北京快网科技有限公司 | The CDN system and file of a kind of high performance-price ratio push away in advance, the method for fragment cache memory |
| CN110401657B (en) | 2019-07-24 | 2020-09-25 | 网宿科技股份有限公司 | Processing method and device for access log |
| CN110912769B (en) * | 2019-11-12 | 2021-08-10 | 中移（杭州）信息技术有限公司 | CDN cache hit rate statistical method, system, network device and storage medium |
| CN112068956A (en) * | 2020-08-24 | 2020-12-11 | 北京首汽智行科技有限公司 | Load balancing method based on redis cache and server |
| CN113572828B (en) * | 2021-07-13 | 2024-04-09 | 壹药网科技(上海)股份有限公司 | System for improving client load balance based on URL grouping granularity |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101115016A (en) * | 2007-08-23 | 2008-01-30 | 创新科存储技术（深圳）有限公司 | Method and system for implementing application server load balancing |
| CN101222424A (en) * | 2007-12-24 | 2008-07-16 | 中国电信股份有限公司 | Content distribution network and scheduling method based on content in the network |
-
2009
- 2009-03-26 CN CN2009101303176A patent/CN101848137B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101115016A (en) * | 2007-08-23 | 2008-01-30 | 创新科存储技术（深圳）有限公司 | Method and system for implementing application server load balancing |
| CN101222424A (en) * | 2007-12-24 | 2008-07-16 | 中国电信股份有限公司 | Content distribution network and scheduling method based on content in the network |
Non-Patent Citations (1)
| Title |
|---|
| JP特開2006-72715A 2006.03.16 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101848137A (en) | 2010-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101848137B (en) | Load balancing method and system applied to three-layer network | |
| US10601767B2 (en) | DNS query processing based on application information | |
| JP4789942B2 (en) | Apparatus and method for optimizing connections | |
| US9712484B1 (en) | Managing request routing information utilizing client identifiers | |
| EP3156911B1 (en) | Content management | |
| KR101383905B1 (en) | method and apparatus for processing server load balancing with the result of hash function | |
| US10958501B1 (en) | Request routing information based on client IP groupings | |
| US11064043B2 (en) | System and method for providing an adjunct device in a content distribution network | |
| US20150288652A1 (en) | Systems and Method to Discover Clients Associated with Local Domain Name Server Using Sampling | |
| EP2695358A1 (en) | Selection of service nodes for provision of services | |
| CN102263828A (en) | Load balanced sharing method and equipment | |
| JP2017536606A (en) | Long tail content processing in content distribution networks | |
| CN103312629A (en) | Method, device and system of CDN (content delivery network) flow distribution | |
| CN105610930A (en) | Data optimization method based on DNS (Domain Name Server) | |
| EP4115580B1 (en) | Hostname pre-localization | |
| Chen et al. | Using service brokers for accessing backend servers for web applications | |
| KR20040001355A (en) | Network constructing method for multimedia contents distribution and multimedia contents service method using it | |
| Nakai et al. | DNS-based Load Balancing for Web Services. | |
| Semchedine et al. | DNS-based load balancing with Cache for geographically distributed Web server systems | |
| Zhao et al. | Design and implementation of streaming media server cluster based on FFMpeg | |
| Gao et al. | New architecture and algorithm for webserver cluster based on linux virtual server | |
| Tseng et al. | Revisiting effectiveness of content-aware switching for Web traffic distribution | |
| Wei et al. | Load balancing on the Internet | |
| Hu | Large-capacity, noise-filtering, multi-level, associate memory system | |
| NANO | Network resource identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121121 Termination date: 20200326 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |