CN109246229B - Method and device for distributing resource acquisition request - Google Patents
Method and device for distributing resource acquisition request Download PDFInfo
- Publication number
- CN109246229B CN109246229B CN201811141742.0A CN201811141742A CN109246229B CN 109246229 B CN109246229 B CN 109246229B CN 201811141742 A CN201811141742 A CN 201811141742A CN 109246229 B CN109246229 B CN 109246229B
- Authority
- CN
- China
- Prior art keywords
- resource
- target
- target service
- hotspot
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Abstract
The invention discloses a method and a device for distributing resource acquisition requests, and belongs to the technical field of data transmission. The method comprises the following steps: receiving a resource acquisition request of a target service resource, and determining a resource attribute of the target service resource; when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource; when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to the real-time load conditions of all cache servers in the background service system; and sending the resource acquisition request of the target service resource to the target cache server. The invention can improve the resource hit rate of the cached resources and avoid the overload of the cache server.
Description
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a method and an apparatus for distributing resource acquisition requests.
Background
With the continuous development of internet technology, cloud computing services are also rapidly developed. The background service system of the cloud computing service may include a large number of cache servers for storing the service resources, and when the user terminal needs to obtain the service resources from the background service system, the cache servers may provide the user terminal with a download service of the corresponding service resources.
The background service system is generally provided with a scheduling server, and the scheduling server can receive a resource acquisition request submitted by a user terminal to the background service system and provide the resource acquisition request to a selected cache server, so that the cache server responds to the resource acquisition request. When selecting the cache server, the scheduling server may generally select a cache server with the lowest current load or a cache server with the highest current remaining load amount from all available cache servers of the background service system based on a load balancing principle.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
the resource acquisition request of the same service resource may reach different cache servers, on one hand, the cache server which does not store the service resource needs to configure corresponding service resources locally first, so that the request response time is prolonged, on the other hand, one service resource needs to be stored in a plurality of cache servers simultaneously, so that the resources in the background service system are greatly redundant, and a large amount of storage space is occupied.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a method and an apparatus for distributing a resource acquisition request. The technical scheme is as follows:
in a first aspect, a method for distributing resource acquisition requests is provided, the method comprising:
receiving a resource acquisition request of a target service resource, and determining a resource attribute of the target service resource;
when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource;
when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to the real-time load conditions of all cache servers in the background service system;
and sending the resource acquisition request of the target service resource to the target cache server.
Optionally, the determining the resource attribute of the target service resource includes:
determining that the resource attribute of the target service resource is a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request;
and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
Optionally, the determining, according to the number of times of receiving the resource acquisition request of the target service resource within a preset historical duration and a preset hot spot threshold, that the resource attribute of the target service resource is a hot spot resource or a non-hot spot resource includes:
determining a target time interval to which the current moment belongs and a preset number of historical time intervals before the target time interval;
calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all the historical time intervals;
and if the total receiving times are larger than a preset hotspot threshold, determining that the resource attribute of the target service resource is a hotspot resource, otherwise, determining that the resource attribute of the target service resource is a non-hotspot resource.
Optionally, after determining that the resource attribute of the target service resource is a hotspot resource, the method further includes:
acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment;
if the ratio is larger than or equal to a preset hotspot ratio, determining the minimum receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource;
and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as a non-hotspot resource.
Optionally, when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource, including:
and when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
Optionally, when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to the real-time load conditions of all cache servers in the background service system includes:
when the resource attribute is a dynamic resource or a static hot spot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system;
judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the cache servers;
and if so, selecting the alternative cache server as a target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
Optionally, the method further includes:
and if the receiving times of the resource acquisition requests of the target service resources in a preset time length are larger than a preset burst threshold value and the data volume of the target service resources is larger than a preset data threshold value, sending configuration requests of the target service resources to a plurality of cache servers with the residual load quota arranged from high to low in the background service system.
In a second aspect, an apparatus for distributing resource acquisition requests is provided, the apparatus comprising:
the determining module is used for receiving a resource obtaining request of a target service resource and determining the resource attribute of the target service resource;
the selection module is used for selecting a target cache server in a background service system according to the identification information of the target service resource when the resource attribute is a static non-hotspot resource, and selecting the target cache server according to the real-time load condition of all cache servers in the background service system when the resource attribute is a dynamic resource or a static hotspot resource;
and the sending module is used for sending the resource obtaining request of the target service resource to the target cache server.
Optionally, the determining module is specifically configured to:
determining that the resource attribute of the target service resource is a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request;
and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
Optionally, the determining module is specifically configured to:
determining a target time interval to which the current moment belongs and a preset number of historical time intervals before the target time interval; (ii) a
Calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all the historical time intervals;
and if the total receiving times are larger than a preset hotspot threshold, determining that the resource attribute of the target service resource is a hotspot resource, otherwise, determining that the resource attribute of the target service resource is a non-hotspot resource.
Optionally, the determining module is further configured to:
acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment;
if the ratio is larger than or equal to a preset hotspot ratio, determining the minimum receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource;
and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as a non-hotspot resource.
Optionally, the selecting module is specifically configured to:
and when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
Optionally, the selecting module is specifically configured to:
when the resource attribute is a dynamic resource or a static hot spot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system;
judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the cache servers;
and if so, selecting the alternative cache server as a target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
Optionally, the apparatus further includes a configuration module, configured to:
and if the receiving times of the resource acquisition requests of the target service resources in a preset time length are larger than a preset burst threshold value and the data volume of the target service resources is larger than a preset data threshold value, sending configuration requests of the target service resources to a plurality of cache servers with the residual load quota arranged from high to low in the background service system.
In a third aspect, there is provided a scheduling server comprising a processor and a memory, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the method for distributing resource acquisition requests according to the first aspect.
In a fourth aspect, there is provided a computer readable storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to implement the method of distributing resource acquisition requests according to the first aspect.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, a resource acquisition request of a target service resource is received, and the resource attribute of the target service resource is determined; when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource; when the resource attribute is a dynamic resource or a static hot resource, selecting a target cache server according to the real-time load condition of all cache servers in the background service system; and sending the resource acquisition request of the target service resource to the target cache server. Therefore, for static non-hotspot resources, the resource acquisition request can be guided to the cache server which stores corresponding resources, so that the resource hit rate of the cached resources can be improved, the request response time can be reduced, repeated storage in other cache servers is not needed, the storage space is saved, and for dynamic resources and hotspot resources, the resource acquisition request can be guided to the cache server with lower load, so that load balance among a plurality of cache servers can be realized, and overload of part of cache servers is avoided.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a network framework of a background service system according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for distributing a resource acquisition request according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an apparatus for distributing a resource obtaining request according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an apparatus for distributing a resource obtaining request according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a scheduling server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
An execution main body of the method can be a scheduling server of a background service system, the background service system at least comprises one scheduling server and a plurality of cache servers, and a specific network framework can be shown in fig. 1, wherein the scheduling server can receive resource acquisition requests of all user terminals, and then selects a cache server for responding to the resource acquisition request from all the cache servers of the background service system based on a locally preset request distribution strategy. After receiving the resource acquisition request, the cache server may perform feedback based on a pre-stored service resource or a temporarily configured service resource. The scheduling server may include a processor, a memory, and a transceiver, where the processor may be configured to perform a process of distributing the resource obtaining request in the following process, the memory may be configured to store data required in the process and generated data, such as the number of times of receiving the resource obtaining request, the receiving time, and the like, and the transceiver may be configured to receive and transmit data related to the process, such as receiving or transmitting the resource obtaining request, and the like.
The process flow shown in fig. 2 will be described in detail below with reference to the specific embodiments, and the contents may be as follows:
In implementation, a user may request a service resource (taking a target service resource as an example) from a background service system of a cloud computing service through a user terminal, and a scheduling server of the background service system may receive a resource acquisition request of the target service resource sent by the user terminal, and then may determine a resource attribute of the target service resource. Here, determining the resource attribute of the target service resource may include at least determining whether the target service resource belongs to a dynamic resource or a static resource, and determining whether the target service resource belongs to a hot resource or a non-hot resource.
Optionally, the specific processing for distinguishing static resources from dynamic resources and determining hot spot resources in step 201 may be as follows: determining the resource attribute of the target service resource as a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request; and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
In implementation, in the process of determining the resource attribute of the target service resource, the scheduling server may extract a URL of the target service resource carried in the resource acquisition request, and then determine whether the resource attribute of the target service resource is a dynamic resource or a static resource according to a suffix of the URL, specifically, when the suffix of the URL is a character string such as ". php", ". aspx", ". asp", ". jsp", ". do", ". dwr", ". cgi", ". fcgi", ". action", ". ashx", ". axd", ". json", and the like, it may be determined that the resource attribute is a dynamic resource, and in other cases, the resource attribute is a static resource. Then, the scheduling server may obtain the number of times of receiving the resource obtaining request of the target service resource within a preset historical duration (e.g., within the previous 10 minutes), and compare the number of times of receiving with a preset hot spot threshold, thereby determining whether the resource attribute of the target service resource is a hot spot resource or a non-hot spot resource according to the comparison result. Generally, the hot resource is determined when the receiving times are greater than a preset hot threshold, and the non-hot resource is determined when the receiving times are not greater than the preset hot threshold.
Optionally, when counting the number of times of receiving the resource acquisition request, a concept of a time period may be introduced to reduce the workload of statistics, and accordingly, the processing for identifying the hot resource and the non-hot resource may be as follows: determining a target time period to which the current moment belongs and a preset number of historical time periods before the target time period; calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all historical time intervals; and if the total receiving times are larger than a preset hot spot threshold value, determining the resource attribute of the target service resource as the hot spot resource, otherwise, determining the resource attribute of the target service resource as the non-hot spot resource.
In implementation, a technician may set a period for counting the number of times of reception of the resource acquisition request at the scheduling server, so that the scheduling server may count the number of times of reception in a time period, for example, one period may be set to be 10 minutes long, and the scheduling server may count the number of times of reception in one time of 10 minutes, and then store the counted number of times of reception in a time period. Taking the resource obtaining request of the target service resource as an example, the scheduling server may use the first receiving time of the resource obtaining request of the target service resource as the starting point of the first time period of the resource obtaining request, and then count and store the receiving times of the resource obtaining request of the target service resource in the time period every preset time period length. In this way, when receiving the resource acquisition request of the target service resource, the scheduling server may determine the target time period to which the current time belongs and a preset number of historical time periods before the target time period. It is understood that the number of the historical periods is determined by the reference time for judging the hot spot resource and the duration of the single period, for example, if the judgment needs to refer to the number of times of reception within one hour, and the duration of each period is 10 minutes, then the number of the historical periods that can be determined may be 5 (considering the target period) or 6 (not considering the target period). Then, the scheduling server may obtain the number of times of receiving the resource acquisition request of the target service resource counted in all the stored history periods, and determine the number of times of receiving the resource acquisition request of the target service resource in the target period at the same time, thereby adding the number of times of receiving to obtain the total number of times of receiving. If the total reception is greater than the preset hot spot threshold value, the scheduling server can determine that the resource attribute of the target service resource is the hot spot resource, otherwise, the scheduling server can determine that the resource attribute of the target service resource is the non-hot spot resource.
Optionally, in order to ensure the resource hit rate of the background service system, the number of the hot spot resources may be controlled to some extent, and accordingly, after the target service resource is determined to be the hot spot resource, the following processing may also be performed: acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment; if the occupation ratio is larger than or equal to the preset hotspot proportion, determining the minimum value of the receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource; and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as the non-hotspot resource.
In implementation, after determining that the target service resource is a hotspot resource, the scheduling server may first obtain the number of hotspot resources and the number of non-hotspot resources determined within a preset statistical duration before the current time, so as to obtain the occupation ratio of the hotspot resource within the preset statistical duration. If the ratio is greater than or equal to the preset hotspot ratio, it indicates that the preset hotspot threshold value may be set to be lower, and the determined hotspot resources are excessive, the resource hit rate of the background service system will be greatly reduced, so that the minimum value of the number of receptions can be further determined in the number of receptions of the resource acquisition request corresponding to each hotspot resource, and then the size relationship between the number of receptions of the resource acquisition request of the target service resource and the minimum value of the number of receptions is determined. If the number of times of receiving the resource acquisition request of the target service resource is small, the heat degree of the target service resource is considered to be lower than that of the determined hot spot resource, so that the resource attribute of the target service resource can be determined as the non-hot spot resource again.
In implementation, if it is determined that the resource attribute of the target service resource is a static non-hotspot resource, the scheduling server may extract the identification information of the target service resource, and then select the target cache server in the background service system according to the identification information of the target service resource. The identification information here may be a URL of the target service resource, a resource name of the target service resource, or a unique identification number allocated to the target service resource by the background service system. It can be understood that, since the identification information of the service resource is basically fixed, selecting the cache server based on the identification information can distribute the resource acquisition request of the static non-hot resource to the same cache server, thereby improving the hit rate of the service resource stored in the background service system.
Optionally, for static non-hotspot resources, the manner of selecting the target cache server may specifically be as follows: and when the resource attribute is the static non-hotspot resource, selecting a target cache server in the background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
In implementation, after the scheduling server determines that the target service resource is a static non-hotspot resource, the URL of the target service resource can be acquired, then the URL of the target service resource is used as a key value, a hash value corresponding to the target service resource is obtained by calculation through a consistent hash algorithm, and then the target cache server can be selected in a background service system according to the hash value.
And 203, when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to the real-time load conditions of all cache servers in the background service system.
In implementation, if the determined resource attribute of the target service resource is a dynamic resource or a static hotspot resource, the scheduling server may obtain real-time load conditions of all cache servers in the background service system, and then select the target cache server according to the real-time load conditions of all cache servers. It is easy to think that, in order to realize load balancing, the selected target cache server should be the cache server with lower current load or higher remaining load amount. It can be understood that the load balancing principle can be followed when selecting the cache server because the dynamic resource cannot be cached in advance, and the load balancing principle can be followed when selecting the cache server to avoid overloading part of the cache servers because the request amount of the static hotspot resource is large.
Optionally, when the cache server is selected according to the real-time load condition, the resource hit rate may also be considered to a certain extent, and correspondingly, the processing in step 203 may specifically be as follows: when the resource attribute is dynamic resource or static hotspot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system; judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the plurality of cache servers; if so, selecting the alternative cache server as the target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
In implementation, after determining that the resource attribute of the target service resource is a dynamic resource or a static non-hotspot resource, the scheduling server may acquire real-time load conditions of all cache servers in the background service system, and then arrange all cache servers in order of remaining load quota from high to low. Then, the scheduling server may select a plurality of cache servers arranged in front, and then determine whether there is an alternative cache server that has received the resource acquisition request of the target service resource in the selected plurality of cache servers. If the cache server exists, the alternative cache server can be determined as the target cache server, and if the cache server does not exist, the cache server with the highest remaining load amount can be selected as the target cache server. It should be noted that the selecting cache server may select a preset number of cache servers arranged in front, may select all cache servers whose remaining load quota is greater than a predetermined threshold, and may select a cache server in a designated proportion arranged in front.
And step 204, sending the resource acquisition request of the target service resource to the target cache server.
In implementation, after the scheduling server determines the resource attribute of the target service resource and selects the target cache server in a targeted manner, the scheduling server may send a resource acquisition request of the target service resource to the target cache server. If the target cache server stores the target service resource in advance, the target service resource stored locally can be directly fed back to the user terminal, if the target cache server does not store the target service resource, the target service resource can be temporarily configured locally, and then the configured target service resource is fed back to the user terminal. The configuration may be downloading from a source station of the target service resource, copying the target service resource from another cache server, notifying a technician to manually load the target service resource, and the like.
Optionally, for some hotspot resources with a large data volume, configuration may be performed in the cache server in advance to reduce the request feedback time, and the corresponding processing may be as follows: and if the receiving times of the resource acquisition requests of the target service resources in the preset duration are greater than a preset burst threshold and the data volume of the target service resources is greater than a preset data threshold, sending configuration requests of the target service resources to a plurality of cache servers with the front arranged residual load quota from high to low in the background service system.
In implementation, the scheduling server may record the number of times of receiving the resource acquisition request while receiving the resource acquisition request of the target service resource, and if the number of times of receiving within a preset time period is greater than a preset burst threshold, may detect a data amount of the target service resource, and if the data amount is greater than the preset data threshold, may arrange all the cache servers in the background service system in an order from high to low of the remaining load limit, and send a configuration request of the target service resource to a plurality of cache servers arranged in front, so as to configure the target service resource in the plurality of cache servers in advance, so that the cache servers do not need to temporarily configure the target service resource after receiving the corresponding resource acquisition request.
In the embodiment of the invention, a resource acquisition request of a target service resource is received, and the resource attribute of the target service resource is determined; when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource; when the resource attribute is a dynamic resource or a static hot resource, selecting a target cache server according to the real-time load condition of all cache servers in the background service system; and sending the resource acquisition request of the target service resource to the target cache server. Therefore, for static non-hotspot resources, the resource acquisition request can be guided to the cache server which stores corresponding resources, so that the resource hit rate of the cached resources can be improved, the request response time can be reduced, repeated storage in other cache servers is not needed, the storage space is saved, and for dynamic resources and hotspot resources, the resource acquisition request can be guided to the cache server with lower load, so that load balance among a plurality of cache servers can be realized, and overload of part of cache servers is avoided.
Based on the same technical concept, an embodiment of the present invention further provides an apparatus for distributing a resource acquisition request, as shown in fig. 3, the apparatus includes:
a determining module 301, configured to receive a resource obtaining request of a target service resource, and determine a resource attribute of the target service resource;
a selecting module 302, configured to select a target cache server in a background service system according to identification information of the target service resource when the resource attribute is a static non-hotspot resource, and select a target cache server according to real-time load conditions of all cache servers in the background service system when the resource attribute is a dynamic resource or a static hotspot resource;
a sending module 303, configured to send the resource obtaining request of the target service resource to the target cache server.
Optionally, the determining module 301 is specifically configured to:
determining that the resource attribute of the target service resource is a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request;
and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
Optionally, the determining module 301 is specifically configured to:
determining a target time interval to which the current moment belongs and a preset number of historical time intervals before the target time interval; (ii) a
Calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all the historical time intervals;
and if the total receiving times are larger than a preset hotspot threshold, determining that the resource attribute of the target service resource is a hotspot resource, otherwise, determining that the resource attribute of the target service resource is a non-hotspot resource.
Optionally, the determining module 301 is further configured to:
acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment;
if the ratio is larger than or equal to a preset hotspot ratio, determining the minimum receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource;
and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as a non-hotspot resource.
Optionally, the selecting module 302 is specifically configured to:
and when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
Optionally, the selecting module 302 is specifically configured to:
when the resource attribute is a dynamic resource or a static hot spot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system;
judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the cache servers;
and if so, selecting the alternative cache server as a target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
Optionally, as shown in fig. 4, the apparatus further includes a configuration module 304, configured to:
and if the receiving times of the resource acquisition requests of the target service resources in a preset time length are larger than a preset burst threshold value and the data volume of the target service resources is larger than a preset data threshold value, sending configuration requests of the target service resources to a plurality of cache servers with the residual load quota arranged from high to low in the background service system.
In the embodiment of the invention, a resource acquisition request of a target service resource is received, and the resource attribute of the target service resource is determined; when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource; when the resource attribute is a dynamic resource or a static hot resource, selecting a target cache server according to the real-time load condition of all cache servers in the background service system; and sending the resource acquisition request of the target service resource to the target cache server. Therefore, for static non-hotspot resources, the resource acquisition request can be guided to the cache server which stores corresponding resources, so that the resource hit rate of the cached resources can be improved, the request response time can be reduced, repeated storage in other cache servers is not needed, the storage space is saved, and for dynamic resources and hotspot resources, the resource acquisition request can be guided to the cache server with lower load, so that load balance among a plurality of cache servers can be realized, and overload of part of cache servers is avoided.
It should be noted that: the apparatus for distributing a resource obtaining request according to the foregoing embodiment is illustrated by only dividing the functional modules when distributing a resource obtaining request, and in practical applications, the function allocation may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus for distributing a resource acquisition request and the method embodiment for distributing a resource acquisition request provided in the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
Fig. 5 is a schematic structural diagram of a scheduling server according to an embodiment of the present invention. The dispatch server 500 may vary widely due to configuration or performance differences and may include one or more central processors 522 (e.g., one or more processors) and memory 532, one or more storage media 530 (e.g., one or more mass storage devices) storing applications 542 or data 544. Memory 532 and storage media 530 may be, among other things, transient storage or persistent storage. The program stored on storage medium 530 may include one or more modules (not shown), each of which may include a sequence of instructions operating on the schedule server 500. Still further, the central processor 522 may be configured to communicate with the storage medium 530, and execute a series of instruction operations in the storage medium 530 on the scheduling server 500.
The dispatch Server 500 may also include one or more power supplies 529, one or more wired or wireless network interfaces 550, one or more input-output interfaces 558, one or more keyboards 556, and/or one or more operating systems 541, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (14)
1. A method of distributing resource acquisition requests, the method comprising:
receiving a resource acquisition request of a target service resource, and determining a resource attribute of the target service resource;
when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target service resource;
when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to the real-time load conditions of all cache servers in the background service system;
sending a resource acquisition request of the target service resource to the target cache server;
when determining that the resource attribute of the target service resource is a hotspot resource, the method further includes:
acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment;
if the ratio is larger than or equal to a preset hotspot ratio, determining the minimum receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource;
and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as a non-hotspot resource.
2. The method of claim 1, wherein the determining the resource attribute of the target business resource comprises:
determining that the resource attribute of the target service resource is a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request;
and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
3. The method according to claim 2, wherein the determining that the resource attribute of the target service resource is a hot resource or a non-hot resource according to the number of times of receiving the resource acquisition request of the target service resource within a preset history duration and a preset hot threshold comprises:
determining a target time interval to which the current moment belongs and a preset number of historical time intervals before the target time interval;
calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all the historical time intervals;
and if the total receiving times are larger than a preset hotspot threshold, determining that the resource attribute of the target service resource is a hotspot resource, otherwise, determining that the resource attribute of the target service resource is a non-hotspot resource.
4. The method of claim 1, wherein when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system according to the identification information of the target business resource comprises:
and when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
5. The method of claim 1, wherein when the resource attribute is a dynamic resource or a static hotspot resource, selecting a target cache server according to real-time load conditions of all cache servers in the background service system comprises:
when the resource attribute is a dynamic resource or a static hot spot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system;
judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the cache servers;
and if so, selecting the alternative cache server as a target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
6. The method of claim 1, further comprising:
and if the receiving times of the resource acquisition requests of the target service resources in a preset time length are larger than a preset burst threshold value and the data volume of the target service resources is larger than a preset data threshold value, sending configuration requests of the target service resources to a plurality of cache servers with the residual load quota arranged from high to low in the background service system.
7. An apparatus for distributing resource acquisition requests, the apparatus comprising:
the determining module is used for receiving a resource obtaining request of a target service resource and determining the resource attribute of the target service resource;
the selection module is used for selecting a target cache server in a background service system according to the identification information of the target service resource when the resource attribute is a static non-hotspot resource, and selecting the target cache server according to the real-time load condition of all cache servers in the background service system when the resource attribute is a dynamic resource or a static hotspot resource;
a sending module, configured to send a resource acquisition request of the target service resource to the target cache server;
the determining module is further configured to:
acquiring the occupation ratio of the hot spot resources determined in a preset statistical time before the current moment;
if the ratio is larger than or equal to a preset hotspot ratio, determining the minimum receiving times in the receiving times of the resource obtaining request corresponding to each hotspot resource;
and if the receiving times of the resource obtaining request of the target service resource are smaller than the minimum receiving time, determining the resource attribute of the target service resource as a non-hotspot resource.
8. The apparatus of claim 7, wherein the determining module is specifically configured to:
determining that the resource attribute of the target service resource is a dynamic resource or a static resource according to the suffix of the URL of the target service resource carried by the resource acquisition request;
and determining the resource attribute of the target service resource as a hot resource or a non-hot resource according to the receiving times of the resource acquisition request of the target service resource within the preset historical duration and a preset hot threshold.
9. The apparatus of claim 8, wherein the determining module is specifically configured to:
determining a target time interval to which the current moment belongs and a preset number of historical time intervals before the target time interval;
calculating the total receiving times of the resource obtaining requests of the target service resources in the target time interval and all the historical time intervals;
and if the total receiving times are larger than a preset hotspot threshold, determining that the resource attribute of the target service resource is a hotspot resource, otherwise, determining that the resource attribute of the target service resource is a non-hotspot resource.
10. The apparatus according to claim 7, wherein the selection module is specifically configured to:
and when the resource attribute is a static non-hotspot resource, selecting a target cache server in a background service system based on a consistent hash algorithm by taking the URL of the target service resource as a key value.
11. The apparatus according to claim 7, wherein the selection module is specifically configured to:
when the resource attribute is a dynamic resource or a static hot spot resource, selecting a plurality of cache servers with the residual load quota arranged from high to low in the background service system;
judging whether an alternative cache server which has received the resource acquisition request of the target service resource exists in the cache servers;
and if so, selecting the alternative cache server as a target cache server, otherwise, selecting the cache server with the highest remaining load quota as the target cache server.
12. The apparatus of claim 7, further comprising a configuration module to:
and if the receiving times of the resource acquisition requests of the target service resources in a preset time length are larger than a preset burst threshold value and the data volume of the target service resources is larger than a preset data threshold value, sending configuration requests of the target service resources to a plurality of cache servers with the residual load quota arranged from high to low in the background service system.
13. A scheduling server comprising a processor and a memory, said memory having stored therein at least one instruction, at least one program, set of codes or set of instructions, said at least one instruction, said at least one program, set of codes or set of instructions being loaded and executed by said processor to implement the method of distributing resource acquisition requests according to any of claims 1 to 6.
14. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the method of distributing resource acquisition requests according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811141742.0A CN109246229B (en) | 2018-09-28 | 2018-09-28 | Method and device for distributing resource acquisition request |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811141742.0A CN109246229B (en) | 2018-09-28 | 2018-09-28 | Method and device for distributing resource acquisition request |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109246229A CN109246229A (en) | 2019-01-18 |
| CN109246229B true CN109246229B (en) | 2021-08-27 |
Family
ID=65054054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811141742.0A Active CN109246229B (en) | 2018-09-28 | 2018-09-28 | Method and device for distributing resource acquisition request |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109246229B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110049130B (en) * | 2019-04-22 | 2020-07-24 | 北京邮电大学 | Service deployment and task scheduling method and device based on edge computing |
| CN110149394A (en) * | 2019-05-20 | 2019-08-20 | 典基网络科技(上海)有限公司 | Dispatching method, device and the storage medium of system resource |
| CN112019451B (en) * | 2019-05-29 | 2023-11-21 | 中国移动通信集团安徽有限公司 | Bandwidth allocation method, debugging network element, local cache server and computing device |
| CN110290196B (en) * | 2019-06-20 | 2021-08-27 | 网宿科技股份有限公司 | Content distribution method and device |
| CN110866198B (en) * | 2019-09-27 | 2022-10-28 | 上海硬通网络科技有限公司 | Static resource caching method, system, device, computer equipment and storage medium |
| CN112953985B (en) * | 2019-12-10 | 2023-04-07 | 贵州白山云科技股份有限公司 | Request data processing method, device, medium and system |
| CN112953984B (en) * | 2019-12-10 | 2023-07-28 | 贵州白山云科技股份有限公司 | Data processing method, device, medium and system |
| CN111131841A (en) * | 2020-02-24 | 2020-05-08 | 北京达佳互联信息技术有限公司 | Live indirect access method and device, electronic equipment and storage medium |
| CN111371866B (en) * | 2020-02-26 | 2023-03-21 | 厦门网宿有限公司 | Method and device for processing service request |
| CN112003922B (en) * | 2020-08-18 | 2023-06-06 | 杭州指令集智能科技有限公司 | Data transmission method and device |
| WO2022041205A1 (en) * | 2020-08-31 | 2022-03-03 | 华为技术有限公司 | Communication method and multi-access edge computing server |
| CN114640700B (en) * | 2020-11-30 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Calling frequency control method and device |
| CN112764948A (en) * | 2021-01-22 | 2021-05-07 | 土巴兔集团股份有限公司 | Data transmission method, data transmission device, computer device, and storage medium |
| CN113452753B (en) * | 2021-05-21 | 2022-09-30 | 北京奇艺世纪科技有限公司 | Data distribution method, device, equipment and storage medium |
| CN114666319B (en) * | 2022-03-02 | 2024-03-22 | 北京百度网讯科技有限公司 | Data downloading method, device, electronic equipment and readable storage medium |
| CN114629919A (en) * | 2022-03-31 | 2022-06-14 | 北京百度网讯科技有限公司 | Resource acquisition method, device, equipment and storage medium |
| CN114760361B (en) * | 2022-04-25 | 2023-09-15 | 中国平安人寿保险股份有限公司 | Service request distribution method, system, equipment and storage medium |
| CN116708455B (en) * | 2023-02-03 | 2024-01-05 | 天翼云科技有限公司 | Resource acquisition method and device, readable storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2894827A1 (en) * | 2013-12-06 | 2015-07-15 | Huawei Technologies Co., Ltd. | Method, apparatus, and system for managing migration of virtual machine |
| CN106649150A (en) * | 2016-12-26 | 2017-05-10 | 锐捷网络股份有限公司 | Cache management method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101668046B (en) * | 2009-10-13 | 2012-12-19 | 成都市华为赛门铁克科技有限公司 | Resource caching method, device and system thereof |
| CN103051706A (en) * | 2012-12-19 | 2013-04-17 | 新浪网技术(中国)有限公司 | Dynamic webpage request processing system and method for dynamic website |
| US10009439B1 (en) * | 2013-12-05 | 2018-06-26 | Instart Logic, Inc. | Cache preloading |
| CN104539724A (en) * | 2015-01-14 | 2015-04-22 | 浪潮(北京)电子信息产业有限公司 | Information processing method and system |
| CN108322495B (en) * | 2017-01-18 | 2021-07-06 | 阿里巴巴集团控股有限公司 | Method, device and system for processing resource access request |
| CN108347460B (en) * | 2017-01-25 | 2020-04-14 | 华为技术有限公司 | Resource access method and device |
-
2018
- 2018-09-28 CN CN201811141742.0A patent/CN109246229B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2894827A1 (en) * | 2013-12-06 | 2015-07-15 | Huawei Technologies Co., Ltd. | Method, apparatus, and system for managing migration of virtual machine |
| CN106649150A (en) * | 2016-12-26 | 2017-05-10 | 锐捷网络股份有限公司 | Cache management method and device |
Non-Patent Citations (2)
| Title |
|---|
| "Research of Web QoS Control Model Based on Dynamic Resource Reallocation Scheme";Junhuai Li,Yile Wang,Jing Zhang;《2008 International Symposium on Information Science and Engineering》;20081230;全文 * |
| "改进视频点播负载能力的模型与算法研究";舒畅;《中国博士学位论文全文数据库 信息科技辑》;20130531;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109246229A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109246229B (en) | Method and device for distributing resource acquisition request | |
| CN109660607B (en) | Service request distribution method, service request receiving method, service request distribution device, service request receiving device and server cluster | |
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| US11888756B2 (en) | Software load balancer to maximize utilization | |
| CN111131058B (en) | Access quantity control method and device | |
| CN110602254B (en) | Method, device and system for realizing load balance | |
| CN107341050B (en) | Service processing method and device based on dynamic thread pool | |
| CN110795203B (en) | Resource scheduling method, device, system and computing equipment | |
| CN109672757B (en) | File access method and file access processing device | |
| US8572621B2 (en) | Selection of server for relocation of application program based on largest number of algorithms with identical output using selected server resource criteria | |
| CN109510878B (en) | Long connection session keeping method and device | |
| CN110336848B (en) | Scheduling method, scheduling system and scheduling equipment for access request | |
| CN108829519A (en) | Method for scheduling task, cloud platform and computer readable storage medium based on cloud platform | |
| CN113127168A (en) | Service distribution method, system, device, server and medium | |
| CN111131841A (en) | Live indirect access method and device, electronic equipment and storage medium | |
| US20140359182A1 (en) | Methods and apparatus facilitating access to storage among multiple computers | |
| CN110888658B (en) | Method and device for dynamically changing function modules in application program and storage medium | |
| CN111338750A (en) | Pressure adjusting method and device for execution node, server and storage medium | |
| CN109120548B (en) | Flow control method and device | |
| CN107908730B (en) | Method and device for downloading data | |
| CN114003337A (en) | Access request distribution method and device | |
| CN113051456A (en) | Request processing method and device, electronic equipment and computer readable medium | |
| CN113765964A (en) | Method and device for distributing services of distributed system | |
| CN116820769A (en) | Task allocation method, device and system | |
| CN107689979B (en) | method and equipment for processing download request |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |