WO2017025052A1 - Resource caching method and device - Google Patents

Resource caching method and device Download PDF

Info

Publication number
WO2017025052A1
WO2017025052A1 PCT/CN2016/094627 CN2016094627W WO2017025052A1 WO 2017025052 A1 WO2017025052 A1 WO 2017025052A1 CN 2016094627 W CN2016094627 W CN 2016094627W WO 2017025052 A1 WO2017025052 A1 WO 2017025052A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
cache
request message
predetermined
cache subsystem
Prior art date
Application number
PCT/CN2016/094627
Other languages
French (fr)
Chinese (zh)
Inventor
郑小娟
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017025052A1 publication Critical patent/WO2017025052A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching

Definitions

  • the present invention relates to the field of communications, and in particular to a resource caching method and apparatus.
  • the OTT (Over The Top) Cache (Cache) subsystem is deployed in China Mobile's existing Internet Protocol (IP) bearer network for storing hot content or specific content in the Internet.
  • IP Internet Protocol
  • the request is forwarded to the OTT Cache system by hijacking the access request of the user in the network, and the OTT Cache system is used as a proxy server to provide the user with the required Internet access data, thereby avoiding the direct use of the user.
  • the data is downloaded from the Internet source station, thereby effectively avoiding the congestion problem of the network link, reducing the delay of the Chinese mobile user accessing the Internet content, improving the user's data service experience, and also reducing the settlement fee generated by the Internet traffic.
  • URLs Uniform Resource Locators
  • RR request routing
  • proxy mode When proxy mode is enabled, if the resource OTT Cache requested by the user can be cached, it is not a hotspot resource (for example, a resource is only accessed once or twice). At this time, if the RR performs redirection processing, the OTT Cache frequently pulls down the content to the source station, which may be considered as a malicious attack by the source station, and the cache subsystem is restricted from accessing.
  • the RR does not provide the 302 redirect service for all the requests. If the resource requested by the user belongs to the hotspot resource (the same resource is accessed multiple times), the user continuously requests the same resource from the source station, which occupies a large amount of bandwidth. The network is congested and the user experience is poor. It can be seen that the problem that the cache subsystem is restricted from accessing and network congestion may be caused in the related art, and an effective solution has not been proposed yet.
  • the present invention provides a resource caching method and apparatus to at least solve the problem that the cache subsystem is restricted in access and network congestion existing in the related art.
  • a resource caching method comprising: receiving a first request message, wherein the first request message is used to request access to a first predetermined resource; and the access station is counted according to the first request message Determining the number of accesses of the first predetermined resource; determining whether the number of accesses reaches a first predetermined threshold; and if the determination result is yes, notifying the cache subsystem to cache the first predetermined resource, wherein the cache subsystem The resource used to provide the cache.
  • the method further includes: determining whether the cache subsystem successfully caches the first predetermined resource; and if the determination result is yes, using the location Cache subsystem provides The first predetermined resource.
  • the method further includes: if the determination result is yes, recording the cached first predetermined resource in a cache content distribution table
  • the cache content distribution table records the resources that the cache subsystem has cached, and the cache content distribution table is configured to redirect the request message for accessing the resources recorded in the cache content distribution table to The cache subsystem provides the resources recorded in the cache content distribution table by using the cache subsystem.
  • the method further includes: receiving a second request message, where the second request message is used to request access to the second predetermined resource; and determining whether the second predetermined resource is cached in the cache subsystem; If the determination result is yes, the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
  • determining whether the number of accesses reaches the first predetermined threshold includes: acquiring a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and determining the number of accesses according to the heat statistics table. Whether the first predetermined threshold is reached.
  • the method further includes: determining, according to the heat statistics table, the expired resource, wherein the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold; The time and number of times of accessing the expired resource recorded in the heat statistics table are deleted.
  • a resource cache apparatus including: a first receiving module, configured to receive a first request message, wherein the first request message is used to request access to a first predetermined resource; a statistics module The first determining unit is configured to determine whether the number of accesses reaches the first predetermined threshold according to the first request message, and the notification module is configured to be in the first When the judgment result of the determination module is YES, the notification cache subsystem caches the first predetermined resource, wherein the cache subsystem is used to provide the cached resource.
  • the device further includes: a second determining module, configured to determine whether the cache subsystem successfully caches the first predetermined resource; and a providing module, configured to be determined by the second determining module The first predetermined resource is provided by the cache subsystem.
  • the device further includes: a recording module, configured to: when the determination result of the second determining module is yes, record the cached first predetermined resource in a cache content distribution table, where The cache content distribution table records the resource that the cache subsystem has cached, and the cache content distribution table is configured to redirect a request message for accessing the resource recorded in the cache content distribution table to the cache a subsystem that utilizes the cache subsystem to provide resources recorded in the cache content distribution table.
  • a recording module configured to: when the determination result of the second determining module is yes, record the cached first predetermined resource in a cache content distribution table, where The cache content distribution table records the resource that the cache subsystem has cached, and the cache content distribution table is configured to redirect a request message for accessing the resource recorded in the cache content distribution table to the cache a subsystem that utilizes the cache subsystem to provide resources recorded in the cache content distribution table.
  • the device further includes: a second receiving module, configured to receive a second request message, where the second request message is used to request access to the second predetermined resource; and the third determining module is configured to determine the Whether the second predetermined resource is cached in the cache subsystem; and the redirecting module is configured to redirect the second request message to the cacher if the determination result of the third determining module is yes a system, wherein the second request message is used to indicate the cache The subsystem provides the second predetermined resource.
  • a second receiving module configured to receive a second request message, where the second request message is used to request access to the second predetermined resource
  • the third determining module is configured to determine the Whether the second predetermined resource is cached in the cache subsystem
  • the redirecting module is configured to redirect the second request message to the cacher if the determination result of the third determining module is yes a system, wherein the second request message is used to indicate the cache The subsystem provides the second predetermined resource.
  • the first determining module includes: an obtaining unit, configured to obtain a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and the determining unit is configured to be configured according to the heat statistics table. Determining whether the number of accesses reaches the first predetermined threshold.
  • the apparatus further includes: a determining module, configured to determine an expired resource according to the heat statistics table, wherein the expired resource is a difference between a current time and a latest time of accessing the expired resource exceeds a second a resource of a predetermined threshold; a deletion module configured to delete a time and a number of times of accessing the expired resource recorded in the heat statistics table.
  • a determining module configured to determine an expired resource according to the heat statistics table, wherein the expired resource is a difference between a current time and a latest time of accessing the expired resource exceeds a second a resource of a predetermined threshold
  • a deletion module configured to delete a time and a number of times of accessing the expired resource recorded in the heat statistics table.
  • the first request message is received by the present invention, wherein the first request message is used to request access to the first predetermined resource; the number of accesses to the first predetermined resource is counted according to the first request message; Whether the number of accesses reaches a first predetermined threshold; if the result of the determination is YES, the notification cache subsystem caches the first predetermined resource, wherein the cache subsystem is configured to provide the cached resource.
  • FIG. 1 is a flowchart of a resource caching method according to an embodiment of the present invention.
  • FIG. 2 is a structural block diagram of a resource cache apparatus according to an embodiment of the present invention.
  • FIG. 3 is a block diagram 1 of a preferred structure of a resource cache device according to an embodiment of the present invention.
  • FIG. 4 is a block diagram 2 of a preferred structure of a resource cache device according to an embodiment of the present invention.
  • FIG. 5 is a block diagram 3 of a preferred structure of a resource cache device according to an embodiment of the present invention.
  • FIG. 6 is a structural block diagram of a first determining module 26 in a resource cache device according to an embodiment of the present invention.
  • FIG. 7 is a block diagram 4 of a preferred structure of a resource cache device according to an embodiment of the present invention.
  • FIG. 8 is a flowchart of user request scheduling and heat statistics according to an embodiment of the present invention.
  • FIG. 10 is a flowchart of content cache result reporting according to an embodiment of the present invention.
  • FIG. 11 is a simplified networking diagram of a content distribution network system according to an embodiment of the present invention.
  • FIG. 12 is a diagram of a non-proxy mode processing mechanism in accordance with an embodiment of the present invention.
  • FIG. 1 is a flowchart of a resource caching method according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps:
  • Step S102 receiving a first request message, where the first request message is used to request access to the first predetermined resource
  • Step S104 counting the number of accesses to access the first predetermined resource according to the first request message
  • Step S106 determining whether the number of accesses reaches a first predetermined threshold
  • Step S108 When the determination result is yes, the notification cache subsystem caches the first predetermined resource, where the cache subsystem is used to provide the cached resource.
  • the cache subsystem when the cache subsystem is notified of the cache resource, only the cache subsystem caches the resources whose cache access times exceed the first predetermined threshold, thereby achieving the purpose of the selected cache resource, that is, the first reservation is reached for the number of accesses.
  • the resource of the threshold that is, the resource is hot enough and the number of users accessed is relatively large
  • the resource can be directly provided by the cache subsystem. Thereby reducing network congestion.
  • the problem that the cache subsystem is restricted access and network congestion existing in the related art is solved, thereby achieving the effect of avoiding the cache subsystem being restricted by access and reducing network congestion.
  • the method further includes: determining whether the cache subsystem successfully caches the first predetermined resource; and if the determination result is yes, using The cache subsystem provides the first predetermined resource described above. Moreover, whether the cache subsystem supports the caching of the resources can be determined by whether the cache subsystem successfully caches resources (eg, the first predetermined resource described above).
  • the method further includes: when the determination result is yes, recording the cached first predetermined resource in the cache content distribution table.
  • the cache content distribution table records the resource that has been cached by the cache subsystem, and the cache content distribution table is used to redirect the request message for accessing the resource recorded in the cache content distribution table to the cache subsystem.
  • the cache subsystem is used to provide resources recorded in the cache content distribution table. That is to say, the resources recorded in the cache content distribution table are all resources successfully cached by the cache subsystem, so that when the user needs to access the resources recorded in the cache content distribution table, the cache subsystem can be directly used to provide resources. So as to effectively reduce the pressure on the network and reduce network congestion.
  • the request message in this embodiment may be a first request message, a second request message, or other request message.
  • the second request message when the first request message is received in step S102, the second request message may be received at the same time, wherein the second request message is used to request access to the second predetermined resource;
  • the cache has a second predetermined resource; if the result of the determination is yes, the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
  • the second reservation is already cached in the cache subsystem
  • the second request message can be directly redirected to the cache subsystem, thereby improving the efficiency of obtaining the second predetermined resource.
  • the first predetermined resource and the second predetermined resource may be the same resource or different resources.
  • the timing of receiving the second request message may occur at any time, such as before or after step S104, or before or after step S108.
  • determining whether the number of accesses reaches the first predetermined threshold includes: acquiring a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and determining the number of accesses according to the heat statistics table. Whether the above first predetermined threshold is reached.
  • the heat statistics table is used to record the number of accesses and access times of the resources that the user has visited, and each time a resource is accessed, the number of accesses to the corresponding resources is increased by one, and the corresponding records are also corresponding to The access time of the visit, so as to clearly determine the access of resources.
  • the expired resource may also be determined according to the heat statistics table, where the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold; deleting the heat The time and number of times recorded in the statistics table to access the above-mentioned expired resources.
  • the second predetermined threshold may be set to 10 days, or 5 days, or other values.
  • the foregoing expired resource is a resource with reduced access, when a resource is not accessed for longer than the second predetermined threshold. Then, it can be determined that the access heat of the resource is low enough, and the record of the expired resource is deleted in the heat statistics table, thereby ensuring that the heat statistics table can count more access times and times of resources.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform.
  • the technical solution of the present invention which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
  • the optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.
  • a resource buffering device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and has not been described again.
  • the term "module” may implement software of a predetermined function or a combination of software and hardware.
  • the apparatus described in the following embodiments is preferably implemented in software, implementation of a combination of software and hardware is also possible and contemplated.
  • FIG. 2 is a structural block diagram of a resource cache apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus includes a first receiving module 22, a statistic module 24, a first judging module 26, and a notification module 28. Be explained.
  • the first receiving module 22 is configured to receive a first request message, where the first request message is used to request access to the first predetermined resource, and the statistics module 24 is connected to the first receiving module 22, and configured to be according to the first request.
  • the number of accesses of the first predetermined resource is accessed by the message statistics.
  • the first determining module 26 is connected to the statistics module 24, and is configured to determine whether the number of accesses reaches a first predetermined threshold.
  • the notification module 28 is connected to the first determining module 26. And being configured to notify the cache subsystem to cache the first predetermined resource when the determination result of the first determining module 26 is YES, wherein the cache subsystem is configured to provide the cached resource.
  • FIG. 3 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 3, the device includes a second judging module 32 and a providing module 34, in addition to all the modules shown in FIG. The device will be described.
  • the second determining module 32 is connected to the notification module 28, and is configured to determine whether the cache subsystem successfully caches the first predetermined resource.
  • the providing module 34 is connected to the second determining module 32, and is configured to be in the second determining module. In the case where the judgment result of 32 is YES, the above-mentioned first predetermined resource is provided by the above-described cache subsystem.
  • FIG. 4 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 4, the device includes a recording module 42 in addition to all the modules shown in FIG. .
  • the recording module 42 is connected to the second judging module 32, and configured to record the cached first predetermined resource in the cache content distribution table if the judgment result of the second judging module 32 is YES, wherein the The cache content distribution table records the resources that the cache subsystem has cached, and the cache content distribution table is used to redirect the request message for accessing the resources recorded in the cache content distribution table to the cache subsystem, and utilize the cache subsystem. Provides resources recorded in the cached content distribution table.
  • the device includes a second receiving module 52 and a third determining module 54 in addition to all the modules shown in FIG. And the redirection module, the second receiving module 52 can be connected to the first receiving module 22, the statistic module 24, the first judging module 26 or the notification module 28, and the second receiving module 52 is connected to the first receiving module 22 as an example.
  • the device will be described.
  • the second receiving module 52 is connected to the first receiving module 22 and configured to receive the second request message, where the second request message is used to request access to the second predetermined resource; the third determining module, the block 54 is connected to the second receiving The module 52 is configured to determine whether the second predetermined resource is cached in the cache subsystem.
  • the redirecting module 56 is connected to the third determining module 54 and is configured to be in the case that the determination result of the third determining module 54 is YES.
  • the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
  • FIG. 6 is a structural block diagram of a first determining module 26 in a resource cache device according to an embodiment of the present invention. As shown in FIG. 6, the first determining module 26 includes an obtaining unit 62 and a determining unit 64. Module 26 is described.
  • the obtaining unit 62 is configured to obtain a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; the determining unit 64 is connected to the obtaining unit 52, and is configured to determine whether the number of accesses is reached according to the heat statistics table.
  • the first predetermined threshold described above.
  • FIG. 7 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 7, the device includes a determination module 72 and a deletion module 74, in addition to all the modules shown in FIG. The device is described.
  • the determining module 72 is connected to the first determining module 26, and is configured to determine the expired resource according to the heat statistics table, wherein the expired resource is that the difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold.
  • the resource deletion module 74 is connected to the determining module 72, and is configured to delete the time and the number of times the access to the expired resource recorded in the heat statistics table.
  • the main body of execution of each of the foregoing methods may be an RR, and each of the foregoing devices may be applied to the RR.
  • the embodiment of the present invention provides a non-proxy mode Hypertext Transfer Protocol (HTTP) request scheduling method, according to whether the cache subsystem can request the resources.
  • HTTP Hypertext Transfer Protocol
  • a cache service is provided to determine whether to perform 302 redirect processing on the HTTP request and when the cache subsystem performs content caching. The method includes:
  • the RR does not perform the 302 redirection process, and the source station directly provides services to the user, so as to avoid the problem that the cache subsystem cannot provide the service and the user experience is poor.
  • the RR does not redirect the request, but only performs the heat statistics, and directly provides the service to the user by the source station to avoid frequent access by the cache subsystem.
  • the source station may result in restricted access.
  • the RR When the content requested by the user belongs to the hotspot resource, and the number of accesses reaches the access heat threshold, the RR notifies the OTT Cache of the drop-down content. When the user requests the same resource, the RR forwards the request to the OTT Cache, and the cache subsystem Providing services directly, avoiding network congestion caused by direct large-scale access to the source station, saving bandwidth and improving user experience, and improving the cache hit ratio of OTT Cache.
  • the implementation of the solution in the embodiment of the present invention may depend on the design of two tables: a url heat statistics table (same as the above heat statistics table) and a cache content distribution table.
  • the url heat statistics table is used to count the url access heat.
  • the cache content distribution table is used to record the resources that the cache subsystem can provide services for users.
  • FIG. 8 is a flow chart of user request scheduling and heat statistics according to an embodiment of the present invention. As shown in FIG. 8, the process includes the following steps:
  • Step S802 the user initiates an HTTP request.
  • Step S804 according to the black and white list, IP segment and other rules, determine whether the url (ie, the above HTTP request) needs to be hijacked, when the determination result is necessary, go to step S806, otherwise, go to step S810;
  • the url ie, the above HTTP request
  • Step S806 when it is determined that the HTTP request of the user needs to be hijacked by the RR, the RR records the current request in the heat statistics table according to the user access url (steps S812-S814), and determines whether the content exists in the cache content distribution table. If yes, go to step S808, otherwise, go to step S810;
  • Step S808 responding to the user request, redirecting to the OTT Cache, and providing the service by using the cache subsystem;
  • Step S810 ignoring the current request, and allowing the user to directly download the foregoing resource from the source station;
  • Step S812 determining whether there is an access record of the url in the heat statistics table, if the result of the determination is non-existent, go to step S814, otherwise, go to step S816;
  • step S814 the heat statistics table adds a record, the number of accesses is recorded as 1, and the request time is recorded; the recorded statistics items include: a.key: the complete url requested by the user uses the md5 algorithm to calculate the 32-bit encrypted string as the key.
  • Service node id The id number of the service node that the user has scheduled this time.
  • Last access time Record the time of the last user visit, in milliseconds.
  • the hotspot threshold (same first threshold as described above): the threshold is obtained, and the url of the user is matched from the blacklist list of the HTTP domain name to the corresponding domain name record, and the hotspot threshold of the url is obtained from the recorded entry. .
  • the system sets different thresholds and sends them to the RR through the HTTP domain name blacklist and whitelist interface.
  • the RR notifies the OTT Cache to cache.
  • the OTT Cache notifies the RR to update the content.
  • the RR updates the memory data and saves it in the content distribution table.
  • the hotspot threshold statistics and content pull-down process are as shown in Figure 9, and the content reporting process is attached. Figure 10 shows.
  • Step S816 if there is a record corresponding to the key of the url in the heat meter, the number of accesses of the record is increased by 1, and the last access time is changed to the current time.
  • the md5 value can be calculated as the record key according to the url requested by the user.
  • the content distribution table it is checked whether the content corresponding to the key has been cached on this node. If the content exists, the user request is redirected to the cache subsystem in response to the user request. If it does not exist, the user request is ignored and the user is returned directly to the source.
  • FIG. 9 is a flowchart of threshold statistics and content pull-down according to an embodiment of the present invention. As shown in FIG. 9, the process includes the following steps:
  • Step S902 the RR background process, periodically scans the heat statistics table, and processes each record in turn;
  • Step S904 taking out the next record
  • step S906 it is determined whether the record has expired (the current time minus whether the last access time of the url is greater than or equal to the expired time). If the result of the determination is yes, it is determined that the url is overdue, and the process goes to step S908; otherwise, the process goes to step S910;
  • Step S908 deleting the url related record
  • Step S910 determining whether the number of accesses is greater than or equal to the hotspot threshold, if the determination result is yes, go to step S912, otherwise, go to step S916;
  • Step S912 according to the key value of the url and the node id and other information to determine whether there is the url record in the cache content table, if the determination result is yes, then go to step S916, otherwise, go to step S914;
  • Step S914 notifying the OTT Cache to download the content corresponding to the url;
  • Step S916 indicating that the content is cached, and does not need to notify the OTT Cache, and the hot record processing is completed.
  • Step S918 it is determined whether there are still unprocessed records, if the determination result is yes, go to step S904, otherwise, go to step S920;
  • Step S920 waiting for an access request of the user.
  • the RR can find the record that reaches the hotspot threshold, notify the cache subsystem to perform content caching, and clean up the url records that have not been accessed again for a long time.
  • RR needs to support different website or file suffixes. Different thresholds, such as www.sohu.com or sina.com, can be set separately for different cache threshold policies.
  • the RR receives a user's HTTP download request, the RR finds the corresponding matching rule record in the table according to the matching rule, and then finds a heat threshold for the current request url, and saves it to the content heat statistics table.
  • the cache subsystem When the number of times the user requests the same resource reaches the website cache popularity threshold, the cache subsystem issues a request drop-down to the source station. After the content is successfully cached, if the user requests the same resource again during the validity period, the RR determines that the cache subsystem can provide the service by querying the cache content distribution table, and forwards the request to the cache subsystem, which is directly This user provides content services.
  • FIG. 10 is a flowchart of content cache result reporting according to an embodiment of the present invention. As shown in FIG. 10, the process includes the following steps:
  • step S1002 when the RR main program starts, the content distribution table is first loaded from the database (disk), and the content distribution data is saved in the memory cache for use by the scheduling process. (corresponding to steps S1-S3 in Fig. 10)
  • step S1004 when the cache subsystem has content update, the RR is notified to perform content update. (corresponding to step S4 in Fig. 10)
  • step S1006 the RR updates related records in the memory cache, adding/deleting. (corresponding to step S5 in Fig. 10)
  • step S1008 the RR persists the record to the disk. (corresponding to step S6 in Fig. 10)
  • FIG. 11 is a simplified networking diagram of a Content Delivery Network (CDN) system according to an embodiment of the present invention.
  • the entire system mainly includes a scheduling subsystem and a cache subsystem.
  • the RR scheduling subsystem hijacks and redirects the user's HTTP request.
  • the cache subsystem is composed of SLB7 and OTT Cache. When the RR redirects the user's HTTP request to the cache subsystem, the cache subsystem provides services for the user.
  • FIG. 12 is a diagram of a non-proxy mode processing mechanism according to an embodiment of the present invention.
  • the FIG. 12 depicts that after the RR is hijacked to the user's HTTP download request, the cache subsystem misses and reaches a heat threshold for content pull-down (corresponding to FIG. 12).
  • the process of reaching the threshold proxy download) and the cache hit includes the following steps:
  • the processing flow of the cache subsystem miss includes:
  • Step S1202 The user sends an HTTP file download request.
  • Step S1204 the RR hijacks the download request.
  • Step S1206 determining that the file is not cached in the cache subsystem (corresponding to the foregoing resource), and ignoring the HTTP file download request of the user;
  • step S1208 the source station provides a file download service for the user.
  • the content pulldown process that reaches the heat threshold includes:
  • Step S1210 the number of accesses in the heat statistics table is +1;
  • Step S1212 The hotspot threshold is reached, and the OTT Cache is notified to download.
  • step S1214 the OTT Cache downloads the file successfully.
  • the processing flow of cache hits includes:
  • Step S1216 the user sends an HTTP file download request.
  • Step S1218 the RR hijacks the above download request
  • Step S1220 determining that the foregoing file has been cached in the cache subsystem, and notifying the user to redirect to the cache subsystem;
  • Step S1222 redirecting the HTTP file download request of the user to the OTT Cache
  • step S1224 the OTT Cache provides a file download service for the user.
  • a non-proxy mode HTTP request scheduling method is implemented.
  • the scheduling subsystem determines when the cache subsystem pulls down the content through the maintenance of the heat statistics table, records the content cache condition through the cache content distribution table, and determines the cache subsystem. Can you provide services to users?
  • the scheduling subsystem determines whether to perform 302 redirect processing on the HTTP request according to whether the cache subsystem can provide a cache service to the requested resource.
  • each of the above modules may be implemented by software or a combination of software and hardware.
  • the foregoing may be implemented by, but not limited to, the above modules are all located in the same processor; or, the above modules Located in multiple processors.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be configured to store program code for performing the following steps:
  • the first request message is received, where the first request message is used to request access to the first predetermined resource.
  • the foregoing storage medium may include, but is not limited to, a USB flash drive, a Read-Only Memory (ROM), and a Random Access Memory (RAM).
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • the processor performs the above steps S11-S14 according to the stored program code in the storage medium.
  • modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices.
  • they may be implemented by program code executable by a computing device such that they may be stored in a storage device by a computing device and, in some cases, may be executed in a different order than herein.
  • the steps shown or described are either made separately into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • the resource cache method and apparatus provided by the embodiments of the present invention have the following beneficial effects: the problem that the cache subsystem is restricted from accessing and network congestion exists in the related art, thereby avoiding the cache subsystem being accessed. Limit and reduce the effects of network congestion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

Provided are a resource caching method and device. The method comprises: receiving a first request message, wherein the first request message is used for requesting to access a first predetermined resource; according to the first request message, counting the number of access times for accessing the first predetermined resource; judging whether the number of access times has reached a first predetermined threshold; when a judgement result is yes, notifying a caching sub-system to cache the first predetermined resource, wherein the caching sub-system is used for providing the cached resources. By means of the present invention, the problems of restricted access to a caching sub-system and network congestion existing in the related art are solved, thereby achieving the effects of avoiding restricted access to the caching sub-system and reducing network congestion.

Description

资源缓存方法及装置Resource cache method and device 技术领域Technical field
本发明涉及通信领域,具体而言,涉及一种资源缓存方法及装置。The present invention relates to the field of communications, and in particular to a resource caching method and apparatus.
背景技术Background technique
OTT(Over The Top)缓存(Cache)子系统部署在中国移动现有的互联网协议(Internet Protocol,简称为IP)承载网络中,用于存储互联网中的热点内容或特定内容。在相关技术中,在代理模式下,通过劫持网内用户的访问请求,将请求重定向转发到OTT Cache系统中,由OTT Cache系统作为代理服务器为用户提供所需的互联网访问数据,避免用户直接从互联网源站下载数据,从而有效规避网络链路的拥塞问题,降低中国移动用户访问互联网内容的时延,提升用户的数据业务体验,同时也可以降低因网间流量所产生的结算费用。The OTT (Over The Top) Cache (Cache) subsystem is deployed in China Mobile's existing Internet Protocol (IP) bearer network for storing hot content or specific content in the Internet. In the related art, in the proxy mode, the request is forwarded to the OTT Cache system by hijacking the access request of the user in the network, and the OTT Cache system is used as a proxy server to provide the user with the required Internet access data, thereby avoiding the direct use of the user. The data is downloaded from the Internet source station, thereby effectively avoiding the congestion problem of the network link, reducing the delay of the Chinese mobile user accessing the Internet content, improving the user's data service experience, and also reducing the settlement fee generated by the Internet traffic.
但在实际应用中,并不是所有统一资源定位符(Uniform Resource Locator,简称为url)都适合做302重定向处理,其中,当启用代理模式,即请求路由(Request Routing,简称为RR)对用户所有的请求都提供302重定向服务时,若用户请求的资源不包含头域或无跳转信息等,OTT Cache无法缓存该资源,此时不能用作代理服务器为用户提供服务。However, in practical applications, not all Uniform Resource Locators (URLs) are suitable for 302 redirection processing. When proxy mode is enabled, request routing (RR) is used for users. When all the requests provide the 302 redirect service, if the resource requested by the user does not include the header field or no jump information, the OTT Cache cannot cache the resource, and cannot be used as a proxy server to provide services for the user.
当启用代理模式时,若用户请求的资源OTT Cache可以缓存,但不属于热点资源(如某资源只被访问1次或2次)。此时RR如果做重定向处理,OTT Cache频繁向源站下拉内容,可能会被源站认为是恶意攻击,导致缓存子系统被限制访问。When proxy mode is enabled, if the resource OTT Cache requested by the user can be cached, it is not a hotspot resource (for example, a resource is only accessed once or twice). At this time, if the RR performs redirection processing, the OTT Cache frequently pulls down the content to the source station, which may be considered as a malicious attack by the source station, and the cache subsystem is restricted from accessing.
当不启用代理模式,即RR对所有请求都不提供302重定向服务,若用户请求的资源属于热点资源(相同资源被多次访问),用户不断向源站请求相同资源,占用大量带宽,容易造成网络拥塞,用户的体验也会较差。由此可知,在相关技术中可能导致的缓存子系统被限制访问以及网络拥塞的问题,目前尚未提出有效的解决方案。When the proxy mode is not enabled, the RR does not provide the 302 redirect service for all the requests. If the resource requested by the user belongs to the hotspot resource (the same resource is accessed multiple times), the user continuously requests the same resource from the source station, which occupies a large amount of bandwidth. The network is congested and the user experience is poor. It can be seen that the problem that the cache subsystem is restricted from accessing and network congestion may be caused in the related art, and an effective solution has not been proposed yet.
发明内容Summary of the invention
本发明提供了一种资源缓存方法及装置,以至少解决相关技术中存在的缓存子系统被限制访问以及网络拥塞的问题。The present invention provides a resource caching method and apparatus to at least solve the problem that the cache subsystem is restricted in access and network congestion existing in the related art.
根据本发明的一个方面,提供了一种资源缓存方法,包括:接收第一请求消息,其中,所述第一请求消息用于请求访问第一预定资源;根据所述第一请求消息统计访问所述第一预定资源的访问次数;判断所述访问次数是否达到第一预定阈值;在判断结果为是的情况下时,通知缓存子系统缓存所述第一预定资源,其中,所述缓存子系统用于提供缓存的资源。According to an aspect of the present invention, a resource caching method is provided, comprising: receiving a first request message, wherein the first request message is used to request access to a first predetermined resource; and the access station is counted according to the first request message Determining the number of accesses of the first predetermined resource; determining whether the number of accesses reaches a first predetermined threshold; and if the determination result is yes, notifying the cache subsystem to cache the first predetermined resource, wherein the cache subsystem The resource used to provide the cache.
可选地,在通知所述缓存子系统缓存所述第一预定资源之后,还包括:判断所述缓存子系统是否成功缓存所述第一预定资源;在判断结果为是的情况下,利用所述缓存子系统提供 所述第一预定资源。Optionally, after the buffering subsystem is configured to cache the first predetermined resource, the method further includes: determining whether the cache subsystem successfully caches the first predetermined resource; and if the determination result is yes, using the location Cache subsystem provides The first predetermined resource.
可选地,在判断所述缓存子系统是否成功缓存所述第一预定资源之后,还包括:在判断结果为是的情况下,将缓存的所述第一预定资源记录在缓存内容分布表中,其中,所述缓存内容分布表中记录有所述缓存子系统已经缓存的资源,所述缓存内容分布表用于将用于访问所述缓存内容分布表中记录的资源的请求消息重定向到所述缓存子系统,利用所述缓存子系统提供所述缓存内容分布表中记录的资源。Optionally, after determining whether the cache subsystem successfully caches the first predetermined resource, the method further includes: if the determination result is yes, recording the cached first predetermined resource in a cache content distribution table The cache content distribution table records the resources that the cache subsystem has cached, and the cache content distribution table is configured to redirect the request message for accessing the resources recorded in the cache content distribution table to The cache subsystem provides the resources recorded in the cache content distribution table by using the cache subsystem.
可选地,所述方法还包括:接收第二请求消息,其中,所述第二请求消息用于请求访问第二预定资源;判断所述缓存子系统中是否缓存有所述第二预定资源;在判断结果为是的情况下,将所述第二请求消息重定向到所述缓存子系统,其中,所述第二请求消息用于指示所述缓存子系统提供所述第二预定资源。Optionally, the method further includes: receiving a second request message, where the second request message is used to request access to the second predetermined resource; and determining whether the second predetermined resource is cached in the cache subsystem; If the determination result is yes, the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
可选地,判断所述访问次数是否达到第一预定阈值包括:获取热度统计表,其中,所述热度统计表用于记录访问资源的时间和次数;根据所述热度统计表判断所述访问次数是否达到所述第一预定阈值。Optionally, determining whether the number of accesses reaches the first predetermined threshold includes: acquiring a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and determining the number of accesses according to the heat statistics table. Whether the first predetermined threshold is reached.
可选地,所述方法还包括:根据所述热度统计表确定超期资源,其中,所述超期资源为当前时间与访问所述超期资源的最近的时间的差值超过第二预定阈值的资源;删除所述热度统计表中记录的访问所述超期资源的时间和次数。Optionally, the method further includes: determining, according to the heat statistics table, the expired resource, wherein the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold; The time and number of times of accessing the expired resource recorded in the heat statistics table are deleted.
根据本发明的另一方面,提供了一种资源缓存装置,包括:第一接收模块,设置为接收第一请求消息,其中,所述第一请求消息用于请求访问第一预定资源;统计模块,设置为根据所述第一请求消息统计访问所述第一预定资源的访问次数;第一判断模块,设置为判断所述访问次数是否达到第一预定阈值;通知模块,设置为在所述第一判断模块的判断结果为是的情况下时,通知缓存子系统缓存所述第一预定资源,其中,所述缓存子系统用于提供缓存的资源。According to another aspect of the present invention, a resource cache apparatus is provided, including: a first receiving module, configured to receive a first request message, wherein the first request message is used to request access to a first predetermined resource; a statistics module The first determining unit is configured to determine whether the number of accesses reaches the first predetermined threshold according to the first request message, and the notification module is configured to be in the first When the judgment result of the determination module is YES, the notification cache subsystem caches the first predetermined resource, wherein the cache subsystem is used to provide the cached resource.
可选地,所述装置还包括:第二判断模块,设置为判断所述缓存子系统是否成功缓存所述第一预定资源;提供模块,设置为在所述第二判断模块的判断结果为是的情况下,利用所述缓存子系统提供所述第一预定资源。Optionally, the device further includes: a second determining module, configured to determine whether the cache subsystem successfully caches the first predetermined resource; and a providing module, configured to be determined by the second determining module The first predetermined resource is provided by the cache subsystem.
可选地,所述装置还包括:记录模块,设置为在所述第二判断模块的判断结果为是的情况下,将缓存的所述第一预定资源记录在缓存内容分布表中,其中,所述缓存内容分布表中记录有所述缓存子系统已经缓存的资源,所述缓存内容分布表用于将用于访问所述缓存内容分布表中记录的资源的请求消息重定向到所述缓存子系统,利用所述缓存子系统提供所述缓存内容分布表中记录的资源。Optionally, the device further includes: a recording module, configured to: when the determination result of the second determining module is yes, record the cached first predetermined resource in a cache content distribution table, where The cache content distribution table records the resource that the cache subsystem has cached, and the cache content distribution table is configured to redirect a request message for accessing the resource recorded in the cache content distribution table to the cache a subsystem that utilizes the cache subsystem to provide resources recorded in the cache content distribution table.
可选地,所述装置还包括:第二接收模块,设置为接收第二请求消息,其中,所述第二请求消息用于请求访问第二预定资源;第三判断模块,设置为判断所述缓存子系统中是否缓存有所述第二预定资源;重定向模块,设置为在所述第三判断模块的判断结果为是的情况下,将所述第二请求消息重定向到所述缓存子系统,其中,所述第二请求消息用于指示所述缓存 子系统提供所述第二预定资源。Optionally, the device further includes: a second receiving module, configured to receive a second request message, where the second request message is used to request access to the second predetermined resource; and the third determining module is configured to determine the Whether the second predetermined resource is cached in the cache subsystem; and the redirecting module is configured to redirect the second request message to the cacher if the determination result of the third determining module is yes a system, wherein the second request message is used to indicate the cache The subsystem provides the second predetermined resource.
可选地,所述第一判断模块包括:获取单元,设置为获取热度统计表,其中,所述热度统计表用于记录访问资源的时间和次数;判断单元,设置为根据所述热度统计表判断所述访问次数是否达到所述第一预定阈值。Optionally, the first determining module includes: an obtaining unit, configured to obtain a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and the determining unit is configured to be configured according to the heat statistics table. Determining whether the number of accesses reaches the first predetermined threshold.
可选地,所述装置还包括:确定模块,设置为根据所述热度统计表确定超期资源,其中,所述超期资源为当前时间与访问所述超期资源的最近的时间的差值超过第二预定阈值的资源;删除模块,设置为删除所述热度统计表中记录的访问所述超期资源的时间和次数。Optionally, the apparatus further includes: a determining module, configured to determine an expired resource according to the heat statistics table, wherein the expired resource is a difference between a current time and a latest time of accessing the expired resource exceeds a second a resource of a predetermined threshold; a deletion module configured to delete a time and a number of times of accessing the expired resource recorded in the heat statistics table.
通过本发明,采用接收第一请求消息,其中,所述第一请求消息用于请求访问第一预定资源;根据所述第一请求消息统计访问所述第一预定资源的访问次数;判断所述访问次数是否达到第一预定阈值;在判断结果为是的情况下时,通知缓存子系统缓存所述第一预定资源,其中,所述缓存子系统用于提供缓存的资源。解决了相关技术中存在的缓存子系统被限制访问以及网络拥塞的问题,进而达到了避免缓存子系统被访问限制以及降低网络拥塞的效果。The first request message is received by the present invention, wherein the first request message is used to request access to the first predetermined resource; the number of accesses to the first predetermined resource is counted according to the first request message; Whether the number of accesses reaches a first predetermined threshold; if the result of the determination is YES, the notification cache subsystem caches the first predetermined resource, wherein the cache subsystem is configured to provide the cached resource. The problem that the cache subsystem is restricted access and network congestion existing in the related art is solved, thereby achieving the effect of avoiding the cache subsystem being restricted by access and reducing network congestion.
附图说明DRAWINGS
此处所说明的附图用来提供对本发明的进一步理解,构成本申请的一部分,本发明的示意性实施例及其说明用于解释本发明,并不构成对本发明的不当限定。在附图中:The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:
图1是根据本发明实施例的资源缓存方法的流程图;1 is a flowchart of a resource caching method according to an embodiment of the present invention;
图2是根据本发明实施例的资源缓存装置的结构框图;2 is a structural block diagram of a resource cache apparatus according to an embodiment of the present invention;
图3是根据本发明实施例的资源缓存装置的优选结构框图一;3 is a block diagram 1 of a preferred structure of a resource cache device according to an embodiment of the present invention;
图4是根据本发明实施例的资源缓存装置的优选结构框图二;4 is a block diagram 2 of a preferred structure of a resource cache device according to an embodiment of the present invention;
图5是根据本发明实施例的资源缓存装置的优选结构框图三;FIG. 5 is a block diagram 3 of a preferred structure of a resource cache device according to an embodiment of the present invention; FIG.
图6是根据本发明实施例的资源缓存装置中第一判断模块26的结构框图;FIG. 6 is a structural block diagram of a first determining module 26 in a resource cache device according to an embodiment of the present invention;
图7是根据本发明实施例的资源缓存装置的优选结构框图四;FIG. 7 is a block diagram 4 of a preferred structure of a resource cache device according to an embodiment of the present invention; FIG.
图8是根据本发明实施例的用户请求调度与热度统计流程图;FIG. 8 is a flowchart of user request scheduling and heat statistics according to an embodiment of the present invention; FIG.
图9是根据本发明实施例的阈值统计及内容下拉流程图;9 is a flowchart of threshold statistics and content pull-down according to an embodiment of the present invention;
图10是根据本发明实施例的内容缓存结果上报流程图;FIG. 10 is a flowchart of content cache result reporting according to an embodiment of the present invention; FIG.
图11是根据本发明实施例的内容分发网络系统的简易组网图;11 is a simplified networking diagram of a content distribution network system according to an embodiment of the present invention;
图12是根据本发明实施例的非代理模式处理机制图。12 is a diagram of a non-proxy mode processing mechanism in accordance with an embodiment of the present invention.
具体实施方式 detailed description
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是,在不冲突的情况下,本申请中的实施例及实施例中的特征可以相互组合。The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.
需要说明的是,本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order.
在本实施例中提供了一种资源缓存方法,图1是根据本发明实施例的资源缓存方法的流程图,如图1所示,该流程包括如下步骤:In this embodiment, a resource caching method is provided. FIG. 1 is a flowchart of a resource caching method according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps:
步骤S102,接收第一请求消息,其中,该第一请求消息用于请求访问第一预定资源;Step S102, receiving a first request message, where the first request message is used to request access to the first predetermined resource;
步骤S104,根据上述第一请求消息统计访问第一预定资源的访问次数;Step S104, counting the number of accesses to access the first predetermined resource according to the first request message;
步骤S106,判断上述访问次数是否达到第一预定阈值;Step S106, determining whether the number of accesses reaches a first predetermined threshold;
步骤S108,在判断结果为是的情况下时,通知缓存子系统缓存上述第一预定资源,其中,该缓存子系统用于提供缓存的资源。Step S108: When the determination result is yes, the notification cache subsystem caches the first predetermined resource, where the cache subsystem is used to provide the cached resource.
通过上述步骤,在通知缓存子系统缓存资源时,仅通知缓存子系统缓存访问次数超过了第一预定阈值的资源,从而实现了有选择的缓存资源的目的,即,针对访问次数达到第一预定阈值(即,该资源足够热门,访问的用户比较多)的资源进行缓存,从而避免了缓存子系统被限制访问的问题,当用户再次访问该资源时,可以直接由缓存子系统提供该资源,从而减轻网络拥塞。解决了相关技术中存在的缓存子系统被限制访问以及网络拥塞的问题,进而达到了避免缓存子系统被访问限制以及降低网络拥塞的效果。Through the above steps, when the cache subsystem is notified of the cache resource, only the cache subsystem caches the resources whose cache access times exceed the first predetermined threshold, thereby achieving the purpose of the selected cache resource, that is, the first reservation is reached for the number of accesses. The resource of the threshold (that is, the resource is hot enough and the number of users accessed is relatively large) is cached, thereby avoiding the problem that the cache subsystem is restricted from accessing. When the user accesses the resource again, the resource can be directly provided by the cache subsystem. Thereby reducing network congestion. The problem that the cache subsystem is restricted access and network congestion existing in the related art is solved, thereby achieving the effect of avoiding the cache subsystem being restricted by access and reducing network congestion.
在一个可选的实施例中,在通知上述缓存子系统缓存上述第一预定资源之后,还包括:判断该缓存子系统是否成功缓存上述第一预定资源;在判断结果为是的情况下,利用该缓存子系统提供上述第一预定资源。并且,通过缓存子系统是否成功缓存资源(如,上述的第一预定资源)可以确定该缓存子系统是否支持缓存上述资源。In an optional embodiment, after the buffering subsystem is configured to buffer the first predetermined resource, the method further includes: determining whether the cache subsystem successfully caches the first predetermined resource; and if the determination result is yes, using The cache subsystem provides the first predetermined resource described above. Moreover, whether the cache subsystem supports the caching of the resources can be determined by whether the cache subsystem successfully caches resources (eg, the first predetermined resource described above).
在一个可选的实施例中,在判断上述缓存子系统是否成功缓存第一预定资源之后,还包括:在判断结果为是的情况下,将缓存的上述第一预定资源记录在缓存内容分布表中,其中,该缓存内容分布表中记录有上述缓存子系统已经缓存的资源,该缓存内容分布表用于将用于访问上述缓存内容分布表中记录的资源的请求消息重定向到缓存子系统,利用该缓存子系统提供缓存内容分布表中记录的资源。也就是说,上述的缓存内容分布表中记录的资源都是该缓存子系统成功缓存的资源,从而当用户需要访问该缓存内容分布表中记录的资源时,可以直接利用该缓存子系统提供资源,从而有效的减轻网络的压力,降低网络拥塞。本实施例中的请求消息可以为第一请求消息、第二请求消息,或者其他请求消息。In an optional embodiment, after determining whether the cache subsystem successfully caches the first predetermined resource, the method further includes: when the determination result is yes, recording the cached first predetermined resource in the cache content distribution table. The cache content distribution table records the resource that has been cached by the cache subsystem, and the cache content distribution table is used to redirect the request message for accessing the resource recorded in the cache content distribution table to the cache subsystem. The cache subsystem is used to provide resources recorded in the cache content distribution table. That is to say, the resources recorded in the cache content distribution table are all resources successfully cached by the cache subsystem, so that when the user needs to access the resources recorded in the cache content distribution table, the cache subsystem can be directly used to provide resources. So as to effectively reduce the pressure on the network and reduce network congestion. The request message in this embodiment may be a first request message, a second request message, or other request message.
在一个可选的实施例中,在步骤S102中接收第一请求消息时,同时可以接收第二请求消息,其中,该第二请求消息用于请求访问第二预定资源;判断缓存子系统中是否缓存有第二预定资源;在判断结果为是的情况下,将第二请求消息重定向到缓存子系统,其中,第二请求消息用于指示缓存子系统提供第二预定资源。从而当缓存子系统中已经缓存有该第二预定 资源的时候,可以直接将第二请求消息重定向到缓存子系统中,从而提高了获得第二预定资源的效率。在该可选实施例中,第一预定资源和第二预定资源可以为相同的资源,也可以为不同的资源。并且,第二请求消息的接收时机可以发生在任何时刻,比如可以在步骤S104之前或之后,或者在步骤S108之前或者之后。In an optional embodiment, when the first request message is received in step S102, the second request message may be received at the same time, wherein the second request message is used to request access to the second predetermined resource; The cache has a second predetermined resource; if the result of the determination is yes, the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource. Thereby the second reservation is already cached in the cache subsystem When the resource is used, the second request message can be directly redirected to the cache subsystem, thereby improving the efficiency of obtaining the second predetermined resource. In this alternative embodiment, the first predetermined resource and the second predetermined resource may be the same resource or different resources. And, the timing of receiving the second request message may occur at any time, such as before or after step S104, or before or after step S108.
在一个可选的实施例中,判断上述访问次数是否达到第一预定阈值包括:获取热度统计表,其中,该热度统计表用于记录访问资源的时间和次数;根据该热度统计表判断访问次数是否达到上述第一预定阈值。其中,该热度统计表就是用于记录用户曾经访问过的资源的访问次数和访问时间的,且每访问一次某个资源,对应该资源的访问次数会相应加1,同时记录的还有对应于该次访问的访问时间,从而清楚的确定资源的访问情况。In an optional embodiment, determining whether the number of accesses reaches the first predetermined threshold includes: acquiring a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; and determining the number of accesses according to the heat statistics table. Whether the above first predetermined threshold is reached. The heat statistics table is used to record the number of accesses and access times of the resources that the user has visited, and each time a resource is accessed, the number of accesses to the corresponding resources is increased by one, and the corresponding records are also corresponding to The access time of the visit, so as to clearly determine the access of resources.
在一个可选的实施例中,还可以根据上述热度统计表确定超期资源,其中,该超期资源为当前时间与访问超期资源的最近的时间的差值超过第二预定阈值的资源;删除该热度统计表中记录的访问上述超期资源的时间和次数。其中,该第二预定阈值可以设置为10天,或者5天,或者其他的值,上述的超期资源即访问热度降低的资源,当某个资源未被访问的时间超过了上述第二预定阈值时,即可确定该资源的访问热度已足够低,将超期资源的记录在热度统计表中删除,从而保证该热度统计表能够统计更多的资源的访问时间及次数。In an optional embodiment, the expired resource may also be determined according to the heat statistics table, where the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold; deleting the heat The time and number of times recorded in the statistics table to access the above-mentioned expired resources. The second predetermined threshold may be set to 10 days, or 5 days, or other values. The foregoing expired resource is a resource with reduced access, when a resource is not accessed for longer than the second predetermined threshold. Then, it can be determined that the access heat of the resource is low enough, and the record of the expired resource is deleted in the heat statistics table, thereby ensuring that the heat statistics table can count more access times and times of resources.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,或者网络设备等)执行本发明各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.
在本实施例中还提供了一种资源缓存装置,该装置用于实现上述实施例及优选实施方式,已经进行过说明的不再赘述。如以下所使用的,术语“模块”可以实现预定功能的软件或者软件和硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现,但是软件和硬件的组合的实现也是可能并被构想的。In the embodiment, a resource buffering device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and has not been described again. As used hereinafter, the term "module" may implement software of a predetermined function or a combination of software and hardware. Although the apparatus described in the following embodiments is preferably implemented in software, implementation of a combination of software and hardware is also possible and contemplated.
图2是根据本发明实施例的资源缓存装置的结构框图,如图2所示,该装置包括第一接收模块22、统计模块24、第一判断模块26和通知模块28,下面对该装置进行说明。FIG. 2 is a structural block diagram of a resource cache apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus includes a first receiving module 22, a statistic module 24, a first judging module 26, and a notification module 28. Be explained.
第一接收模块22,设置为接收第一请求消息,其中,该第一请求消息用于请求访问第一预定资源;统计模块24,连接至上述第一接收模块22,设置为根据上述第一请求消息统计访问该第一预定资源的访问次数;第一判断模块26,连接至上述统计模块24,设置为判断上述访问次数是否达到第一预定阈值;通知模块28,连接至上述第一判断模块26,设置为在上述第一判断模块26的判断结果为是的情况下时,通知缓存子系统缓存上述第一预定资源,其中,该缓存子系统用于提供缓存的资源。The first receiving module 22 is configured to receive a first request message, where the first request message is used to request access to the first predetermined resource, and the statistics module 24 is connected to the first receiving module 22, and configured to be according to the first request. The number of accesses of the first predetermined resource is accessed by the message statistics. The first determining module 26 is connected to the statistics module 24, and is configured to determine whether the number of accesses reaches a first predetermined threshold. The notification module 28 is connected to the first determining module 26. And being configured to notify the cache subsystem to cache the first predetermined resource when the determination result of the first determining module 26 is YES, wherein the cache subsystem is configured to provide the cached resource.
图3是根据本发明实施例的资源缓存装置的优选结构框图一,如图3所示,该装置除包括图2所示的所有模块外,还包括第二判断模块32和提供模块34,下面对该装置进行说明。 3 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 3, the device includes a second judging module 32 and a providing module 34, in addition to all the modules shown in FIG. The device will be described.
第二判断模块32,连接至上述通知模块28,设置为判断上述缓存子系统是否成功缓存上述第一预定资源;提供模块34,连接至上述第二判断模块32,设置为在上述第二判断模块32的判断结果为是的情况下,利用上述缓存子系统提供上述第一预定资源。The second determining module 32 is connected to the notification module 28, and is configured to determine whether the cache subsystem successfully caches the first predetermined resource. The providing module 34 is connected to the second determining module 32, and is configured to be in the second determining module. In the case where the judgment result of 32 is YES, the above-mentioned first predetermined resource is provided by the above-described cache subsystem.
图4是根据本发明实施例的资源缓存装置的优选结构框图二,如图4所示,该装置除包括图3所示的所有模块外,还包括记录模块42,下面对该装置进行说明。4 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 4, the device includes a recording module 42 in addition to all the modules shown in FIG. .
记录模块42,连接至上述第二判断模块32,设置为在上述第二判断模块32的判断结果为是的情况下,将缓存的上述第一预定资源记录在缓存内容分布表中,其中,该缓存内容分布表中记录有缓存子系统已经缓存的资源,该缓存内容分布表用于将用于访问上述缓存内容分布表中记录的资源的请求消息重定向到缓存子系统,利用该缓存子系统提供缓存内容分布表中记录的资源。The recording module 42 is connected to the second judging module 32, and configured to record the cached first predetermined resource in the cache content distribution table if the judgment result of the second judging module 32 is YES, wherein the The cache content distribution table records the resources that the cache subsystem has cached, and the cache content distribution table is used to redirect the request message for accessing the resources recorded in the cache content distribution table to the cache subsystem, and utilize the cache subsystem. Provides resources recorded in the cached content distribution table.
图5是根据本发明实施例的资源缓存装置的优选结构框图三,如图5所示,该装置除了包括图2所示的所有模块外,还包括第二接收模块52、第三判断模块54和重定向模块,第二接收模块52可以连接于第一接收模块22、统计模块24、第一判断模块26或者通知模块28,下面以第二接收模块52连接于第一接收模块22为例,对该装置进行说明。5 is a block diagram 3 of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 5, the device includes a second receiving module 52 and a third determining module 54 in addition to all the modules shown in FIG. And the redirection module, the second receiving module 52 can be connected to the first receiving module 22, the statistic module 24, the first judging module 26 or the notification module 28, and the second receiving module 52 is connected to the first receiving module 22 as an example. The device will be described.
第二接收模块52,连接于第一接收模块22,设置为接收第二请求消息,其中,第二请求消息用于请求访问第二预定资源;第三判断模,块54,连接于第二接收模块52,设置为判断缓存子系统中是否缓存有第二预定资源;重定向模块56,连接于第三判断模块54,设置为在第三判断模块54的判断结果为是的情况下,将第二请求消息重定向到缓存子系统,其中,第二请求消息用于指示缓存子系统提供第二预定资源。The second receiving module 52 is connected to the first receiving module 22 and configured to receive the second request message, where the second request message is used to request access to the second predetermined resource; the third determining module, the block 54 is connected to the second receiving The module 52 is configured to determine whether the second predetermined resource is cached in the cache subsystem. The redirecting module 56 is connected to the third determining module 54 and is configured to be in the case that the determination result of the third determining module 54 is YES. The second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
图6是根据本发明实施例的资源缓存装置中第一判断模块26的结构框图,如图6所示,该第一判断模块26包括获取单元62和判断单元64,下面对该第一判断模块26进行说明。FIG. 6 is a structural block diagram of a first determining module 26 in a resource cache device according to an embodiment of the present invention. As shown in FIG. 6, the first determining module 26 includes an obtaining unit 62 and a determining unit 64. Module 26 is described.
获取单元62,设置为获取热度统计表,其中,该热度统计表用于记录访问资源的时间和次数;判断单元64,连接至上述获取单元52,设置为根据上述热度统计表判断访问次数是否达到上述第一预定阈值。The obtaining unit 62 is configured to obtain a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed; the determining unit 64 is connected to the obtaining unit 52, and is configured to determine whether the number of accesses is reached according to the heat statistics table. The first predetermined threshold described above.
图7是根据本发明实施例的资源缓存装置的优选结构框图四,如图7所示,该装置除包括图6所示的所有模块外,还包括确定模块72和删除模块74,下面对该装置进行说明。FIG. 7 is a block diagram of a preferred structure of a resource cache device according to an embodiment of the present invention. As shown in FIG. 7, the device includes a determination module 72 and a deletion module 74, in addition to all the modules shown in FIG. The device is described.
确定模块72,连接至上述第一判断模块26,设置为根据上述热度统计表确定超期资源,其中,该超期资源为当前时间与访问上述超期资源的最近的时间的差值超过第二预定阈值的资源;删除模块74,连接至上述确定模块72,设置为删除上述热度统计表中记录的访问该超期资源的时间和次数。其中,上述的确定模块72和图6中所示的各模块的连接关系可以有多种,图7中所示的仅是一种示例,例如,该确定模块72还可以连接于上述的通知模块28之后。The determining module 72 is connected to the first determining module 26, and is configured to determine the expired resource according to the heat statistics table, wherein the expired resource is that the difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold. The resource deletion module 74 is connected to the determining module 72, and is configured to delete the time and the number of times the access to the expired resource recorded in the heat statistics table. The connection relationship between the above-mentioned determination module 72 and each module shown in FIG. 6 can be various. The example shown in FIG. 7 is only an example. For example, the determination module 72 can also be connected to the notification module. After 28.
其中,上述的各个方法的执行的主体可以是RR,上述的各个装置可以应用于RR中。 The main body of execution of each of the foregoing methods may be an RR, and each of the foregoing devices may be applied to the RR.
下面结合RR对本发明实施例进行说明:The embodiments of the present invention are described below in conjunction with RR:
为解决相关技术中存在的上述问题,本发明实施例中提供了一种非代理模式的超文本传输协议(HyperText Transfer Protocol,简称为HTTP)请求调度方法,根据缓存子系统能否对请求的资源提供缓存服务,判断是否对HTTP请求做302重定向处理,以及缓存子系统何时进行内容缓存。该方法包括:In order to solve the above problems in the related art, the embodiment of the present invention provides a non-proxy mode Hypertext Transfer Protocol (HTTP) request scheduling method, according to whether the cache subsystem can request the resources. A cache service is provided to determine whether to perform 302 redirect processing on the HTTP request and when the cache subsystem performs content caching. The method includes:
1、对于缓存子系统无法缓存的内容,RR不做302重定向处理,由源站直接为用户提供服务,避免缓存子系统无法提供服务造成用户体验差的问题。1. For content that cannot be cached by the cache subsystem, the RR does not perform the 302 redirection process, and the source station directly provides services to the user, so as to avoid the problem that the cache subsystem cannot provide the service and the user experience is poor.
2、对于缓存子系统可以缓存的内容,当用户请求的内容不属于热点资源,RR不对该请求进行重定向,而只做热度统计,直接由源站为用户提供服务,避免缓存子系统频繁访问源站可能导致被限制访问。2. For the content that the cache subsystem can cache, when the content requested by the user does not belong to the hotspot resource, the RR does not redirect the request, but only performs the heat statistics, and directly provides the service to the user by the source station to avoid frequent access by the cache subsystem. The source station may result in restricted access.
3、当用户请求的内容属于热点资源,且访问次数达到访问热度阈值时,RR通知OTT Cache下拉内容,后续再有用户请求相同资源时,RR将请求重定向转发到OTT Cache,由缓存子系统直接提供服务,避免用户对源站直接大量访问造成网络拥塞,达到节省带宽、提高用户体验的效果,同时也提高了OTT Cache的缓存命中率。3. When the content requested by the user belongs to the hotspot resource, and the number of accesses reaches the access heat threshold, the RR notifies the OTT Cache of the drop-down content. When the user requests the same resource, the RR forwards the request to the OTT Cache, and the cache subsystem Providing services directly, avoiding network congestion caused by direct large-scale access to the source station, saving bandwidth and improving user experience, and improving the cache hit ratio of OTT Cache.
本发明实施例中的方案的实现可以依赖于两张表的设计:url热度统计表(同上述的热度统计表)和缓存内容分布表。url热度统计表用于统计url访问热度,缓存内容分布表用于记录缓存子系统可以为用户提供服务的资源。The implementation of the solution in the embodiment of the present invention may depend on the design of two tables: a url heat statistics table (same as the above heat statistics table) and a cache content distribution table. The url heat statistics table is used to count the url access heat. The cache content distribution table is used to record the resources that the cache subsystem can provide services for users.
图8是根据本发明实施例的用户请求调度与热度统计流程图,如图8所示,该流程包括如下步骤:FIG. 8 is a flow chart of user request scheduling and heat statistics according to an embodiment of the present invention. As shown in FIG. 8, the process includes the following steps:
步骤S802,用户发起HTTP请求;Step S802, the user initiates an HTTP request.
步骤S804,根据黑白名单、IP段等规则,判断该url(即,上述的HTTP请求)是否需要被劫持,在判断结果为需要时,转至步骤S806,否则,转至步骤S810;Step S804, according to the black and white list, IP segment and other rules, determine whether the url (ie, the above HTTP request) needs to be hijacked, when the determination result is necessary, go to step S806, otherwise, go to step S810;
步骤S806,当确定用户的HTTP请求需要被RR劫持时,RR根据用户访问url,在热度统计表中记录本次请求(步骤S812-S814),同时判断该内容在缓存内容分布表中是否存在,如果存在,转至步骤S808,否则,转至步骤S810;Step S806, when it is determined that the HTTP request of the user needs to be hijacked by the RR, the RR records the current request in the heat statistics table according to the user access url (steps S812-S814), and determines whether the content exists in the cache content distribution table. If yes, go to step S808, otherwise, go to step S810;
步骤S808,响应用户请求,重定向到OTT Cache,利用该缓存子系统提供服务;Step S808, responding to the user request, redirecting to the OTT Cache, and providing the service by using the cache subsystem;
步骤S810,忽略本次请求,让用户直接从源站请求下载上述资源;Step S810, ignoring the current request, and allowing the user to directly download the foregoing resource from the source station;
步骤S812,判断热度统计表中是否有该url的访问记录,在判断结果为不存在的情况下,转至步骤S814,否则,转至步骤S816;Step S812, determining whether there is an access record of the url in the heat statistics table, if the result of the determination is non-existent, go to step S814, otherwise, go to step S816;
步骤S814,热度统计表新增记录,访问次数记为1,并记录请求时间;记录的统计项包括:a.key:用户请求的完整url使用md5算法计算得出的32位加密串为key。In step S814, the heat statistics table adds a record, the number of accesses is recorded as 1, and the request time is recorded; the recorded statistics items include: a.key: the complete url requested by the user uses the md5 algorithm to calculate the 32-bit encrypted string as the key.
b.服务节点id:用户本次调度到的服务节点的id号。 b. Service node id: The id number of the service node that the user has scheduled this time.
c.访问次数:每有一次用户访问,计数加1。c. Number of visits: Each time a user visits, the count is incremented by 1.
d.最后一次访问时间:记录最后一次用户访问的时间,单位毫秒。d. Last access time: Record the time of the last user visit, in milliseconds.
e.热点阈值(同上述的第一预定阈值):阈值的获取,需要根据用户访问的url从HTTP域名黑白名单列表中匹配到相应的域名记录,从记录的条目中获取该url的热点阈值大小。针对不同的网站,系统设置不同的阈值,通过HTTP域名黑白名单接口下发到RR上。当访问次数达到此阈值时,RR通知OTT Cache进行缓存。OTT Cache将内容下拉完成后,通知RR进行内容更新,RR收到通知后更新内存数据,并保存到内容分布表中,热点阈值统计和内容下拉流程如附图9所示,内容上报流程如附图10所示。e. The hotspot threshold (same first threshold as described above): the threshold is obtained, and the url of the user is matched from the blacklist list of the HTTP domain name to the corresponding domain name record, and the hotspot threshold of the url is obtained from the recorded entry. . For different websites, the system sets different thresholds and sends them to the RR through the HTTP domain name blacklist and whitelist interface. When the number of accesses reaches this threshold, the RR notifies the OTT Cache to cache. After the content is pulled down, the OTT Cache notifies the RR to update the content. After receiving the notification, the RR updates the memory data and saves it in the content distribution table. The hotspot threshold statistics and content pull-down process are as shown in Figure 9, and the content reporting process is attached. Figure 10 shows.
步骤S816,如果热度表中有该url对应key的记录,将此条记录的访问次数加1,最后一次访问时间更改为当前时间。Step S816, if there is a record corresponding to the key of the url in the heat meter, the number of accesses of the record is increased by 1, and the last access time is changed to the current time.
其中,可以根据用户请求的url,计算一个md5值,作为记录key。通过内容分布表,查询该key对应内容在此节点上是否已经被缓存。如果内容存在,响应用户请求,将用户请求重定向到缓存子系统。如果不存在,则忽略用户请求,让用户直接回源。The md5 value can be calculated as the record key according to the url requested by the user. Through the content distribution table, it is checked whether the content corresponding to the key has been cached on this node. If the content exists, the user request is redirected to the cache subsystem in response to the user request. If it does not exist, the user request is ignored and the user is returned directly to the source.
图9是根据本发明实施例的阈值统计及内容下拉流程图,如图9所示,该流程包括如下步骤:FIG. 9 is a flowchart of threshold statistics and content pull-down according to an embodiment of the present invention. As shown in FIG. 9, the process includes the following steps:
步骤S902,RR后台进程,定时扫描热度统计表,依次处理每条记录;Step S902, the RR background process, periodically scans the heat statistics table, and processes each record in turn;
步骤S904,取出下一条记录;Step S904, taking out the next record;
步骤S906,判断该记录是否已经超期(当前时间减去url最后一次访问时间是否大于或等于超期时间),在判断结果为是的情况下,确定url超期,转至步骤S908,否则,转至步骤S910;In step S906, it is determined whether the record has expired (the current time minus whether the last access time of the url is greater than or equal to the expired time). If the result of the determination is yes, it is determined that the url is overdue, and the process goes to step S908; otherwise, the process goes to step S910;
步骤S908,删除该url相关记录;Step S908, deleting the url related record;
步骤S910,判断访问次数是否大于或等于热点阈值,在判断结果为是的情况下,转至步骤S912,否则,转至步骤S916;Step S910, determining whether the number of accesses is greater than or equal to the hotspot threshold, if the determination result is yes, go to step S912, otherwise, go to step S916;
步骤S912,根据url的key值以及节点id等信息判断缓存内容表中是否有该url记录,在判断结果为是的情况下,转至步骤S916,否则,转至步骤S914;Step S912, according to the key value of the url and the node id and other information to determine whether there is the url record in the cache content table, if the determination result is yes, then go to step S916, otherwise, go to step S914;
步骤S914,通知OTT Cache下载该url对应的内容;Step S914, notifying the OTT Cache to download the content corresponding to the url;
步骤S916,则表明该内容已缓存,不需要通知OTT Cache,本条热度记录处理完毕;Step S916, indicating that the content is cached, and does not need to notify the OTT Cache, and the hot record processing is completed.
步骤S918,判断是否还有未处理的记录,在判断结果为是的情况下,转至步骤S904,否则,转至步骤S920;Step S918, it is determined whether there are still unprocessed records, if the determination result is yes, go to step S904, otherwise, go to step S920;
步骤S920,等待用户的访问请求。Step S920, waiting for an access request of the user.
从上述的流程中可以看出,RR可以找出达到热点阈值的记录,通知缓存子系统进行内容缓存,同时清理长期未被再次访问的url记录。RR需要支持对不同的网站或者文件后缀名设 定不同的阈值,如针对www.sohu.com或sina.com,可以分别设置不同的缓存阈值策略。当RR收到一个用户的HTTP下载请求,根据匹配规则,在表内找到对应的匹配规则记录,进而找到针对本次请求url的一个热度阈值,保存到内容热度统计表中。当用户请求同一资源的次数达到网站缓存热度阈值时,缓存子系统向源站发出请求下拉内容。内容被成功缓存后,在其有效期内,若有用户再次请求相同资源,RR通过查询缓存内容分布表判断出缓存子系统可以提供服务,将转发该请求到缓存子系统,由缓存子系统直接为该用户提供内容服务。As can be seen from the above process, the RR can find the record that reaches the hotspot threshold, notify the cache subsystem to perform content caching, and clean up the url records that have not been accessed again for a long time. RR needs to support different website or file suffixes. Different thresholds, such as www.sohu.com or sina.com, can be set separately for different cache threshold policies. When the RR receives a user's HTTP download request, the RR finds the corresponding matching rule record in the table according to the matching rule, and then finds a heat threshold for the current request url, and saves it to the content heat statistics table. When the number of times the user requests the same resource reaches the website cache popularity threshold, the cache subsystem issues a request drop-down to the source station. After the content is successfully cached, if the user requests the same resource again during the validity period, the RR determines that the cache subsystem can provide the service by querying the cache content distribution table, and forwards the request to the cache subsystem, which is directly This user provides content services.
图10是根据本发明实施例的内容缓存结果上报流程图,如图10所示,该流程包括如下步骤:FIG. 10 is a flowchart of content cache result reporting according to an embodiment of the present invention. As shown in FIG. 10, the process includes the following steps:
步骤S1002,RR主程序在启动时,首先从数据库(磁盘)中加载内容分布表,将内容分布数据保存在内存缓存中,供调度进程使用。(对应于图10中的步骤S1-S3)In step S1002, when the RR main program starts, the content distribution table is first loaded from the database (disk), and the content distribution data is saved in the memory cache for use by the scheduling process. (corresponding to steps S1-S3 in Fig. 10)
步骤S1004,当缓存子系统有内容更新时,通知RR进行内容更新。(对应于图10中的步骤S4)In step S1004, when the cache subsystem has content update, the RR is notified to perform content update. (corresponding to step S4 in Fig. 10)
步骤S1006,RR更新内存缓存中的相关记录,新增/删除。(对应于图10中的步骤S5)In step S1006, the RR updates related records in the memory cache, adding/deleting. (corresponding to step S5 in Fig. 10)
步骤S1008,RR将记录持久化到磁盘中。(对应于图10中的步骤S6)In step S1008, the RR persists the record to the disk. (corresponding to step S6 in Fig. 10)
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present invention will be clearly and completely described in the following with reference to the accompanying drawings.
图11是根据本发明实施例的内容分发网络(Content Delivery Network,简称为CDN)系统的简易组网图,整个系统主要包括调度子系统和缓存子系统。RR调度子系统对用户的HTTP请求进行劫持和重定向处理,缓存子系统由SLB7和OTT Cache共同组成,当RR将用户的HTTP请求重定向到缓存子系统后,缓存子系统为用户提供服务。FIG. 11 is a simplified networking diagram of a Content Delivery Network (CDN) system according to an embodiment of the present invention. The entire system mainly includes a scheduling subsystem and a cache subsystem. The RR scheduling subsystem hijacks and redirects the user's HTTP request. The cache subsystem is composed of SLB7 and OTT Cache. When the RR redirects the user's HTTP request to the cache subsystem, the cache subsystem provides services for the user.
图12是根据本发明实施例的非代理模式处理机制图,该图12描述了RR在劫持到用户的HTTP下载请求后,缓存子系统未命中、达到热度阈值进行内容下拉(对应于图12中的达到阈值代理下载)以及缓存命中的处理流程。如图12所示,该流程包括如下步骤:12 is a diagram of a non-proxy mode processing mechanism according to an embodiment of the present invention. The FIG. 12 depicts that after the RR is hijacked to the user's HTTP download request, the cache subsystem misses and reaches a heat threshold for content pull-down (corresponding to FIG. 12). The process of reaching the threshold proxy download) and the cache hit. As shown in FIG. 12, the process includes the following steps:
缓存子系统未命中的处理流程包括:The processing flow of the cache subsystem miss includes:
步骤S1202,用户发送HTTP文件下载请求;Step S1202: The user sends an HTTP file download request.
步骤S1204,RR对上述下载请求进行劫持;Step S1204, the RR hijacks the download request.
步骤S1206,确定缓存子系统中未缓存上述文件(对应于上述的资源),忽略此次用户的HTTP文件下载请求;Step S1206, determining that the file is not cached in the cache subsystem (corresponding to the foregoing resource), and ignoring the HTTP file download request of the user;
步骤S1208,源站为用户提供文件下载服务。In step S1208, the source station provides a file download service for the user.
达到热度阈值进行内容下拉流程包括:The content pulldown process that reaches the heat threshold includes:
步骤S1210,热度统计表中的访问次数+1; Step S1210, the number of accesses in the heat statistics table is +1;
步骤S1212,达到上述热点阈值,通知OTT Cache下载;Step S1212: The hotspot threshold is reached, and the OTT Cache is notified to download.
步骤S1214,OTT Cache下载文件成功。In step S1214, the OTT Cache downloads the file successfully.
缓存命中的处理流程包括:The processing flow of cache hits includes:
步骤S1216,用户发送HTTP文件下载请求;Step S1216, the user sends an HTTP file download request.
步骤S1218,RR对上述下载请求进行劫持;Step S1218, the RR hijacks the above download request;
步骤S1220,确定缓存子系统中已经缓存了上述文件,通知用户重定向至缓存子系统;Step S1220, determining that the foregoing file has been cached in the cache subsystem, and notifying the user to redirect to the cache subsystem;
步骤S1222,将用户的HTTP文件下载请求重定向至OTT Cache;Step S1222, redirecting the HTTP file download request of the user to the OTT Cache;
步骤S1224,OTT Cache为用户提供文件下载服务。In step S1224, the OTT Cache provides a file download service for the user.
本发明实施例中实现了一种非代理模式的HTTP请求调度方法,调度子系统通过热度统计表的维护判断缓存子系统何时下拉内容,通过缓存内容分布表记录内容缓存情况以及判断缓存子系统能否为用户提供服务。调度子系统根据缓存子系统能否对请求的资源提供缓存服务,判断是否对HTTP请求做302重定向处理。In the embodiment of the present invention, a non-proxy mode HTTP request scheduling method is implemented. The scheduling subsystem determines when the cache subsystem pulls down the content through the maintenance of the heat statistics table, records the content cache condition through the cache content distribution table, and determines the cache subsystem. Can you provide services to users? The scheduling subsystem determines whether to perform 302 redirect processing on the HTTP request according to whether the cache subsystem can provide a cache service to the requested resource.
需要说明的是,上述各个模块是可以通过软件或软件与硬件的结合来实现的,对于后者,可以通过以下方式实现,但不限于此:上述模块均位于同一处理器中;或者,上述模块分别位于多个处理器中。It should be noted that each of the above modules may be implemented by software or a combination of software and hardware. For the latter, the foregoing may be implemented by, but not limited to, the above modules are all located in the same processor; or, the above modules Located in multiple processors.
本发明的实施例还提供了一种存储介质。可选地,在本实施例中,上述存储介质可以被设置为存储用于执行以下步骤的程序代码:Embodiments of the present invention also provide a storage medium. Optionally, in the embodiment, the foregoing storage medium may be configured to store program code for performing the following steps:
S11,接收第一请求消息,其中,该第一请求消息用于请求访问第一预定资源;S11. The first request message is received, where the first request message is used to request access to the first predetermined resource.
S12,根据上述第一请求消息统计访问第一预定资源的访问次数;S12. Count, according to the foregoing first request message, the number of accesses to access the first predetermined resource.
S13,判断上述访问次数是否达到第一预定阈值;S13. Determine whether the number of accesses reaches a first predetermined threshold.
S14,在判断结果为是的情况下时,通知缓存子系统缓存上述第一预定资源,其中,该缓存子系统用于提供缓存的资源S14. When the determination result is yes, notify the cache subsystem to cache the first predetermined resource, where the cache subsystem is configured to provide the cached resource.
可选地,在本实施例中,上述存储介质可以包括但不限于:U盘、只读存储器(Read-Only Memory,简称为ROM)、随机存取存储器(Random Access Memory,简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。Optionally, in the embodiment, the foregoing storage medium may include, but is not limited to, a USB flash drive, a Read-Only Memory (ROM), and a Random Access Memory (RAM). A variety of media that can store program code, such as a hard disk, a disk, or an optical disk.
可选地,在本实施例中,处理器根据存储介质中已存储的程序代码执行上述步骤S11-S14。Optionally, in the embodiment, the processor performs the above steps S11-S14 according to the stored program code in the storage medium.
可选地,本实施例中的具体示例可以参考上述实施例及可选实施方式中所描述的示例,本实施例在此不再赘述。For example, the specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the optional embodiments, and details are not described herein again.
显然,本领域的技术人员应该明白,上述的本发明的各模块或各步骤可以用通用的计算装置来实现,它们可以集中在单个的计算装置上,或者分布在多个计算装置所组成的网络上, 可选地,它们可以用计算装置可执行的程序代码来实现,从而,可以将它们存储在存储装置中由计算装置来执行,并且在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤,或者将它们分别制作成各个集成电路模块,或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样,本发明不限制于任何特定的硬件和软件结合。It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. On, Alternatively, they may be implemented by program code executable by a computing device such that they may be stored in a storage device by a computing device and, in some cases, may be executed in a different order than herein. The steps shown or described are either made separately into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.
以上所述仅为本发明的优选实施例而已,并不用于限制本发明,对于本领域的技术人员来说,本发明可以有各种更改和变化。凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.
工业实用性Industrial applicability
如上所述,本发明实施例提供的一种资源缓存方法及装置具有以下有益效果:解决了相关技术中存在的缓存子系统被限制访问以及网络拥塞的问题,进而达到了避免缓存子系统被访问限制以及降低网络拥塞的效果。 As described above, the resource cache method and apparatus provided by the embodiments of the present invention have the following beneficial effects: the problem that the cache subsystem is restricted from accessing and network congestion exists in the related art, thereby avoiding the cache subsystem being accessed. Limit and reduce the effects of network congestion.

Claims (12)

  1. 一种资源缓存方法,包括:A resource caching method includes:
    接收第一请求消息,其中,所述第一请求消息用于请求访问第一预定资源;Receiving a first request message, where the first request message is used to request access to a first predetermined resource;
    根据所述第一请求消息统计访问所述第一预定资源的访问次数;And counting, according to the first request message, a number of accesses to access the first predetermined resource;
    判断所述访问次数是否达到第一预定阈值;Determining whether the number of accesses reaches a first predetermined threshold;
    在判断结果为是的情况下时,通知缓存子系统缓存所述第一预定资源,其中,所述缓存子系统用于提供缓存的资源。When the determination result is yes, the notification cache subsystem caches the first predetermined resource, wherein the cache subsystem is used to provide the cached resource.
  2. 根据权利要求1所述的方法,其中,在通知所述缓存子系统缓存所述第一预定资源之后,还包括:The method according to claim 1, wherein after notifying the cache subsystem to cache the first predetermined resource, the method further comprises:
    判断所述缓存子系统是否成功缓存所述第一预定资源;Determining whether the cache subsystem successfully caches the first predetermined resource;
    在判断结果为是的情况下,利用所述缓存子系统提供所述第一预定资源。In the case where the determination result is yes, the first predetermined resource is provided by the cache subsystem.
  3. 根据权利要求2所述的方法,其中,在判断所述缓存子系统是否成功缓存所述第一预定资源之后,还包括:The method of claim 2, after determining whether the cache subsystem successfully caches the first predetermined resource, further comprising:
    在判断结果为是的情况下,将缓存的所述第一预定资源记录在缓存内容分布表中,其中,所述缓存内容分布表中记录有所述缓存子系统已经缓存的资源,所述缓存内容分布表用于将用于访问所述缓存内容分布表中记录的资源的请求消息重定向到所述缓存子系统,利用所述缓存子系统提供所述缓存内容分布表中记录的资源。If the result of the determination is yes, the cached first predetermined resource is recorded in the cache content distribution table, wherein the cache content distribution table records the resource that the cache subsystem has cached, and the cache The content distribution table is configured to redirect a request message for accessing the resource recorded in the cache content distribution table to the cache subsystem, and use the cache subsystem to provide the resource recorded in the cache content distribution table.
  4. 根据权利要求1所述的方法,其中,还包括:The method of claim 1 further comprising:
    接收第二请求消息,其中,所述第二请求消息用于请求访问第二预定资源;Receiving a second request message, where the second request message is used to request access to a second predetermined resource;
    判断所述缓存子系统中是否缓存有所述第二预定资源;Determining whether the second predetermined resource is cached in the cache subsystem;
    在判断结果为是的情况下,将所述第二请求消息重定向到所述缓存子系统,其中,所述第二请求消息用于指示所述缓存子系统提供所述第二预定资源。If the determination result is yes, the second request message is redirected to the cache subsystem, wherein the second request message is used to instruct the cache subsystem to provide the second predetermined resource.
  5. 根据权利要求1所述的方法,其中,判断所述访问次数是否达到第一预定阈值包括:The method of claim 1, wherein determining whether the number of accesses reaches a first predetermined threshold comprises:
    获取热度统计表,其中,所述热度统计表用于记录访问资源的时间和次数;Obtaining a heat statistics table, wherein the heat statistics table is used to record the time and number of times the resource is accessed;
    根据所述热度统计表判断所述访问次数是否达到所述第一预定阈值。Determining, according to the heat statistics table, whether the number of accesses reaches the first predetermined threshold.
  6. 根据权利要求5所述的方法,其中,还包括:The method of claim 5, further comprising:
    根据所述热度统计表确定超期资源,其中,所述超期资源为当前时间与访问所述超期资源的最近的时间的差值超过第二预定阈值的资源;Determining the expired resource according to the heat statistics table, wherein the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold;
    删除所述热度统计表中记录的访问所述超期资源的时间和次数。 The time and number of times of accessing the expired resource recorded in the heat statistics table are deleted.
  7. 一种资源缓存装置,包括:A resource cache device includes:
    第一接收模块,设置为接收第一请求消息,其中,所述第一请求消息用于请求访问第一预定资源;The first receiving module is configured to receive the first request message, where the first request message is used to request access to the first predetermined resource;
    统计模块,设置为根据所述第一请求消息统计访问所述第一预定资源的访问次数;a statistics module, configured to count, according to the first request message, a number of accesses to access the first predetermined resource;
    第一判断模块,设置为判断所述访问次数是否达到第一预定阈值;The first determining module is configured to determine whether the number of accesses reaches a first predetermined threshold;
    通知模块,设置为在所述第一判断模块的判断结果为是的情况下时,通知缓存子系统缓存所述第一预定资源,其中,所述缓存子系统用于提供缓存的资源。The notification module is configured to notify the cache subsystem to cache the first predetermined resource when the determination result of the first determining module is YES, wherein the cache subsystem is configured to provide the cached resource.
  8. 根据权利要求7所述的装置,其中,还包括:The apparatus according to claim 7, further comprising:
    第二判断模块,设置为判断所述缓存子系统是否成功缓存所述第一预定资源;a second determining module, configured to determine whether the cache subsystem successfully caches the first predetermined resource;
    提供模块,设置为在所述第二判断模块的判断结果为是的情况下,利用所述缓存子系统提供所述第一预定资源。And providing a module, configured to provide the first predetermined resource by using the cache subsystem if the determination result of the second determining module is yes.
  9. 根据权利要求8所述的装置,其中,还包括:The apparatus of claim 8 further comprising:
    记录模块,设置为在所述第二判断模块的判断结果为是的情况下,将缓存的所述第一预定资源记录在缓存内容分布表中,其中,所述缓存内容分布表中记录有所述缓存子系统已经缓存的资源,所述缓存内容分布表用于将用于访问所述缓存内容分布表中记录的资源的请求消息重定向到所述缓存子系统,利用所述缓存子系统提供所述缓存内容分布表中记录的资源。a recording module, configured to record, in a case where the determination result of the second determining module is YES, the cached first predetermined resource in a cache content distribution table, wherein the cache content distribution table records a resource that has been cached by a cache subsystem, the cache content distribution table for redirecting a request message for accessing a resource recorded in the cache content distribution table to the cache subsystem, provided by the cache subsystem The resource recorded in the cache content distribution table.
  10. 根据权利要求7所述的装置,其中,还包括:The apparatus according to claim 7, further comprising:
    第二接收模块,设置为接收第二请求消息,其中,所述第二请求消息用于请求访问第二预定资源;a second receiving module, configured to receive a second request message, where the second request message is used to request access to a second predetermined resource;
    第三判断模块,设置为判断所述缓存子系统中是否缓存有所述第二预定资源;a third determining module, configured to determine whether the second predetermined resource is cached in the cache subsystem;
    重定向模块,设置为在所述第三判断模块的判断结果为是的情况下,将所述第二请求消息重定向到所述缓存子系统,其中,所述第二请求消息用于指示所述缓存子系统提供所述第二预定资源。a redirecting module, configured to redirect the second request message to the cache subsystem if the determination result of the third determining module is yes, wherein the second request message is used to indicate The cache subsystem provides the second predetermined resource.
  11. 根据权利要求7所述的装置,其中,所述第一判断模块包括:The apparatus of claim 7, wherein the first determining module comprises:
    获取单元,设置为获取热度统计表,其中,所述热度统计表用于记录访问资源的时间和次数;An obtaining unit, configured to obtain a heat statistics table, wherein the heat statistics table is used to record the time and the number of times the resource is accessed;
    判断单元,设置为根据所述热度统计表判断所述访问次数是否达到所述第一预定阈值。The determining unit is configured to determine, according to the heat statistics table, whether the number of accesses reaches the first predetermined threshold.
  12. 根据权利要求11所述的装置,其中,还包括: The device according to claim 11, further comprising:
    确定模块,设置为根据所述热度统计表确定超期资源,其中,所述超期资源为当前时间与访问所述超期资源的最近的时间的差值超过第二预定阈值的资源;a determining module, configured to determine an expired resource according to the heat statistics table, wherein the expired resource is a resource whose difference between the current time and the latest time of accessing the expired resource exceeds a second predetermined threshold;
    删除模块,设置为删除所述热度统计表中记录的访问所述超期资源的时间和次数。 And deleting a module, configured to delete the time and the number of times the access to the expired resource recorded in the heat statistics table is accessed.
PCT/CN2016/094627 2015-08-12 2016-08-11 Resource caching method and device WO2017025052A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510493301.7A CN106453461A (en) 2015-08-12 2015-08-12 Resource caching method and resource caching device
CN201510493301.7 2015-08-12

Publications (1)

Publication Number Publication Date
WO2017025052A1 true WO2017025052A1 (en) 2017-02-16

Family

ID=57983054

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/094627 WO2017025052A1 (en) 2015-08-12 2016-08-11 Resource caching method and device

Country Status (2)

Country Link
CN (1) CN106453461A (en)
WO (1) WO2017025052A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109379395A (en) * 2018-08-22 2019-02-22 中国平安人寿保险股份有限公司 A kind of interface data buffer setting method and terminal device
CN111104528A (en) * 2018-10-29 2020-05-05 浙江宇视科技有限公司 Picture obtaining method and device and client
CN112487326A (en) * 2020-11-27 2021-03-12 杭州安恒信息技术股份有限公司 Data caching method, system, storage medium and equipment
CN113612823A (en) * 2021-07-22 2021-11-05 北京金山云网络技术有限公司 Cache duration adjusting method and device for content distribution network and storage medium
CN114490749A (en) * 2021-12-28 2022-05-13 珠海大横琴科技发展有限公司 Resource access method and device

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107707597A (en) * 2017-04-26 2018-02-16 贵州白山云科技有限公司 One kind burst focus accesses equalization processing method and device
CN109040153B (en) * 2017-06-08 2022-06-28 中兴通讯股份有限公司 Caching method for caching content, related device and computer-readable storage medium
CN108712498B (en) * 2018-05-23 2020-02-18 新华三大数据技术有限公司 Service calling method and device
CN109246234B (en) * 2018-09-30 2021-09-24 北京金山云网络技术有限公司 Image file downloading method and device, electronic equipment and storage medium
CN114422522B (en) * 2020-10-13 2024-02-13 贵州白山云科技股份有限公司 Cache distribution method, device, medium and equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668046A (en) * 2009-10-13 2010-03-10 成都市华为赛门铁克科技有限公司 Resource caching method, resource obtaining method, device and system thereof
CN102841931A (en) * 2012-08-03 2012-12-26 中兴通讯股份有限公司 Storage method and storage device of distributive-type file system
CN103581051A (en) * 2012-08-02 2014-02-12 深圳市腾讯计算机系统有限公司 File buffering storage method and device and system
US20140052814A1 (en) * 2012-08-14 2014-02-20 Calix, Inc. Distributed cache system for optical networks
CN104778271A (en) * 2015-04-24 2015-07-15 无锡天脉聚源传媒科技有限公司 Video data caching method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668046A (en) * 2009-10-13 2010-03-10 成都市华为赛门铁克科技有限公司 Resource caching method, resource obtaining method, device and system thereof
CN103581051A (en) * 2012-08-02 2014-02-12 深圳市腾讯计算机系统有限公司 File buffering storage method and device and system
CN102841931A (en) * 2012-08-03 2012-12-26 中兴通讯股份有限公司 Storage method and storage device of distributive-type file system
US20140052814A1 (en) * 2012-08-14 2014-02-20 Calix, Inc. Distributed cache system for optical networks
CN104778271A (en) * 2015-04-24 2015-07-15 无锡天脉聚源传媒科技有限公司 Video data caching method and device

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109379395A (en) * 2018-08-22 2019-02-22 中国平安人寿保险股份有限公司 A kind of interface data buffer setting method and terminal device
CN109379395B (en) * 2018-08-22 2022-11-11 中国平安人寿保险股份有限公司 Interface data cache setting method and terminal equipment
CN111104528A (en) * 2018-10-29 2020-05-05 浙江宇视科技有限公司 Picture obtaining method and device and client
CN111104528B (en) * 2018-10-29 2023-05-16 浙江宇视科技有限公司 Picture acquisition method and device and client
CN112487326A (en) * 2020-11-27 2021-03-12 杭州安恒信息技术股份有限公司 Data caching method, system, storage medium and equipment
CN112487326B (en) * 2020-11-27 2024-03-19 杭州安恒信息技术股份有限公司 Data caching method, system, storage medium and equipment
CN113612823A (en) * 2021-07-22 2021-11-05 北京金山云网络技术有限公司 Cache duration adjusting method and device for content distribution network and storage medium
CN114490749A (en) * 2021-12-28 2022-05-13 珠海大横琴科技发展有限公司 Resource access method and device

Also Published As

Publication number Publication date
CN106453461A (en) 2017-02-22

Similar Documents

Publication Publication Date Title
WO2017025052A1 (en) Resource caching method and device
US11194719B2 (en) Cache optimization
CN107025234B (en) Information pushing method and cache server
US9043428B2 (en) Edge caching using HTTP headers
US9191458B2 (en) Request routing using a popularity identifier at a DNS nameserver
US8990357B2 (en) Method and apparatus for reducing loading time of web pages
US9451046B2 (en) Managing CDN registration by a storage provider
US10601767B2 (en) DNS query processing based on application information
US20070226229A1 (en) Method and system for class-based management of dynamic content in a networked environment
US20050117558A1 (en) Method for reducing data transport volume in data networks
EP3175364B1 (en) Web redirection for caching
WO2017080459A1 (en) Method, device and system for caching and providing service contents and storage medium
US20130346552A1 (en) Download method, system, and device for mobile terminal
KR20160143666A (en) Dynamic cache allocation and network management
US11741174B2 (en) Direct leg access for proxy web scraping
WO2018228234A1 (en) Video caching method and video caching device
EP3389240B1 (en) Method and system for processing cache cluster service
KR20130087810A (en) Method and apparatus for cooperative caching in mobile communication system
US20180302489A1 (en) Architecture for proactively providing bundled content items to client devices
WO2013113150A1 (en) Cache optimization method, cache and cache optimization system
US20230018983A1 (en) Traffic counting for proxy web scraping
JP2004297494A (en) Name resolution method and its apparatus in a plurality of network connection
KR20160111648A (en) Method for caching of contents and cache apparatus therefor
US9288153B2 (en) Processing encoded content
US20200402115A1 (en) Serving an Online Advertisement Asynchronously

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16834676

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16834676

Country of ref document: EP

Kind code of ref document: A1