WO2024124663A1 - 一种支持cdn缓存批量刷新的方法及装置 - Google Patents

一种支持cdn缓存批量刷新的方法及装置 Download PDF

Info

Publication number
WO2024124663A1
WO2024124663A1 PCT/CN2023/073996 CN2023073996W WO2024124663A1 WO 2024124663 A1 WO2024124663 A1 WO 2024124663A1 CN 2023073996 W CN2023073996 W CN 2023073996W WO 2024124663 A1 WO2024124663 A1 WO 2024124663A1
Authority
WO
WIPO (PCT)
Prior art keywords
cache
refresh
batch
rule
access request
Prior art date
Application number
PCT/CN2023/073996
Other languages
English (en)
French (fr)
Inventor
孙涛
Original Assignee
江苏云工场信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 江苏云工场信息技术有限公司 filed Critical 江苏云工场信息技术有限公司
Publication of WO2024124663A1 publication Critical patent/WO2024124663A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682—Policies or rules for updating, deleting or replacing the stored data

Definitions

  • the present application relates to the field of CDN node cache management, and in particular to a method for supporting CDN cache batch refresh.
  • the present application also relates to a device for supporting CDN cache batch refresh.
  • CDN edge node refers to a business platform built on the edge of the network close to users, which provides storage, computing, network and other resources, and sinks some key business applications to the edge of the access network to reduce the width and delay losses caused by network transmission and multi-level forwarding.
  • edge computing nodes are in the early stages of layout and development. Edge nodes are located between users and cloud centers, and are closer to users (data sources) than traditional cloud center edge nodes. Compared with cloud data centers, edge nodes are small, distributed, and closer to users. Massive amounts of data no longer need to be uploaded to the cloud for processing, and data can be processed at the edge of the network, reducing request response time and network bandwidth while ensuring data security and privacy.
  • CDN edge node cache is a resource that users directly request and obtain.
  • the general operation method is to delete the source site content first, and then clean up the CDN edge node content.
  • CDN nodes are spread all over the country. If you clean up the cache of a node alone, it will take a lot of time to process. If the time is too long, the impact or spread will be larger and wider.
  • Abnormal resources may be a single URL or multiple URLs, or a single directory or multiple directories.
  • the commonality of the resources may also be the same. For example, if all abnormal resources are resources with the JPG suffix, you can refresh /*.jpg to quickly update resources when users are using CDN.
  • Chinese patent 201811603640.6 proposes a method and device for refreshing data objects.
  • the method includes: generating a task record corresponding to the refresh task based on the refresh object information contained in the received refresh task; obtaining the object identifier contained in the received data object request; when the object identifier matches the refresh object information contained in the generated task record, performing refresh processing on the data object corresponding to the object identifier. It can be seen that the refresh object information in the refresh task is recorded in the corresponding task record, and when a data object request is received later, the refresh object information is recorded based on the object identifier contained in the data object request and the refresh object information contained in the generated task record.
  • Whether the refresh object information contained in the generated task record matches is used to determine whether to perform refresh processing on the data object corresponding to the object identifier, thereby achieving an on-demand refresh effect and effectively avoiding the problem of increased system overhead caused by processing a large number of refresh requests at the same time.
  • the CDN edge node cache cannot be refreshed quickly;
  • the purpose of the present invention is to overcome the defects in the prior art and provide a method for supporting CDN cache batch refresh.
  • the present application also relates to a device for supporting CDN cache batch refresh.
  • the present application provides a method for supporting CDN cache batch refresh, including:
  • the rule effective time in the preset header and the cache time are determined.
  • the rule effective time is less than the local cache time, the cache content is read, otherwise the hit state is changed to a miss.
  • the cache batch refresh rule sets an effective time limit.
  • the refreshing of the data corresponding to the current cache requested from the source station is set to refresh multiple files or directories at a time.
  • the single resource corresponding to each CDN node, or the resource with the suffix name, or the resource belonging to the refresh directory is cleared uniformly.
  • the present application also provides a device supporting CDN cache batch refresh, including:
  • Cache module used to set and load cache batch refresh rules
  • a matching module configured to receive an access request, match the cache batch refresh rule according to a uniform resource identifier in the access request, and add a preset header to the access request matched to the cache batch refresh rule;
  • a search module used to send the access request or the access request with a preset header to the cache ATS component, search based on the uniform resource identifier, and request the corresponding data of the current cache to be refreshed from the source station when no hit is found;
  • the judgment module is used to judge the size of the rule effective time and cache time in the preset header if the search hits. When the rule effective time is less than the local cache time, the cache content is read, otherwise the hit status is changed to a miss.
  • the cache batch refresh rule sets an effective time limit.
  • the refreshing of the data corresponding to the current cache requested from the source station is set to refresh multiple files or directories at a time.
  • the single resource corresponding to each CDN node, or the resource with the suffix name, or the resource belonging to the refresh directory is cleared uniformly.
  • the present application provides a method for supporting CDN cache batch refresh, including: setting and loading cache batch refresh rules; receiving access requests, matching the cache batch refresh rules according to the uniform resource identifier in the access requests, and adding a preset header to the access requests that match the cache batch refresh rules; sending the access request or the access request with the preset header to the cache ATS component, searching based on the uniform resource identifier, and requesting the data corresponding to the current cache from the source station for refresh when no hit is found; if the search is successful, determining the size of the rule effective time and the cache time in the preset header, and when the rule effective time is less than the local cache time, reading the cache content, otherwise The hit status is changed to miss.
  • This application uses the cache component to determine whether the CDN edge node cache is expired, and determines the Last-Modified of the CDN node. If the Last-Modified on the CDN node is consistent with that of the source station, no refresh is performed. If not, the latest content will be pulled from the source station, meeting the customer's daily CDN refresh needs, whether it is a single URL refresh, or multiple URL refreshes, or a single directory refresh, or multiple directories refresh.
  • FIG1 is a schematic diagram of the CDN cache batch refresh process supported in this application.
  • FIG2 is a schematic diagram of the edge node network structure in this application.
  • FIG. 3 is a schematic diagram of the routing group structure in this application.
  • FIG. 4 is a schematic diagram of a device supporting CDN cache batch refresh in the present application.
  • the present application provides a method for supporting CDN cache batch refresh, including:
  • the rule effective time in the preset header and the cache time are determined.
  • the rule effective time is less than the local cache time, the cache content is read, otherwise the hit state is changed to a miss.
  • This application uses a cache component to determine whether the CDN edge node cache is expired. By determining the Last-Modified of the CDN node, if the Last-Modified on the CDN node is consistent with that of the source site, no refresh is performed. If they are inconsistent, the latest content will be pulled from the source site, meeting the customer's daily CDN refresh needs, whether it is a single URL refresh or multiple URL refreshes, whether it is a single directory refresh or multiple directory refreshes.
  • FIG1 is a schematic diagram of the CDN cache batch refresh process supported in this application.
  • S101 sets and loads cache batch refresh rules.
  • the cache batch refresh rule can be set as: setting a specific character or a specific string in the same resource identifier (URI) in the request information as a rule character or rule string that triggers refresh.
  • URI resource identifier
  • the cache batch refresh rule is that when /a/*, /*.jpg appears, the cache is refreshed.
  • the cache batch refresh rules are cached in an edge node (CDN), which is connected to a cloud server and a client, respectively, and can pull information from the server based on a client request and forward it to the client.
  • CDN edge node
  • the edge node receives the client request information, searches for the data corresponding to the request information from the local storage at the edge node based on the request information, and sends it to the client. Specifically, when receiving the request information, data processing is performed based on the format of the client request data corresponding to the request information, and data processing is performed on the original data based on the client request data, and the processed data result is sent to the client.
  • the request information is received, if the corresponding data is not found in the local storage, the corresponding data is pulled from the cloud server, and the data is processed and cached or stored. When the user requests the data again, the processed data can be sent directly.
  • An intermediate CDN is also set between the edge node and the cloud server.
  • the intermediate CDN belongs to the edge node relative to the cloud server and to the cloud server relative to the edge node. That is, when the edge node requests data from the cloud server, the edge node first makes a data request from the intermediate CDN.
  • Each edge node can be connected to multiple intermediate CDNs. When the edge node requests data from the intermediate CDN, it sends data request information to multiple intermediate CDNs at the same time.
  • Each CDN replies to the edge node whether it has found the data corresponding to the request based on its own local search. When the messages replied by the preset multiple intermediate CDNs are all unsuccessful, the designated intermediate CDN will make a request to the cloud server, as shown in Figure 2.
  • the intermediate CDN is connected to the edge node via a gateway, and the gateway performs data routing and load balancing for the edge node and the intermediate CDN.
  • the grid is composed of a plurality of routers forming a routing group. Within the routing group, each router can call other routers to process data, realize the load balancing, and speed up the data transmission.
  • the present application is provided with a plurality of interconnected routing groups. When a group of routing groups is overloaded, other routing groups with smaller loads can be called to participate in data routing processing, as shown in FIG3 .
  • the cloud server is connected to the intermediate CDN via an edge router, and the edge router manages data input and output.
  • the edge router receives data and sends the data to the virtual machine corresponding to the client, and can also receive data sent by the cloud server to the client and forward it.
  • the cache batch refresh rules are also loaded into the intermediate CDN node at the same time, and the cache of the edge node and the intermediate CDN are refreshed based on the cache batch refresh rules respectively.
  • S102 receives an access request, matches the cache batch refresh rule according to a uniform resource identifier in the access request, and adds a preset header to the access request that matches the cache batch refresh rule.
  • the client When the client requests data from the cloud server, it first sends an access request. After the access request is transmitted to the edge node, the edge node parses and obtains the content of the access request, queries the corresponding data information based on the access request, and returns the data information to the client.
  • the uniform resource identifier (Uniform Resource Identifier, or URI) is a string used to identify the name of an Internet resource. This identifier allows users to interact with any (including local and Internet) resources through a specific protocol.
  • URI is defined by a scheme that includes a certain syntax and related protocols.
  • URI Universal Resource Identifier
  • the cache batch refresh rule is matched based on the uniform resource identifier.
  • the cache batch refresh rule is a specific character or a specific field that is pre-set and cached in the edge node, and the character or field in the uniform resource identifier is matched with the specific character or field in the cache batch refresh rule to determine whether the match is successful.
  • ATS is a component that can be used as both an HTTP proxy and an HTTP cache. ATS can cache any byte stream and supports byte streams transmitted by the HTTP protocol. When such a stream is cached, it will carry an HTTP protocol header and will be named as an object in the cache. Each object is identified by a globally unique value called a cache key.
  • the edge node determines whether the current uniform resource identifier (URI) matches the cache batch refresh rule. If it does not match, it is directly forwarded to the cache ATS component; if it matches, it is forwarded to the cache ATS component with a preset header of X-Ats-Cache-Ims: ⁇ timestamp>, where timestamp indicates the effective time of the rule.
  • URI uniform resource identifier
  • the ATS component caches data from the intermediate CDN.
  • the intermediate CDN can repeat the matching of the cache batch refresh rule based on the access request of the client to determine that the intermediate CDN needs to refresh the data.
  • the matching of the cache batch refresh rule by the intermediate CDN is the same as the step at the edge node, which will not be repeated here.
  • whether the edge node is connected to the cloud server through the intermediate CDN can be set according to actual conditions, and the number of intermediate CDN nodes connected to the edge node can also be set according to actual conditions.
  • the edge node is closer to the client, so it can send data to the client more quickly.
  • multiple levels of intermediate CDNs can be set between the edge node and the server to achieve multi-level caching of data.
  • each level of the intermediate CDN is set with the cache batch refresh rules of the edge node and determines whether to refresh the data.
  • S103 sends the access request or the access request with a preset header to the cache ATS component, searches based on the uniform resource identifier, and requests the corresponding data in the current cache to be refreshed from the source station when no hit is found.
  • the cache ATS component After obtaining the uniform resource identifier, the cache ATS component performs resource search based on the uniform resource identifier. When the URI requested by the cache ATS component searches the local cache and no hit is found, it returns to the source site to obtain the file content.
  • the edge node performs resource search locally, including searching for the same resource in different formats, that is, when a resource is searched, the resource may have been stored in the edge node, but its format is different from the requested format.
  • the edge node searches for the picture according to the URI. If the picture in the format requested by the client is not found but the source picture is found, the source picture can be processed in the edge node to make it meet the requirements of the client request. The processed picture is then stored.
  • the edge node may further process the processed image to conform to the format requested by the client. In this case, it is necessary to judge the image to determine whether the processed image meets the clarity requirement of the client display.
  • the TS is the pixel of the image to be processed
  • TC is the size of the image to be processed
  • KS is the pixel of the required image
  • the KC is the required image size.
  • F may also be ⁇ 0.8.
  • the is the image pixel requested by the client, A is the number of times the pixel image is requested, is the size of the image requested by the client, and B is the number of times the image of this size is requested.
  • the source station is the cloud server.
  • the source station, cloud server or data center refers to the original source of the client's requested data, that is, the final server connected to the client.
  • the edge node repackages the client's access request and sends it to the intermediate CDN node of the next level.
  • the intermediate CDN server then performs the cache batch refresh rule again.
  • the source station is matched, and a local search is performed according to the matching result, or an access request is sent to a higher level, and the process is repeated until the source station is accessed.
  • the rule effective time and cache time in the preset header are determined.
  • the rule effective time is less than the local cache time, the cache content is read, otherwise the hit status is changed to a miss.
  • the cache is hit, it is checked whether the access request carries an X-Ats-Cache-Ims header, and if not, the content is directly returned to the edge node.
  • the source station is the cloud server.
  • the source station, cloud server or data center refers to the original source of the client's requested data, that is, the final server connected to the client.
  • the edge node repackages the client's access request and sends it to the intermediate CDN node of the next level.
  • the intermediate CDN server matches the cache batch refresh rule again and performs a local search or sends an access request to the next level based on the matching result, and repeats this process until the source station is accessed.
  • the cache batch refresh rules are automatically invalidated and expired after exceeding the maximum cache lifetime (eg, 7 days), so that the size of the cache batch refresh rule list does not continue to grow.
  • the present invention does not need to clear the contents of the cache component in advance during batch refresh. It only needs to determine whether to return to the source station to pull new files when the files are accessed next time, thereby reducing the pressure on the cache component caused by batch cache refresh.
  • the present application also provides a device supporting CDN cache batch refresh, including: a cache module 401, a matching module 402, a search module 403, and a judgment module 404.
  • FIG. 4 is a schematic diagram of a device supporting CDN cache batch refresh in the present application.
  • the cache module 401 is used to set and load cache batch refresh rules.
  • the cache batch refresh rule can be set as: setting a specific character or a specific string in the same resource identifier (URI) in the request information as a rule character or rule string that triggers refresh.
  • URI resource identifier
  • the cache batch refresh rule is that when /a/*, /*.jpg appears, the cache is refreshed.
  • the cache batch refresh rules are cached in an edge node (CDN), which is connected to a cloud server and a client, respectively, and can pull information from the server based on a client request and forward it to the client.
  • CDN edge node
  • the edge node receives the client request information, searches for the data corresponding to the request information from the local storage at the edge node based on the request information, and sends it to the client. Specifically, when receiving the request information, data processing is performed based on the format of the client request data corresponding to the request information, and data processing is performed on the original data based on the client request data, and the processed data is sent to the client.
  • the request information is received, if the corresponding data is not found in the local storage, the corresponding data is pulled from the cloud server, and the data is processed and cached or stored. When the user requests the data again, the processed data can be sent directly.
  • An intermediate CDN is also set between the edge node and the cloud server.
  • the intermediate CDN belongs to the edge node relative to the cloud server and to the cloud server relative to the edge node. That is, when the edge node requests data from the cloud server, the edge node first makes a data request from the intermediate CDN.
  • Each edge node can be connected to multiple intermediate CDNs. When the edge node requests data from the intermediate CDN, it sends data request information to multiple intermediate CDNs at the same time.
  • Each CDN replies to the edge node whether it has found the data corresponding to the request based on its own local search. When the messages replied by the preset multiple intermediate CDNs are all unsuccessful, the designated intermediate CDN will make a request to the cloud server.
  • the intermediate CDN is connected to the edge node via a gateway, and the gateway performs data routing and load balancing for the edge node and the intermediate CDN.
  • the grid is composed of a plurality of routers forming a routing group. Within the routing group, each router can call other routers to process data, realize the load balancing, and speed up the data transmission.
  • the present application is provided with a plurality of interconnected routing groups. When a group of the routing groups is overloaded, other routing groups with smaller loads can be called to participate in data routing processing.
  • the cloud server is connected to the intermediate CDN through an edge router, and the edge router manages data input and output.
  • the edge router receives data and sends the data to the
  • the virtual machine corresponding to the client can also receive and forward the data sent by the cloud server to the client.
  • the cache batch refresh rules are also loaded into the intermediate CDN node at the same time, and the cache of the edge node and the intermediate CDN are refreshed based on the cache batch refresh rules respectively.
  • the matching module 402 is used to receive an access request, match the cache batch refresh rule according to a uniform resource identifier in the access request, and add a preset header to the access request matched to the cache batch refresh rule.
  • the client When the client requests data from the cloud server, it first sends an access request. After the access request is transmitted to the edge node, the edge node parses and obtains the content of the access request, queries the corresponding data information based on the access request, and returns the data information to the client.
  • the uniform resource identifier (Uniform Resource Identifier, or URI) is a string used to identify the name of an Internet resource. This identifier allows users to interact with any (including local and Internet) resources through a specific protocol.
  • URI is defined by a scheme that includes a certain syntax and related protocols.
  • URI Universal Resource Identifier
  • the cache batch refresh rule is matched based on the uniform resource identifier.
  • the cache batch refresh rule is a specific character or a specific field that is pre-set and cached in the edge node, and the character or field in the uniform resource identifier is matched with the specific character or field in the cache batch refresh rule to determine whether the match is successful.
  • ATS is a component that can be used as both an HTTP proxy and an HTTP cache. ATS can cache any byte stream and supports byte streams transmitted by the HTTP protocol. When such a stream is cached, it will carry an HTTP protocol header and will be named as an object in the cache. Each object is identified by a globally unique value called a cache key.
  • the edge node determines whether the current uniform resource identifier (URI) matches the cache batch refresh rule. If it does not match, it is directly forwarded to the cache ATS component; if it matches The rule is forwarded to the cache ATS component with a preset header of X-Ats-Cache-Ims: ⁇ timestamp>, where timestamp indicates the effective time of the rule.
  • URI uniform resource identifier
  • the ATS component caches data from the intermediate CDN.
  • the intermediate CDN can repeat the matching of the cache batch refresh rule based on the access request of the client to determine that the intermediate CDN needs to refresh the data.
  • the matching of the cache batch refresh rule by the intermediate CDN is the same as the step at the edge node, which will not be repeated here.
  • whether the edge node is connected to the cloud server through the intermediate CDN can be set according to actual conditions, and the number of intermediate CDN nodes connected to the edge node can also be set according to actual conditions.
  • the edge node is closer to the client, so it can send data to the client more quickly.
  • multiple levels of intermediate CDNs can be set between the edge node and the server to achieve multi-level caching of data.
  • each level of the intermediate CDN is set with the cache batch refresh rules of the edge node and determines whether to refresh the data.
  • the search module 403 is used to send the access request or the access request with a preset header to the cache ATS component, perform a search based on the uniform resource identifier, and request the corresponding data in the current cache to be refreshed from the source station when a hit is not found.
  • the cache ATS component After obtaining the uniform resource identifier, the cache ATS component performs resource search based on the uniform resource identifier. When the URI requested by the cache ATS component searches the local cache and no hit is found, it returns to the source site to obtain the file content.
  • the edge node performs resource search locally, including searching for the same resource in different formats, that is, when a resource is searched, the resource may have been stored in the edge node, but its format is different from the requested format.
  • the edge node searches for the picture according to the URI. If the picture in the format requested by the client is not found but the source picture is found, the source picture can be processed in the edge node to make it meet the requirements of the client request. The processed picture is then stored.
  • the edge node may further process the processed image to conform to the format requested by the client. In this case, it is necessary to judge the image to determine whether the processed image meets the clarity requirement of the client display.
  • the TS is the pixel of the image to be processed
  • TC is the size of the image to be processed
  • KS is the pixel of the required image
  • the KC is the required image size.
  • F may also be ⁇ 0.8.
  • the is the image pixel requested by the client, A is the number of times the pixel image is requested, is the size of the image requested by the client, and B is the number of times the image of this size is requested.
  • the source station is the cloud server.
  • the source station, cloud server or data center refers to the original source of the client's requested data, that is, the final server connected to the client.
  • the edge node repackages the client's access request and sends it to the intermediate CDN node of the next level.
  • the intermediate CDN server matches the cache batch refresh rule again and performs a local search or sends an access request to the next level based on the matching result, and repeats this process until the source station is accessed.
  • the judgment module 404 is used to judge the size of the rule effective time and cache time in the preset header if the search is hit. When the rule effective time is less than the local cache time, the cache content is read, otherwise the hit status is changed to miss.
  • the cache is hit, it is checked whether the access request carries an X-Ats-Cache-Ims header, and if not, the content is directly returned to the edge node.
  • the source station is the cloud server.
  • the source station, cloud server or data center refers to the original source of the client's requested data, that is, the final server connected to the client.
  • the edge node repackages the client's access request and sends it to the intermediate CDN node of the next level.
  • the intermediate CDN server matches the cache batch refresh rule again and performs a local search or sends an access request to the next level based on the matching result, and repeats this process until the source station is accessed.
  • the cache batch refresh rules are automatically invalidated and expired after exceeding the maximum cache lifetime (eg, 7 days), so that the size of the cache batch refresh rule list does not continue to grow.
  • the present invention does not need to clear the contents of the cache component in advance during batch refresh. It only needs to determine whether to return to the source station to pull new files when the files are accessed next time, thereby reducing the pressure on the cache component caused by batch cache refresh.
  • the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes.
  • the scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
  • These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供的一种支持CDN缓存批量刷新的方法及装置，其方法包括：设置并加载缓存批量刷新规则；根据接收的访问请求中统一资源标识符匹配缓存批量刷新规则，将匹配到缓存批量刷新规则的访问请求中添加预设字头；将访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于统一资源标识符进行查找，未命中时从源站请求缓存对应的数据进行刷新；若查找命中，则判断预设字头中的规则生效时间与缓存时间的大小，当规则生效时间小于本地的缓存时间，则读取缓存内容，否则命中状态改为未命中。本申请可判断CDN边缘节点缓存是否过期，进行边缘节点刷新，满足客户CDN刷新需求。

Description

一种支持CDN缓存批量刷新的方法及装置 技术领域
本申请涉及CDN节点缓存管理领域，尤其涉及一种支持CDN缓存批量刷新的方法。本申请还涉及一种支持CDN缓存批量刷新的装置。
背景技术
CDN边缘节点指在靠近用户的网络边缘侧构建的业务平台，提供存储、计算、网络等资源，将部分关键业务应用下沉到接入网络边缘，以减少网络传输和多级转发带来的宽度和时延损耗。
目前，边缘计算节点整体处于布局和发展初期。边缘节点位置介于用户和云中心之间，相比较传统的云中心边缘节点更接近用户(数据源)。边缘节点相比较云数据中心，具有小型化、分布式和更贴近用户的特性，海量的数据无需再上传至云端进行处理，实现在网络边缘侧对数据的处理，减少请求响应时间、减少网络带宽同时保证数据的安全性和私密性。
CDN边缘节点缓存作为用户直接请求获取的资源，当源站资源更新或者CDN缓存资源异常，例如：缓存资源存在违规，图片涉黄或者涉政，资源包存在诱导消费或者资源存在违规广告等情况，作为服务平台，一般的操作方式是先删除源站内容，然后清理CDN边缘节点内容。而CDN节点遍布全国，如果单独针对节点去清理缓存，则需要花费大量的时间进行处理。时间过长则影响或者波及的范围更大、更广。
异常资源可能是单个URL，或者多个URL，也可能是单个目录或者多个目录，资源的共性也可能是同一个，例如：全部异常的资源都是JPG后缀的资源，则可以通过刷新/*.jpg，为了解决用户在CDN使用过程中更新可快速更新资源。
中国专利本201811603640.6提出一种数据对象的刷新方法及装置。其中的方法包括:根据接收到的刷新任务中包含的刷新对象信息,生成与刷新任务相对应的任务记录；获取接收到的数据对象请求中包含的对象标识；当对象标识与已生成的任务记录中包含的刷新对象信息匹配时,针对与对象标识相对应的数据对象执行刷新处理。由此可见,将刷新任务中的刷新对象信息记录在对应的任务记录中,当后续接收到数据对象请求时,根据数据对象请求中包含的对象标识与 已生成的任务记录中包含的刷新对象信息是否匹配来确定是否针对与该对象标识相对应的数据对象执行刷新处理,从而实现了按需刷新的效果,有效避免了因同时处理大量刷新请求而导致的系统开销增大的问题。
但是，现有技术中还存在以下亟待解决的问题：
1、当用户单个资源需要更新时，例如：资源更新、违规图片，无法快速刷新CDN边缘节点缓存；
2、当用户有大量资源需要刷新时，如存在一个目录、或者多个目录时，难以快速通过目录级别维度进行刷新。
发明内容
本发明的目的在于克服现有技术中存在的缺陷，提供一种支持CDN缓存批量刷新的方法。本申请还涉及一种支持CDN缓存批量刷新的装置。
本申请提供的一种支持CDN缓存批量刷新的方法，包括：
设置并加载缓存批量刷新规则；
接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；
将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；
若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
可选的，所述缓存批量刷新规则设置有效时限。
可选的，还包括：
在文件被访问到时判断是否回源站拉取新文件，不预先清除缓存组件中的内容。
可选的，所述从源站请求当前缓存对应的数据进行刷新设置为一次刷新多个文件或者目录。
可选的，执行刷新后，各个CDN节点对应单个资源，或符合后缀名的资源，或属于刷新目录下的资源，统一被清除。
本申请还提供一种支持CDN缓存批量刷新的装置，包括：
缓存模块，用于设置并加载缓存批量刷新规则；
匹配模块，用于接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；
查找模块，用于将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；
判断模块，用于若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
可选的，所述缓存批量刷新规则设置有效时限。
可选的，还包括：
在文件被访问到时判断是否回源站拉取新文件，不预先清除缓存组件中的内容。
可选的，所述从源站请求当前缓存对应的数据进行刷新设置为一次刷新多个文件或者目录。
可选的，执行刷新后，各个CDN节点对应单个资源，或符合后缀名的资源，或属于刷新目录下的资源，统一被清除。
本申请的优点和有益效果：
本申请提供的一种支持CDN缓存批量刷新的方法，包括：设置并加载缓存批量刷新规则；接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否 则命中状态改为未命中。本申请通过缓存组件判断CDN边缘节点缓存是否过期，通过判断CDN节点的Last-Modified，如果CDN节点上的Last-Modified和源站的一致，则不进行刷新，如果不一致，则会从源站拉取最新内容，满足了客户日常的CDN刷新需求，无论单个URL刷新，还是多个ULR刷新，无论单个目录刷新，还是多目录刷新。
附图说明
图1是本申请中支持CDN缓存批量刷新流程示意图。
图2是本申请中边缘节点网络结构示意图。
图3是本申请中路由组结构示意图。
图4是本申请中支持CDN缓存批量刷新装置示意图。
具体实施方式
下面结合附图和具体实施例对本发明作进一步说明，以使本领域的技术人员可以更好地理解本发明并能予以实施。
以下内容均是为了详细说明本申请要保护的技术方案所提供的具体实施过程的示例，但是本申请还可以采用不同于此的描述的其他方式实施，本领域技术人员可以在本申请构思的指引下，采用不同的技术手段实现本申请，因此本申请不受下面具体实施例的限制。
本申请提供的一种支持CDN缓存批量刷新的方法，包括：
设置并加载缓存批量刷新规则；
接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；
将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；
若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
本申请通过缓存组件判断CDN边缘节点缓存是否过期，通过判断CDN节点的Last-Modified，如果CDN节点上的Last-Modified和源站的一致，则不进行刷新，如果不一致，则会从源站拉取最新内容，满足了客户日常的CDN刷新需求，无论单个URL刷新，还是多个ULR刷新，无论单个目录刷新，还是多目录刷新。
图1是本申请中支持CDN缓存批量刷新流程示意图。
请参照图1所述，S101设置并加载缓存批量刷新规则。
所述缓存批量刷新规则可以设置为：将请求信息中同一资源符(uri)中的特定字符、特定字符串设置为触发刷新的规则字符或者规则字符串。例如：所述缓存批量刷新规则是，当/a/*，/*.jpg出现时，则进行缓存刷新。
在本申请中，所述缓存批量刷新规则缓存到边缘节点(CDN)中，该边缘节点分别连接云服务器和客户端，可以基于客户端请求从服务器中拉取信息并转发到客户端。
所述边缘节点接收客户端请求信息，基于所述请求信息从该边缘节点处的本地存储中查找所述请求信息对应的数据发送到客户端。具体的，当接收到所述请求信息时，基于所述请求信息对应的客户端所述请求数据的格式进行数据处理，以及基于所述客户端请求数据，对原始数据进行数据处理，将数据处理后的结果发送到客户端。
当接收到所述请求信息时，本地存储中若未查找到相应数据，则从云服务器中拉取对应的数据，并对所述数据进行处理后进行缓存或者存储，当用户再次请求该数据时，可以直接将该处理过数据进行发送。
所述边缘节点和所述云服务器之间还设置有中间CDN，该中间CDN相对于所述云服务器属于边缘节点，相对于所述边缘节点属于云服务器。即当所述边缘节点向所述云服务器请求数据时，该边缘节点首先项所述中间CDN进行数据请求，每个所述边缘节点可以连接多个所述中间CDN，当所述边缘节点项所述中间CDN请求数据时，同时向多个所述中间CDN发送数据请求信息，每个所述CDN根据自己在本地的检索，回复所述边缘节点是否查询到所述请求对应的数据，当预设的多个所述中间CDN回复的消息都是未查找到，则由指定的中间CDN向云服务器请求，如图2所示。
进一步的，所述中间CDN与所述边缘节点通过网关连接，由所述网关对所述边缘节点和所述中间CDN进行数据路由和负载均衡。
所述网格由多个路由器组成路由组，在所述路由组内，每个路由器可以调用其他路由器进行数据处理，实现所述负载均衡，加快所述数据传输速度。本申请设置有多个相互连接的所述路由组，当一组所述路由组负载过大时，可以调用其他负载较小的路由组参与数据路由处理，如图3所示。
所述云服务器通过边缘路由器与所述中间CDN进行连接，由所述边缘路由器同于进行数据输入输出管理。所述边缘路由器接收数据并将所述数据发送到与所述客户端对应的虚拟机中，同时可以接收所述云服务器向所述客户端发送的数据，并进行转发。
在本申请中，所述缓存批量刷新规则，还同时加载到所述中间CDN节点，并分别基于所述缓存批量刷新规则进行边缘节点和所述中间CDN的缓存刷新。
请参照图1所述，S102接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头。
所述客户端向所述云服务器请求数据时，首先发送访问请求，该访问请求传输到所述边缘节点后，所述边缘节点解析并获取所述访问请求的内容，依据所述访问请求查询对应的数据信息，并将所述数据信息返回到客户端。
在本申请中，当所述访问信息被解析后，首先提取统一资源标识符，所述统一资源标识符(UniformResourceIdentifier，或URI)是一个用于标识某一互联网资源名称的字符串。该种标识允许用户对任何(包括本地和互联网)的资源通过特定的协议进行交互操作。URI由包括确定语法和相关协议的方案所定义。
在Web上可用的每种资源-HTML文档、图像、视频片段、程序等-由一个通用资源标识符(UniformResourceIdentifier,简称"URI")进行定位。
基于所述统一资源标识符进行缓存批量刷新规则的匹配。具体的，所述缓存批量刷新规则是预先设置并缓存到所述边缘节点中的特定字符或者特定字段，将所述统一资源标识符中的字符或者字段与所述缓存批量刷新规则中的特定字符或者字段进行匹配，以确定是否匹配成功。
当所述匹配不成功，则直接将所述统一资源标识符发送到缓存ATS组件，进行数据信息的获取，当所述匹配成功，则触发刷新规则，此时为所述统一资 源标识符添加预设的字头，再发送给缓存ATS组件。ATS是一种既可以用作HTTP代理，也可以用作HTTP缓存的组件，ATS能够缓存任何字节流，支持HTTP协议传输的字节流，当这样的流缓存时，会带一个HTTP协议头部，会命名为缓存中的一个对象object。每个对象通过一个名为cache key的全局唯一值识别。
例如，所述边缘节点接收到访问请求后判断当前统一资源标识符(uri)是否跟缓存批量刷新规则匹配。如果不匹配则直接转发给缓存ATS组件；如果匹配则带上一个X-Ats-Cache-Ims:<timestamp>的预设字头再转发给缓存ATS组件，其中timestamp表示规则的生效时间。
所述统一资源标识符在转发到所述ATS组件后，所述ATS组件从中间CDN处进行数据缓存，此时所述中间CDN可以基于所述客户端的访问请求，进行重复所述缓存批量刷新规则的匹配，判断所述中间CDN实现需要进行数据刷新。所述中间CDN进行缓存批量刷新规则的匹配与所述边缘节点处的步骤相同，在此不再赘述。
在本申请中，所述边缘节点是否通过所述中间CDN进行与云服务器的连接是可以根据实际情况进行设置的，同时所述边缘节点连接的中间CDN节点的数量也是可以根据实际情况进行设置的。
所述边缘节点与客户端距离更近，因此可以更快速的向客户端发送数据，同时所述边缘节点与服务器中还可以设置多级的中间CDN，实现数据的多级缓存。在此，每一级所述中间CDN都设置有上述边缘节点的缓存批量刷新规则，并进行是否进行数据刷新的判断。
请参照图1所述，S103将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新。
所述缓存ATS组件在获取到所述统一资源标识符后，基于所述统一资源标识符进行资源查找，当所述缓存ATS组件请求的uri查找本地缓存，没有命中则回源站获取文件内容。
具体的，所述边缘节点在本地进行资源查找，包括查找不同格式的相同资源，即当一个资源被查找时，该资源可能已经在所述边缘节点进行了存储，但是其格式与所述请求的格式不同。
例如，当所述客户端向所述边缘节点发送一个图片的数据请求，则此时所述边缘节点根据所述uri进行该图片的查找，若未找到该客户端请求的格式图片，而是查找到源图片，则可以在所述边缘节点中对所述源图片进行处理，使该图片符合所述客户端请求的需求。然后将处理后的所述图片进行存储。
进一步的，所述边缘节点还可以根据已经处理的图片进行进一步处理以符合所述客户端请求的格式，这种情况下，需要对所述图片进行判断，确定所述图片经过处理是否符合所述客户端显示的清晰度要求。
具体的，所述判断可由如下公式进行：
其中，所述TS是待处理图片像素，TC是待处理图片尺寸，所述KS是需求图片的像素，所述KC是需求图片尺寸。
当上述公式成立时，则符合所述客户端显示的清晰度要求。进一步的，所述F还可以是≥0.8。
所述的表达式如下：
其中，所述是所述客户端请求过的图片像素，所述A是该像素图片请求的次数，所述是所述客户端请求过的图片的尺寸，所述B是该尺寸图片请求的次数。
进一步的，还需要判读所述TC/KC，设置阈值为D，则当TC/KC≥D时，可以确定所述清晰度是符合预期的。
本申请中，当所述查询未命中时从源站请求当前缓存对应的数据进行刷新。具体的，所述源站即所述云服务器，在本申请中，所述源站、云服务器或者数据中心都是指客户请求数据的原始来源，即与客户端连接的最终服务器。
所述边缘节点将所述客户端的访问请求进行再次打包，并向上一级的中间CDN节点进行发送，所述中间CDN服务器此时再次进行所述缓存批量刷新规 则的匹配，并根据匹配结果进行本地搜索或者项更上一级发送访问请求，依次重复直到访问到所述源站。
请参照图1所述，S104若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
具体的，如果命中缓存，则检查所述访问请求是否带有X-Ats-Cache-Ims字头，如果没有则直接将内容返回给边缘节点。
如果带有X-Ats-Cache-Ims字头，则检查所述字头里的timestamp是否小于本地缓存的时间，如果timestamp小于本地缓存的时间则直接将缓存内容返回给边缘节点；否则将缓存命中状态改为未命中。
本申请中，当所述查询未命中时从源站请求当前缓存对应的数据进行刷新。具体的，所述源站即所述云服务器，在本申请中，所述源站、云服务器或者数据中心都是指客户请求数据的原始来源，即与客户端连接的最终服务器。
所述边缘节点将所述客户端的访问请求进行再次打包，并向上一级的中间CDN节点进行发送，所述中间CDN服务器此时再次进行所述缓存批量刷新规则的匹配，并根据匹配结果进行本地搜索或者项更上一级发送访问请求，依次重复直到访问到所述源站。
进一步的，在本申请中，缓存批量刷新规则超过缓存最大生存时间(例如7天)后主动失效过期，这样可以保持缓存批量刷新规则列表的大小不会一直增长。
本发明在批量刷新时不需要预先清除缓存组件中的内容，只需要在下一次文件被访问到时判断是否回源站拉取新文件，减小了批量缓存刷新对缓存组件造成的压力。
本申请还提供一种支持CDN缓存批量刷新的装置，包括：缓存模块401，匹配模块402，查找模块403，判断模块404。
图4是本申请中支持CDN缓存批量刷新装置示意图。
请参照图4所示，缓存模块401，用于设置并加载缓存批量刷新规则。
所述缓存批量刷新规则可以设置为：将请求信息中同一资源符(uri)中的特定字符、特定字符串设置为触发刷新的规则字符或者规则字符串。例如：所述缓存批量刷新规则是，当/a/*，/*.jpg出现时，则进行缓存刷新。
在本申请中，所述缓存批量刷新规则缓存到边缘节点(CDN)中，该边缘节点分别连接云服务器和客户端，可以基于客户端请求从服务器中拉取信息并转发到客户端。
所述边缘节点接收客户端请求信息，基于所述请求信息从该边缘节点处的本地存储中查找所述请求信息对应的数据发送到客户端。具体的，当接收到所述请求信息时，基于所述请求信息对应的客户端所述请求数据的格式进行数据处理，以及基于所述客户端请求数据，对原始数据进行数据处理，将数据处理后的结果发送到客户端。
当接收到所述请求信息时，本地存储中若未查找到相应数据，则从云服务器中拉取对应的数据，并对所述数据进行处理后进行缓存或者存储，当用户再次请求该数据时，可以直接将该处理过数据进行发送。
所述边缘节点和所述云服务器之间还设置有中间CDN，该中间CDN相对于所述云服务器属于边缘节点，相对于所述边缘节点属于云服务器。即当所述边缘节点向所述云服务器请求数据时，该边缘节点首先项所述中间CDN进行数据请求，每个所述边缘节点可以连接多个所述中间CDN，当所述边缘节点项所述中间CDN请求数据时，同时向多个所述中间CDN发送数据请求信息，每个所述CDN根据自己在本地的检索，回复所述边缘节点是否查询到所述请求对应的数据，当预设的多个所述中间CDN回复的消息都是未查找到，则由指定的中间CDN向云服务器请求。
进一步的，所述中间CDN与所述边缘节点通过网关连接，由所述网关对所述边缘节点和所述中间CDN进行数据路由和负载均衡。
所述网格由多个路由器组成路由组，在所述路由组内，每个路由器可以调用其他路由器进行数据处理，实现所述负载均衡，加快所述数据传输速度。本申请设置有多个相互连接的所述路由组，当一组所述路由组负载过大时，可以调用其他负载较小的路由组参与数据路由处理。
所述云服务器通过边缘路由器与所述中间CDN进行连接，由所述边缘路由器同于进行数据输入输出管理。所述边缘路由器接收数据并将所述数据发送到 与所述客户端对应的虚拟机中，同时可以接收所述云服务器向所述客户端发送的数据，并进行转发。
在本申请中，所述缓存批量刷新规则，还同时加载到所述中间CDN节点，并分别基于所述缓存批量刷新规则进行边缘节点和所述中间CDN的缓存刷新。
请参照图4所述，匹配模块402，用于接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头。
所述客户端向所述云服务器请求数据时，首先发送访问请求，该访问请求传输到所述边缘节点后，所述边缘节点解析并获取所述访问请求的内容，依据所述访问请求查询对应的数据信息，并将所述数据信息返回到客户端。
在本申请中，当所述访问信息被解析后，首先提取统一资源标识符，所述统一资源标识符(UniformResourceIdentifier，或URI)是一个用于标识某一互联网资源名称的字符串。该种标识允许用户对任何(包括本地和互联网)的资源通过特定的协议进行交互操作。URI由包括确定语法和相关协议的方案所定义。
在Web上可用的每种资源-HTML文档、图像、视频片段、程序等-由一个通用资源标识符(UniformResourceIdentifier,简称"URI")进行定位。
基于所述统一资源标识符进行缓存批量刷新规则的匹配。具体的，所述缓存批量刷新规则是预先设置并缓存到所述边缘节点中的特定字符或者特定字段，将所述统一资源标识符中的字符或者字段与所述缓存批量刷新规则中的特定字符或者字段进行匹配，以确定是否匹配成功。
当所述匹配不成功，则直接将所述统一资源标识符发送到缓存ATS组件，进行数据信息的获取，当所述匹配成功，则触发刷新规则，此时为所述统一资源标识符添加预设的字头，再发送给缓存ATS组件。ATS是一种既可以用作HTTP代理，也可以用作HTTP缓存的组件，ATS能够缓存任何字节流，支持HTTP协议传输的字节流，当这样的流缓存时，会带一个HTTP协议头部，会命名为缓存中的一个对象object。每个对象通过一个名为cache key的全局唯一值识别。
例如，所述边缘节点接收到访问请求后判断当前统一资源标识符(uri)是否跟缓存批量刷新规则匹配。如果不匹配则直接转发给缓存ATS组件；如果匹 配则带上一个X-Ats-Cache-Ims:<timestamp>的预设字头再转发给缓存ATS组件，其中timestamp表示规则的生效时间。
所述统一资源标识符在转发到所述ATS组件后，所述ATS组件从中间CDN处进行数据缓存，此时所述中间CDN可以基于所述客户端的访问请求，进行重复所述缓存批量刷新规则的匹配，判断所述中间CDN实现需要进行数据刷新。所述中间CDN进行缓存批量刷新规则的匹配与所述边缘节点处的步骤相同，在此不再赘述。
在本申请中，所述边缘节点是否通过所述中间CDN进行与云服务器的连接是可以根据实际情况进行设置的，同时所述边缘节点连接的中间CDN节点的数量也是可以根据实际情况进行设置的。
所述边缘节点与客户端距离更近，因此可以更快速的向客户端发送数据，同时所述边缘节点与服务器中还可以设置多级的中间CDN，实现数据的多级缓存。在此，每一级所述中间CDN都设置有上述边缘节点的缓存批量刷新规则，并进行是否进行数据刷新的判断。
请参照图4所述，查找模块403，用于将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新。
所述缓存ATS组件在获取到所述统一资源标识符后，基于所述统一资源标识符进行资源查找，当所述缓存ATS组件请求的uri查找本地缓存，没有命中则回源站获取文件内容。
具体的，所述边缘节点在本地进行资源查找，包括查找不同格式的相同资源，即当一个资源被查找时，该资源可能已经在所述边缘节点进行了存储，但是其格式与所述请求的格式不同。
例如，当所述客户端向所述边缘节点发送一个图片的数据请求，则此时所述边缘节点根据所述uri进行该图片的查找，若未找到该客户端请求的格式图片，而是查找到源图片，则可以在所述边缘节点中对所述源图片进行处理，使该图片符合所述客户端请求的需求。然后将处理后的所述图片进行存储。
进一步的，所述边缘节点还可以根据已经处理的图片进行进一步处理以符合所述客户端请求的格式，这种情况下，需要对所述图片进行判断，确定所述图片经过处理是否符合所述客户端显示的清晰度要求。
具体的，所述判断可由如下公式进行：
其中，所述TS是待处理图片像素，TC是待处理图片尺寸，所述KS是需求图片的像素，所述KC是需求图片尺寸。
当上述公式成立时，则符合所述客户端显示的清晰度要求。进一步的，所述F还可以是≥0.8。
所述的表达式如下：
其中，所述是所述客户端请求过的图片像素，所述A是该像素图片请求的次数，所述是所述客户端请求过的图片的尺寸，所述B是该尺寸图片请求的次数。
进一步的，还需要判读所述TC/KC，设置阈值为D，则当TC/KC≥D时，可以确定所述清晰度是符合预期的。
本申请中，当所述查询未命中时从源站请求当前缓存对应的数据进行刷新。具体的，所述源站即所述云服务器，在本申请中，所述源站、云服务器或者数据中心都是指客户请求数据的原始来源，即与客户端连接的最终服务器。
所述边缘节点将所述客户端的访问请求进行再次打包，并向上一级的中间CDN节点进行发送，所述中间CDN服务器此时再次进行所述缓存批量刷新规则的匹配，并根据匹配结果进行本地搜索或者项更上一级发送访问请求，依次重复直到访问到所述源站。
请参照图4所述，判断模块404，用于若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
具体的，如果命中缓存，则检查所述访问请求是否带有X-Ats-Cache-Ims字头，如果没有则直接将内容返回给边缘节点。
如果带有X-Ats-Cache-Ims字头，则检查所述字头里的timestamp是否小于本地缓存的时间，如果timestamp小于本地缓存的时间则直接将缓存内容返回给边缘节点；否则将缓存命中状态改为未命中。
本申请中，当所述查询未命中时从源站请求当前缓存对应的数据进行刷新。具体的，所述源站即所述云服务器，在本申请中，所述源站、云服务器或者数据中心都是指客户请求数据的原始来源，即与客户端连接的最终服务器。
所述边缘节点将所述客户端的访问请求进行再次打包，并向上一级的中间CDN节点进行发送，所述中间CDN服务器此时再次进行所述缓存批量刷新规则的匹配，并根据匹配结果进行本地搜索或者项更上一级发送访问请求，依次重复直到访问到所述源站。
进一步的，在本申请中，缓存批量刷新规则超过缓存最大生存时间(例如7天)后主动失效过期，这样可以保持缓存批量刷新规则列表的大小不会一直增长。
本发明在批量刷新时不需要预先清除缓存组件中的内容，只需要在下一次文件被访问到时判断是否回源站拉取新文件，减小了批量缓存刷新对缓存组件造成的压力。
本领域内的技术人员应明白，本申请的实施例可提供为方法、装置、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。本申请实施例中的方案可以采用各种计算机语言实现，例如，面向对象的程序设计语言Java和直译式脚本语言JavaScript等。
本申请是参照根据本申请实施例的方法、装置(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (10)

  1. 一种支持CDN缓存批量刷新的方法，其特征在于，包括：
    设置并加载缓存批量刷新规则；
    接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；
    将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；
    若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
  2. 根据权利要求1所述支持CDN缓存批量刷新的方法，其特征在于，所述缓存批量刷新规则设置有效时限。
  3. 根据权利要求1所述支持CDN缓存批量刷新的方法，其特征在于，还包括：
    在文件被访问到时判断是否回源站拉取新文件，不预先清除缓存组件中的内容。
  4. 根据权利要求1所述支持CDN缓存批量刷新的方法，其特征在于，所述从源站请求当前缓存对应的数据进行刷新设置为一次刷新多个文件或者目录。
  5. 根据权利要求1所述支持CDN缓存批量刷新的方法，其特征在于，执行刷新后，各个CDN节点对应单个资源，或符合后缀名的资源，或属于刷新目录下的资源，统一被清除。
  6. 一种支持CDN缓存批量刷新的装置，其特征在于，包括：
    缓存模块，用于设置并加载缓存批量刷新规则；
    匹配模块，用于接收访问请求，根据所述访问请求中的统一资源标识符匹配所述缓存批量刷新规则，将匹配到所述缓存批量刷新规则的访问请求中添加预设字头；
    查找模块，用于将所述访问请求或者添加预设字头的访问请求发送到缓存ATS组件，基于所述统一资源标识符进行查找，未命中时从源站请求当前缓存对应的数据进行刷新；
    判断模块，用于若所述查找命中，则判断所述预设字头中的规则生效时间与缓存时间的大小，当所述规则生效时间小于本地的缓存时间，则读取所述缓存内容，否则命中状态改为未命中。
  7. 根据权利要求6所述支持CDN缓存批量刷新的装置，其特征在于，所述缓存批量刷新规则设置有效时限。
  8. 根据权利要求6所述支持CDN缓存批量刷新的装置，其特征在于，还包括：
    在文件被访问到时判断是否回源站拉取新文件，不预先清除缓存组件中的内容。
  9. 根据权利要求6所述支持CDN缓存批量刷新的装置，其特征在于，所述从源站请求当前缓存对应的数据进行刷新设置为一次刷新多个文件或者目录。
  10. 根据权利要求6所述支持CDN缓存批量刷新的装置，其特征在于，执行刷新后，各个CDN节点对应单个资源，或符合后缀名的资源，或属于刷新目录下的资源，统一被清除。
PCT/CN2023/073996 2022-12-15 2023-01-31 一种支持cdn缓存批量刷新的方法及装置 WO2024124663A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211612035.1 2022-12-15
CN202211612035.1A CN115883659B (zh) 2022-12-15 2022-12-15 一种支持cdn缓存批量刷新的方法及装置

Publications (1)

Publication Number Publication Date
WO2024124663A1 true WO2024124663A1 (zh) 2024-06-20

Family

ID=85767625

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/073996 WO2024124663A1 (zh) 2022-12-15 2023-01-31 一种支持cdn缓存批量刷新的方法及装置

Country Status (2)

Country Link
CN (1) CN115883659B (zh)
WO (1) WO2024124663A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115883659B (zh) * 2022-12-15 2023-10-03 江苏云工场信息技术有限公司 一种支持cdn缓存批量刷新的方法及装置
CN116846979B (zh) * 2023-08-29 2024-03-15 江苏睿鸿网络技术股份有限公司 一种云计算环境下资源的调度方法及调度系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8412823B1 (en) * 2009-03-27 2013-04-02 Amazon Technologies, Inc. Managing tracking information entries in resource cache components
CN106202112A (zh) * 2015-05-06 2016-12-07 阿里巴巴集团控股有限公司 缓存目录刷新方法和装置
US20200153932A1 (en) * 2018-11-14 2020-05-14 International Business Machines Corporation Content acceleration for cross-site resources by cdn
WO2021007752A1 (zh) * 2019-07-15 2021-01-21 华为技术有限公司 内容分发网络中的回源方法及相关装置
CN113886743A (zh) * 2021-12-08 2022-01-04 北京金山云网络技术有限公司 缓存资源的刷新方法、装置及系统
CN115883659A (zh) * 2022-12-15 2023-03-31 江苏云工场信息技术有限公司 一种支持cdn缓存批量刷新的方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8990507B2 (en) * 2012-06-13 2015-03-24 International Business Machines Corporation Storing data in a system memory for a subsequent cache flush
CN111385316B (zh) * 2018-12-27 2022-06-21 阿里巴巴集团控股有限公司 资源刷新方法、访问方法及装置
CN112711730B (zh) * 2021-01-21 2021-10-22 上海观察者信息技术有限公司 一种数据的缓存方法、装置、存储介质及电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8412823B1 (en) * 2009-03-27 2013-04-02 Amazon Technologies, Inc. Managing tracking information entries in resource cache components
CN106202112A (zh) * 2015-05-06 2016-12-07 阿里巴巴集团控股有限公司 缓存目录刷新方法和装置
US20200153932A1 (en) * 2018-11-14 2020-05-14 International Business Machines Corporation Content acceleration for cross-site resources by cdn
WO2021007752A1 (zh) * 2019-07-15 2021-01-21 华为技术有限公司 内容分发网络中的回源方法及相关装置
CN113886743A (zh) * 2021-12-08 2022-01-04 北京金山云网络技术有限公司 缓存资源的刷新方法、装置及系统
CN115883659A (zh) * 2022-12-15 2023-03-31 江苏云工场信息技术有限公司 一种支持cdn缓存批量刷新的方法及装置

Also Published As

Publication number Publication date
CN115883659B (zh) 2023-10-03
CN115883659A (zh) 2023-03-31

Similar Documents

Publication Publication Date Title
US11194719B2 (en) Cache optimization
US9888089B2 (en) Client side cache management
EP2263163B1 (en) Content management
US8510415B2 (en) Data distribution method, data distribution system and relevant devices in edge network
WO2024124663A1 (zh) 一种支持cdn缓存批量刷新的方法及装置
WO2017092351A1 (zh) 缓存数据的更新方法及装置
WO2019057212A1 (zh) Cdn节点内服务调度方法、装置、设备及存储介质
JP2008511078A (ja) ゲスト画像閲覧性能を向上するための写真共有ピア・ツー・ピア・ネットワークにおけるプロキシ・キャッシング
CN104796439B (zh) 网页推送方法、客户端、服务器及系统
JP2013507694A (ja) データ通信高速化および効率化のためのシステムおよびその方法
WO2021253889A1 (zh) 负载均衡方法、装置、代理设备、缓存设备及服务节点
US8909808B2 (en) Redundancy elimination for web caching
JP2010250799A (ja) リソースの位置情報の要求方法、当該方法のためのユーザノードおよびサーバ
RU2642833C2 (ru) Способ и устройство для обеспечения медиаресурса
CN106850572B (zh) 目标资源的访问方法和装置
WO2017080459A1 (zh) 服务内容的缓存及提供方法、装置、系统和存储介质
CN108574686A (zh) 一种在线预览文件的方法及装置
CN107070988A (zh) 报文处理方法及装置
US10491606B2 (en) Method and apparatus for providing website authentication data for search engine
CN103416027A (zh) 缓存优化的方法、缓存器和缓存优化的系统
US20200394678A1 (en) Minimizing impact of experimental content delivery on computing devices
US20160050292A1 (en) Local web resource access
CN114979025B (zh) 资源刷新方法、装置、设备及可读存储介质
CN101668029A (zh) 一种网络设备和网络通讯的方法
CN103685367A (zh) 离线下载系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23901883

Country of ref document: EP

Kind code of ref document: A1