CN108416016A - A kind of CDN is by prefix caching sweep-out method and system - Google Patents
A kind of CDN is by prefix caching sweep-out method and system Download PDFInfo
- Publication number
- CN108416016A CN108416016A CN201810180245.5A CN201810180245A CN108416016A CN 108416016 A CN108416016 A CN 108416016A CN 201810180245 A CN201810180245 A CN 201810180245A CN 108416016 A CN108416016 A CN 108416016A
- Authority
- CN
- China
- Prior art keywords
- prefix
- time
- file
- refresh
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1727—Details of free space management performed by the file system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Abstract
The present invention relates to a kind of this CDN to cache removing system, including data preprocessing module by prefix：Have the functions such as extraction, the refreshing of domain name prefix list；Memory module：The module has domain name list store function, prefix refresh tasks time and cache-time store function；Data Analysis Services module：It carries out the analyzing processing of file prefix data and obtains file from source station again.This CDN caches sweep-out method, and domain name safeguards prefix refresh list, and each machine is arrived in storage, and records the time of the prefix refresh tasks；For each file, the time that storage file caches in file metadata；When request reaches every time, prefix refresh list first more whether is hit, the file cache time is inquired if hit, if refresh time is newer than cache-time, this document is removed and time higher level obtains this document again.The present invention substitutes full dose deleting mechanism by mark function, and the action removed by prefix can quickly come into force.
Description
Technical field
The invention belongs to information technology fields, more particularly to a kind of CDN by prefix caching sweep-out method and system.
Background technology
In CDN distributed systems, for content by caching mechanism, CDN node is arrived in storage, user by CDN node come
Obtain data.Because CDN node caches, it is possible to there is a situation where that CDN node and source station data are inconsistent.In practical industry
In business, it is frequently encountered the function of removing and specify file cache in CDN, to achieve the purpose that CDN and source station data are fully synchronized.
In actual use, bibliographic structure presence-set pattern rule of website is frequently encountered the function of All Files under update specified prefix
(such as：Remove All Files under some particular category).In the mechanism removed by prefix, need CDN node that can store prefix
With the correspondence of file, because the file of node storage is numerous, file is also frequently expired to be eliminated, and safeguards that this relationship is more multiple
It is miscellaneous.Traditional scheme is the cache information record log to CDN node, often accesses the file in primary caching, then visits this
The resource address asked is recorded in file or database.By the resource record to storage sort by prefix, obtains certain
A prefix contains the information of which file.When prefix is removed, according to index situation, go to remove each money in caching one by one
Source.Traditional scheme disadvantage：1. pair request record access daily record every time, and access log is sorted by prefix to performance cost
Greatly；2. each prefix is removed, needing at once all to remove the corresponding All Files of the prefix, the action consumption of removing is big, and
And it is long to remove the time come into force.
Invention content
The present invention provides CDN more than one by prefix caching sweep-out method and system, it is therefore intended that improves caching cleaning performance
And refresh efficiency.
This CDN provided by the invention caches removing system, including data preprocessing module by prefix：Has domain name prefix
The functions such as extraction, the refreshing of list；Memory module：The module have domain name list store function, the prefix refresh tasks time, with
And cache-time store function；Data Analysis Services module：Carry out the analyzing processing of file prefix data and again from source station
Obtain file.
This CDN provided by the invention caches sweep-out method by prefix, and domain name safeguards that prefix refresh list, storage arrive each
Machine, and record the time of the prefix refresh tasks；For each file, storage file caches in file metadata
Time；When request reaches every time, prefix refresh list first more whether is hit, the file cache time is inquired if hit, if
Refresh time is newer than cache-time, then removes this document and time higher level obtains this document again.
It is an advantage of the invention that：1. lookup when asking by label and every time acts, instead of the index of prefix
And sequencer procedure.It realizes just for needing the file removed by prefix to be just indexed inquiry, reduces performance cost；2. passing through
Mark function substitutes full dose deleting mechanism, and the action removed by prefix can quickly come into force..
Description of the drawings
Fig. 1 is schematic diagram of the present invention.
Specific implementation mode
It elaborates below to the preferred embodiment of the present invention.
This CDN provided by the invention caches removing system, including data preprocessing module by prefix：Has domain name prefix
The functions such as extraction, the refreshing of list；Memory module：The module have domain name list store function, the prefix refresh tasks time, with
And cache-time store function；Data Analysis Services module：Carry out the analyzing processing of file prefix data and again from source station
Obtain file.
This CDN of the present invention caches sweep-out method, as shown in Figure 1, domain name safeguards that prefix refresh list, storage arrive each machine
Device, and record the time of the prefix refresh tasks；For each file, storage file caches in file metadata
Time；When request reaches every time, prefix refresh list first more whether is hit, the file cache time is inquired if hit, if brush
The new time is newer than cache-time, then removes this document and time higher level obtains this document again.
Certainly, those skilled in the art are it should be appreciated that above example is intended merely to illustrate the present invention
, and not limitation of the invention, as long as within the scope of the invention to the variation of above example, modification all in the present invention
Protection domain within.
Claims (2)
1. a kind of CDN caches removing system by prefix, it is characterised in that：Including data preprocessing module：Have domain name prefix row
The functions such as extraction, the refreshing of table；Memory module：The module have domain name list store function, the prefix refresh tasks time and
Cache-time store function；Data Analysis Services module：It carries out the analyzing processing of file prefix data and is obtained again from source station
Take file.
2. a kind of CDN caches sweep-out method by prefix, it is characterised in that：Domain name safeguards that prefix refresh list, storage arrive each machine
Device, and record the time of the prefix refresh tasks；For each file, storage file caches in file metadata
Time；When request reaches every time, prefix refresh list first more whether is hit, the file cache time is inquired if hit, if brush
The new time is newer than cache-time, then removes this document and time higher level obtains this document again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810180245.5A CN108416016A (en) | 2018-03-05 | 2018-03-05 | A kind of CDN is by prefix caching sweep-out method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810180245.5A CN108416016A (en) | 2018-03-05 | 2018-03-05 | A kind of CDN is by prefix caching sweep-out method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108416016A true CN108416016A (en) | 2018-08-17 |
Family
ID=63130196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810180245.5A Pending CN108416016A (en) | 2018-03-05 | 2018-03-05 | A kind of CDN is by prefix caching sweep-out method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108416016A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274762A (en) * | 2018-10-22 | 2019-01-25 | 杭州领智云画科技有限公司 | CDN method for refreshing and system |
| CN112513830A (en) * | 2019-07-15 | 2021-03-16 | 华为技术有限公司 | Back-source method and related device in content distribution network |
| CN112861030A (en) * | 2019-11-27 | 2021-05-28 | 北京金山云网络技术有限公司 | CDN refreshing method and device, cache server and storage medium |
| CN113271359A (en) * | 2021-05-19 | 2021-08-17 | 北京百度网讯科技有限公司 | Method and device for refreshing cache data, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1489333A (en) * | 2002-10-10 | 2004-04-14 | 华为技术有限公司 | Method for updating content in content-transmitting network |
| CN101184021A (en) * | 2007-12-14 | 2008-05-21 | 华为技术有限公司 | Method, equipment and system for implementing stream media caching replacement |
| CN103401950A (en) * | 2013-08-21 | 2013-11-20 | 网宿科技股份有限公司 | Cache asynchronism refreshment method, as well as method and system for processing requests by cache server |
| CN103973820A (en) * | 2013-01-28 | 2014-08-06 | 深圳市快播科技有限公司 | Method for optimizing domain name system and optimized domain name system |
| CN105227341A (en) * | 2014-06-30 | 2016-01-06 | 帕洛阿尔托研究中心公司 | For the system and method by content center network management devices |
| CN105653198A (en) * | 2014-11-13 | 2016-06-08 | 杭州迪普科技有限公司 | Data processing method and device |
| CN105868251A (en) * | 2015-12-22 | 2016-08-17 | 乐视云计算有限公司 | Cache data updating method and device |
| CN106202112A (en) * | 2015-05-06 | 2016-12-07 | 阿里巴巴集团控股有限公司 | CACHE DIRECTORY method for refreshing and device |
| CN107729331A (en) * | 2016-08-11 | 2018-02-23 | 凡学（上海）教育科技有限公司 | A kind of method of Mobile solution cache HTML5 page resources |
-
2018
- 2018-03-05 CN CN201810180245.5A patent/CN108416016A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1489333A (en) * | 2002-10-10 | 2004-04-14 | 华为技术有限公司 | Method for updating content in content-transmitting network |
| CN101184021A (en) * | 2007-12-14 | 2008-05-21 | 华为技术有限公司 | Method, equipment and system for implementing stream media caching replacement |
| CN103973820A (en) * | 2013-01-28 | 2014-08-06 | 深圳市快播科技有限公司 | Method for optimizing domain name system and optimized domain name system |
| CN103401950A (en) * | 2013-08-21 | 2013-11-20 | 网宿科技股份有限公司 | Cache asynchronism refreshment method, as well as method and system for processing requests by cache server |
| CN105227341A (en) * | 2014-06-30 | 2016-01-06 | 帕洛阿尔托研究中心公司 | For the system and method by content center network management devices |
| CN105653198A (en) * | 2014-11-13 | 2016-06-08 | 杭州迪普科技有限公司 | Data processing method and device |
| CN106202112A (en) * | 2015-05-06 | 2016-12-07 | 阿里巴巴集团控股有限公司 | CACHE DIRECTORY method for refreshing and device |
| CN105868251A (en) * | 2015-12-22 | 2016-08-17 | 乐视云计算有限公司 | Cache data updating method and device |
| CN107729331A (en) * | 2016-08-11 | 2018-02-23 | 凡学（上海）教育科技有限公司 | A kind of method of Mobile solution cache HTML5 page resources |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274762A (en) * | 2018-10-22 | 2019-01-25 | 杭州领智云画科技有限公司 | CDN method for refreshing and system |
| CN112513830A (en) * | 2019-07-15 | 2021-03-16 | 华为技术有限公司 | Back-source method and related device in content distribution network |
| CN112861030A (en) * | 2019-11-27 | 2021-05-28 | 北京金山云网络技术有限公司 | CDN refreshing method and device, cache server and storage medium |
| CN112861030B (en) * | 2019-11-27 | 2024-04-05 | 北京金山云网络技术有限公司 | CDN refreshing method and device, cache server and storage medium |
| CN113271359A (en) * | 2021-05-19 | 2021-08-17 | 北京百度网讯科技有限公司 | Method and device for refreshing cache data, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108416016A (en) | A kind of CDN is by prefix caching sweep-out method and system | |
| CN111459985B (en) | Identification information processing method and device | |
| GB2595615A (en) | Managing data objects for graph-based data structures | |
| CN105224546B (en) | Data storage and query method and equipment | |
| CN107025243A (en) | A kind of querying method of resource data, inquiring client terminal and inquiry system | |
| DE102013206744A1 (en) | DEDUPLICATING STORAGE WITH IMPROVED DETECTION OF COMMON STOPS | |
| US8041893B1 (en) | System and method for managing large filesystem-based caches | |
| US20150341771A1 (en) | Hotspot aggregation method and device | |
| US9015414B2 (en) | Load balancing based upon data usage | |
| US20130185429A1 (en) | Processing Store Visiting Data | |
| CN104750754A (en) | Website industry classification method and server | |
| CN110888837B (en) | Object storage small file merging method and device | |
| CN105224560B (en) | Cache data searching method and device | |
| CN110109910A (en) | Data processing method and system, electronic equipment and computer readable storage medium | |
| CN106960020B (en) | A kind of method and apparatus creating concordance list | |
| US20140025899A1 (en) | Efficiently Updating and Deleting Data in a Data Storage System | |
| US20220075791A1 (en) | Storing data items and identifying stored data items | |
| CN103617266A (en) | Personalized extension search method, device and system | |
| WO2017165914A1 (en) | "methods and systems for database optimisation" | |
| US20230164171A1 (en) | Systems and methods for rapidly generating security ratings | |
| CN106407442B (en) | A kind of mass text data processing method and device | |
| CN104834650A (en) | Method and system for generating effective query tasks | |
| CN110737821A (en) | Similar event query method, device, storage medium and terminal equipment | |
| CN107193870B (en) | Webpage content extraction method and system | |
| CN109446167A (en) | A kind of storage of daily record data, extracting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180817 |
|
| RJ01 | Rejection of invention patent application after publication |