CN106921713A - A kind of resource caching method and device - Google Patents
A kind of resource caching method and device Download PDFInfo
- Publication number
- CN106921713A CN106921713A CN201510999566.4A CN201510999566A CN106921713A CN 106921713 A CN106921713 A CN 106921713A CN 201510999566 A CN201510999566 A CN 201510999566A CN 106921713 A CN106921713 A CN 106921713A
- Authority
- CN
- China
- Prior art keywords
- resource
- log cache
- caching
- domain name
- url
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9566—URL specific, e.g. using aliases, detecting broken or misspelled links
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of resource caching method and device, it is used to realize automatically generate caching rule according to log cache resource.The method includes：Obtain domain name to be analyzed；For any domain name to be analyzed, obtain and specify in the time period the corresponding log cache of the domain name to be analyzed；First kind key message is extracted from the corresponding any log cache of the domain name to be analyzed；Determine whether the log cache is that can optimize log cache according at least to first kind key message；If it is determined that the log cache in the corresponding regular expression of resource depth levels of the domain name field information input of the URL in the log cache to the URL, will then generate the caching rule of the log cache for that can optimize log cache.The method automatically generates correspondence caching rule according to the resource depth and designated domain name that can optimize log cache URL, and the regular expression for being generated is more targeted, and each cached parameters setting of new caching rule is also more reasonable, can effectively lift buffer efficiency.
Description
Technical field
The present embodiments relate to communication technical field, more particularly to a kind of resource caching method and device.
Background technology
In existing cache memory Cache forthright systems, by artificial newly-increased domain name, focus money is set
The rules such as source cache carry out caching resource and renewal.
Existing resource caching, update method are carried out according to the newly-increased domain name of forthright system and respective cache rule
, Top N domain names are periodically provided by operator, tested gentle one by one for each domain name using manpower
Deposit, be primarily upon network element and coarse-grain flow, it is less efficient based on manual analyzing, and labor intensive
More and excessive cycle, it is impossible to meet the renewal frequency of internet hot spots resource.
The caching rule of existing resource caching method is only capable of according to wall scroll resource URL (Uniform Resoure
Locator, uniform resource locator) write its it is corresponding caching rule and adjustment caching rule in each
Parameter, it is impossible to realize the unitized of caching rule.Specifically, being all resources in Cache forthright systems
There is provided a general caching rule, if certain resource is without dedicated rules, the Auto-matching general rule is carried out
, because all parameters of general rule are empirical value, and there is regular expression and write more general in caching,
The problems such as being related to suffix excessive, in rule match, expense is excessive, and empirical parameter cannot also fully ensure that all
The caching effect of resource, causes part resource to cache effect on driving birds is not good or even cannot cache.
To sum up, exist in the prior art by manpower carry out caching resource and update have efficiency it is low, update
A kind of low deficiency of frequency, it would be highly desirable to method that caching rule is automatically generated according to statistics.
The content of the invention
The embodiment of the present invention provides a kind of resource caching method and device, is used to realize according to log cache resource
Automatically generate caching rule.
The embodiment of the present invention provides a kind of resource caching method, including：
Obtain domain name to be analyzed；
For any domain name to be analyzed, obtain and specify in the time period the corresponding log cache of the domain name to be analyzed；
And
First kind key message is extracted from the corresponding any log cache of the domain name to be analyzed；
Determine whether the log cache is that can optimize log cache according at least to the first kind key message；
If it is determined that the log cache is for that can optimize log cache, then by the domain name of the URL in the log cache
Field information is input in the corresponding regular expression of resource depth levels of the URL, generates the caching day
The caching rule of will；
Wherein, the corresponding regular expression of resource depth levels of the URL is the money according at least to the URL
The regular expression that cached parameters corresponding to Depth grade and the resource depth levels are write in advance.
The embodiment of the present invention provides a kind of caching resource device, including：
First acquisition unit, for obtaining domain name to be analyzed；
Second acquisition unit, for for any domain name to be analyzed, obtaining and specifying the domain to be analyzed in the time period
The corresponding log cache of name；
Resource analysis unit, closes for extracting the first kind from the corresponding any log cache of the domain name to be analyzed
Key information；And determine whether the log cache is that can optimize caching day according at least to the first kind key message
Will；
Rule generating unit, for if it is determined that the log cache is can to optimize log cache, then by the caching day
Resource depth levels corresponding regular expression of the domain name field information input of the URL in will to the URL
In, generate the caching rule of the log cache；
Wherein, the corresponding regular expression of resource depth levels of the URL is the money according at least to the URL
The regular expression that cached parameters corresponding to Depth grade and the resource depth levels are write in advance.
In above-described embodiment, obtain and specify the corresponding log cache of domain name to be analyzed in the time period, treated point from this
First kind key message, such as hit mark, HTTP states are extracted in the corresponding any log cache of analysis domain name
Code, resource size, resource URL, whether with the data such as general caching rule match, and according at least to first
Class key message determines whether the log cache is that can optimize log cache；Further basis can optimize caching day
The resource depth and designated domain name of will URL, automatically generate correspondence caching rule；Certain domain name is divided
Analysis, and it is also more targeted according to the regular expression normal form that different resource URL depth is generated respectively,
Each cached parameters setting is also more reasonable, when subsequently being cached according to the caching rule of optimization, can shorten domain
Name and the match time length of the caching rule of optimization, can effectively lift buffer efficiency.
Brief description of the drawings
Technical scheme in order to illustrate more clearly the embodiments of the present invention, institute in being described to embodiment below
The accompanying drawing for needing to use is briefly introduced, it should be apparent that, drawings in the following description are only of the invention
Some embodiments, for one of ordinary skill in the art, are not paying the premise of creative labor
Under, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of flow chart of resource caching method provided in an embodiment of the present invention；
Fig. 2 is a kind of flow chart for automatically generating optimization caching rule provided in an embodiment of the present invention；
Fig. 3 is a kind of flow chart of resource caching method provided in an embodiment of the present invention；
Fig. 4 is a kind of structural representation of caching resource device provided in an embodiment of the present invention.
Specific embodiment
In order that the object, technical solutions and advantages of the present invention are clearer, below in conjunction with accompanying drawing to this hair
It is bright to be described in further detail, it is clear that described embodiment is only a part of embodiment of the invention,
Rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art are not doing
Go out all other embodiment obtained under the premise of creative work, belong to the scope of protection of the invention.
In order to solve present in prior art to carry out caching resource and update having efficiency low, more by manpower
The low not enough technical problem of new frequency, the embodiment of the invention provides a kind of a kind of resource as shown in Figure 1
Caching method, is used to realize automatically generate caching rule according to log cache resource, and idiographic flow includes：
Step 101, obtains domain name to be analyzed；
Step 102, for any domain name to be analyzed, obtains the domain name to be analyzed in specifying the time period corresponding
Log cache；
Step 103, first kind key message is extracted from the corresponding any log cache of the domain name to be analyzed；
Step 104, determines whether the log cache is that can optimize caching day according at least to first kind key message
Will；
Step 105, however, it is determined that the log cache is that can optimize log cache, then by the URL in the log cache
Domain name field information input in the corresponding regular expression of resource depth levels of the URL, generation should
The caching rule of log cache；Wherein, the corresponding regular expression of resource depth levels of the URL be to
Cached parameters corresponding to few resource depth levels and the resource depth levels according to the URL are write in advance
Regular expression.
In the embodiment of the present invention, performed using the Cache cache optimizations instrument with resource analysis functional module
Above method flow come realize to obtain log cache resource counted, analyze, automatically generate caching rule
Then.And complete to dock by the Cache cache optimizations instrument and existing network caching server, can obtain needs analysis
Log cache, and then in lifting system log cache utilization rate.Using the work Cache cache optimizations
Instrument, operator can independently realize Cache system operations, greatly promote the operating efficiency of cache optimization.
Specifically, in above method process step 101, domain to be analyzed can be obtained by domain name introducting interface
Name, domain name introducting interface can support caching system Top100 the Resources lists and by hand import need analysis
The Resources list, therefore, the domain name data source of importing can be CSS-WEB daily records (Cascading Style
Sheet CSS), or the domain name resources list of importing by hand.Wherein, the embodiment of the present invention
In domain name data source be caching to caching rule optimization and after reaching the standard grade using Cache cache optimizations instrument
Domain name.
In order to whether the link for judging various resource types corresponding to domain name to be analyzed needs optimization and its is excellent
Change space, firstly, it is necessary to be directed to the domain name any to be analyzed of acquisition, obtain corresponding to any domain name to be analyzed
Log cache, secondly, to obtain the corresponding log cache of domain name same to be analyzed parse；Again,
According to analysis result, judge whether the corresponding cache resources of domain name to be analyzed need optimization.
Preferably, by the specified interface of caching server, the specified interface is Cache cache optimization instruments
The interface after docking is completed with existing network caching server, for reading the domain name pair to be analyzed in the specified time period
The log cache answered.The specified interface is encapsulated as that the function of reading daily record in the time of specifying can be provided, and this connects
Mouth can read caching server in from the initial time of input to the time period the end time being input into
The record or access log (access daily records) under Log Directory, and the log cache copy that will be read
To local, log cache have recorded the information that all user uplinks by forthright are asked.
Using the automatic caching day to obtaining of the functional module with resource analysis function in the embodiment of the present invention
Will is analyzed.Specifically, in step 103, to the corresponding log cache of domain name same to be analyzed for obtaining
Parsed, and first kind key message is extracted from the corresponding any log cache of the domain name to be analyzed, carried
The first key message for taking at least includes HTTP conditional codes, may also include：Hit mark, resource size,
Resource URL etc..
Wherein, hit is identified includes the implication of TCP_HIT and TCP_MISS, TCP_HIT to hit Cache
Server buffer resource, the implication of TCP_MISS is miss Cache servers cache resources.
Wherein, if comprising being matched with general caching rule match in parsing the fragment field of log cache
Identification information, it is determined that the composition of the log cache and general caching rule match；If parsing caching day
Not comprising matching identification information with general caching rule match in will, it is determined that the composition of the log cache with
General caching rule is mismatched.
Whether the embodiment of the present invention provides a kind of optional according to first kind key message, determines the log cache
For the mode of log cache can be optimized, specially：If HTTP (the HyperText in first kind key message
Transfer Protocol, HTTP) conditional code is for specified HTTP conditional codes and the first kind is crucial
Do not include the matching identification information of the log cache and general caching rule match in information, then by the caching day
Will is defined as that log cache can be optimized.
For example, the analysis result of several log caches for such as being presented in table 1, wherein, numbering is 4 and 5
In the analysis result of log cache, HTTP conditional codes 200 or 206 are to specify HTTP conditional codes, and
The composition for parsing the log cache that only numbering is 4 and 5 is not matched with general caching rule, therefore, will
Numbering is that 4 and 5 log cache is defined as that log cache can be optimized.
Table 1
| ID | HTTP states | Hit mark | Whether matched rule |
| 1 | 200or 206 | HIT | Matching |
| 2 | 200or 206 | MISS | Matching |
| 3 | Other | Other | Matching |
| 4 | 200or 206 | HIT | Do not match |
| 5 | 200or 206 | MISS | Do not match |
| 6 | Other | Other | Do not match |
Whether in the embodiment of the present invention, it is certainly not limited to according only to HTTP conditional codes, is advised with general caching
The information for then matching determines whether the log cache is that can optimize log cache, it is also possible to as needed will hit
Mark, resource size, resource suffix mark, URL/domain name information etc. information and HTTP conditional codes, whether
Combined to judge whether the log cache is that can optimize log cache with the information of general caching rule match.
Flow is analysed to optimizable caching day in the corresponding all log caches of domain name according to the method described above
Will as can optimize log cache resource, and for each can optimize log cache resource generation optimization caching rule
Then.Specifically, the embodiment of the present invention is by the caching rule generation component in Cache cache optimization instruments
To realize, caching rule generation component automatically generates the step of optimization caching rule according to the data of input
Suddenly, as shown in Fig. 2 including：
Step 201, acquisition can optimize the URL included in log cache；Such as, URL is：
“http://p1.meituan.net/200.120/deal/69fd3838a512e78e1b5bd30774d3efcd19
5473.jpg”。
Step 202, the composition to the URL is analyzed, and obtains the domain name field content of URL；Specifically
Realize that the domain name field content recorded in log cache can be intercepted by domain name analysis tool.
According to the composition of the URL in step 201, its domain name field content is " p1.meituan.net ".Only
Using " p1.meituan.net " in URL as generate caching rule certain domain name, so other
If URL also includes " p1.meituan.net ", the log cache with newly caching rule match can be also generated, its
He is URL such as " http://p1.meituan.net/230.126/utop/1321jjasdfjasdfasdfasdfasdf.png”.
Compared with prior art, the certain domain name part input regular expression of URL is generated into new caching rule,
New caching rule can be improved in use, any domain name and the newly matching efficiency of caching rule.
Step 203, obtains the resource depth of the URL；
In general, the value of resource depth is represented that data area is [0-15] by numeral in { }, specific real
The resource depth of the URL can be now obtained using the instrument with resource depth crawl function.
Step 204, calls the corresponding regular expression of resource depth levels of the URL, input URL's
The value of the resource depth of domain name field content and the URL, exports the caching rule of the log cache.
Wherein, regular expression has certain both regular, automatic according to the establishment of regular expression primitive rule
Change program, according to the specific composition of target URL, is input into the domain name field content of URL with the URL's
The value of resource depth, can automatically generate the caching rule normal form that can match the URL.Using automatic chemical industry
The caching rule that tool is automatically generated, the specific composition according to URL is analyzed to certain domain name, extracts domain name
Field, it is also more targeted according to the regular expression that resource URL depth is generated respectively, during rule match
Efficiency is higher, and each cached parameters setting is also more reasonable, can effectively lift buffer efficiency.
For example, the domain name field content of log cache URL is：http://res.kfc.com.cn；
If the resource depth value of the URL is 3, the caching rule for exporting is：
[policy-res]
matchurl regex http://[^/]*res\.kfc.com.cn(:/[^/\]+){3}(<file>.+)/
Cache_always=yes
Cache_delay=1
Cache_index=res.kfc.com.cn/ $ file
Cache_never=no
Cache_ttl=1209600
If the resource depth value of the URL is 4, the caching rule for exporting is：
[policy-res]
matchurl regex http://[^/]*res\.kfc.com.cn(:/[^/\]+){4}(<file>.+)/
Cache_always=yes
Cache_delay=1
Cache_index=res.kfc.com.cn/ $ file
Cache_never=no
Cache_ttl=1209600
Wherein, the cached parameters in above-mentioned caching rule include：
cache_always：Caching is forced, after setting, no matter whether file header allows caching, carries out
Force caching；
cache_delay：, i.e., be classified as in focus for this request after user's request exceedes threshold value time by hot pixel threshold
Hold and utilize Cache servers to cache；
cache_index：Caching index, improves the regular legibility of caching and labeled；It is specific that caching is indexed
Content is relevant with the domain name field content of URL, and in the example above, the domain name field content of URL is：
“http://res.kfc.com.cn ", caching index is " res.kfc.com.cn/ $ file ", so so that caching rule
Then become apparent from understanding.
cache_never：Forbid caching, the resource forever will not be cached after yes is set；
cache_ttl：File expiration time (unit：S), stop providing clothes more than this document after expired time
Business.
In above-described embodiment, resource analysis module can according to the ruuning situation of domain name of having reached the standard grade, to hit mark,
HTTP conditional codes, resource size, resource URL, whether hit the data such as rule and carry out statistical analysis；Root
According to resource depth and resource type, system automatically generates correspondence caching rule, and using " clonal analysis "
Domain name hit situation before and after the new caching rule application of contrast.Certain domain name is analyzed, according to different resource
Also more targetedly, efficiency is higher during rule match for the regular expression that URL depth is generated respectively, each slow
Deposit parameter setting also more reasonable, can effectively lift buffer efficiency.Cache with resource analysis functional module
Cache optimization instrument simultaneously completes to dock, the utilization rate of log cache in lifting system with existing network caching server
Meanwhile, it is capable of achieving to count specified URL using log cache resource, analyzes, automatically generates caching
Rule and caching effect of optimization comparing function, using the instrument, operator can independently realize that Cache systems are transported
Battalion, greatly promotes cache optimization operating efficiency.
In order that the presentation of the caching rule of output becomes apparent from, and selectable optimization is provided to operation personnel
Scheme, the above-mentioned Cache cache optimization instruments with resource analysis functional module pair can also optimize slow
Deposit daily record to be classified, that is, determine the log cache after can optimizing log cache, also to include：
Equations of The Second Kind key message is extracted from the log cache, according to Equations of The Second Kind key message to the log cache
Affiliated classification is divided.
Specifically, Equations of The Second Kind key message includes resource suffix identification information and resource size information, can be according to
The resource suffix identification information and/or resource size information recorded in log cache, determine belonging to log cache
Classification.
For example, when can optimize log cache according to the size pair of the resource suffix of log cache and dividing, can
The identity type of the big file of resource suffix is set to wmv, asf, asx, mpg, mpeg, mlv, m2v
Deng the identity type of resource suffix small documents being set into wmp, cif, gif, jpg, jpeg, bmp, pcx
Deng.The resource suffix identification information of log cache and above-mentioned setting are compared, if the resource of log cache
Suffix identification information belongs to the identity type of the big file of resource suffix, then after the log cache being categorized into resource
In sewing the classification of big file, if the resource suffix identification information of log cache belongs to the mark of resource suffix small documents
Know type, then the log cache is categorized into the classification of resource suffix small documents.
For example, when can optimize log cache according to the resource size pair of log cache and dividing, can be big
The threshold range of resource file is set to (1024,3145728) KB, can be by the threshold value of small resource file
Scope is set to (0,1024) KB.If the resource size information of log cache meets the threshold of large resource file
, then be categorized into the log cache in the classification of large resource file by value scope；If the resource of the log cache is big
Small information meets the threshold range of small resource file, then the log cache is categorized into the classification of small resource file
In.
For example, log cache can be optimized being：
“23Sep2015:085211.418611 0.000366 0.000413 TCP_HIT 200 14415 0 GET
183.*.*.181
http://i3.itc.cn/20150909/340e_4227c137_0f65_bd6b_f501_e2cc10220cab_1.jpg”
Wherein, the corresponding URL of the log cache is：
“http://i3.itc.cn/20150909/340e_4227c137_0f65_bd6b_f501_e2cc10
220cab_1.jpg”；The domain name field information of the URL is " i3.itc.cn "；The resource of the log cache
Size information is：“14415”；The resource suffix identification information of the log cache is " .jpg ".
In a kind of optional implementation method of the embodiment of the present invention, can be analysed to according to above-mentioned sorting technique
All log caches that optimize of domain name are classified, and for the log cache in each classification, and according to
The domain name field information of the URL of any log cache and the resource depth levels of the URL are corresponding in each classification
Regular expression, be the log cache generation optimization after caching rule, and by the log cache with optimization
The corresponding relation of caching rule afterwards is stored in the category.
In a kind of optional implementation method of the embodiment of the present invention, can be according to above-mentioned sorting technique, after resource
The domain name field information for sewing URL up in the classification of file and/or large resource file in any log cache is defeated
Enter in the corresponding regular expression of resource depth levels of the URL, generate the caching rule of the log cache,
And the caching rule storage that will be generated is in the category.
For example, the embodiment of the present invention provides a kind of Cache cache optimizations instrument, at least including input interface mould
Block, resource analysis processing module caches rule generation component, and the Cache cache optimizations instrument is performed
The step of above method flow as shown in figure 3, including：
Step 301, calls input interface module, obtains CSS-WEB Top100 domain name resources lists or craft
The domain name analyzed of the domain name resources list of importing；
Step 302, calls the interface between Cache cache optimizations instrument and caching server, obtains selected
The log cache of domain name can be analyzed in time period；
Step 303, calls resource analysis processing module, and the log cache that pair can analyze domain name is analyzed；
Wherein, any log cache for analyzing domain name is analyzed including：
Parsing information based on any log cache for analyzing domain name determines that log cache can be optimized；Such as,
The HTTP conditional codes that will be parsed are to specify HTTP conditional codes and do not parse and general caching rule match
The log cache of matching identification is defined as that log cache can be optimized.
Other parsing information pair based on any log cache for analyzing domain name can optimize log cache to be carried out
Classification；Such as, caching day can be optimized according to the resource suffix identification information and resource size information that parse
Will is classified, by log cache be divided into the big file of resource suffix, resource suffix small documents, the big file of resource,
Four classifications of resource small documents；
Step 304, calls caching rule generation component, for the resource class chosen under it is each slow
Daily record is deposited, above-mentioned steps 201 to 204 are performed, each log cache under the resource class that output is chosen
Optimization caching rule；
In this step, resource class that can be as needed only to operator's concern carries out the excellent of caching rule
Change, such as caching rule is carried out to the log cache under the classification of the big file of resource suffix or the big file of resource
Optimization；
Step 305, the optimization caching rule that will cache the output of rule generation component is applied in existing network；
Specifically, stored in caching server with the entitled index of URL/domain, it is slow with the corresponding optimizations of the URL
Deposit the corresponding relation that regular normal form is index content.
Further, in a kind of optional implementation method of the present invention, in order to the knot to cache resources analysis is presented
Really, above-mentioned steps 303 are analyzed to any log cache for analyzing domain name and also include：
Resource analysis and data statistics are carried out to the optimized log cache in each classification, such as, and domain name life
In total flow, analyzable total flow, domain name hit rate, and flow accounting can be analyzed.
Wherein, for the optimized log cache quantity with hit mark (TCP_HIT) in each classification
Counted, obtained the total flow of each classification hit.Can also be according to the domain name to be analyzed counted before classification
Total flow and the hit of each classification total flow, calculate the hit rate of each classification.Wherein, system before classifying
The total flow of the domain name to be analyzed of meter refers to the resource of whole log caches corresponding with the domain name to be analyzed
The superposition value of sizes values.
Resource size value for the optimized log cache in each classification is overlapped, and obtains this class
Analyzable total flow in not.Can also be according to the total flow of the domain name to be analyzed of statistics before classification and each
Analyzable total flow in classification, calculates the flow the analyzed accounting of each classification.
Further, in a kind of optional implementation method of the present invention, pair can optimize in order to clearly present
The effect of optimization of the caching rule of log cache, above-mentioned Cache cache optimizations instrument also includes clone's collection mould
Block, after being reached the standard grade for the caching rule after above-mentioned optimization in a period of time, performs above-mentioned steps 302, adopts
Collect the log cache of same domain name.
According to step 303 and step 304, resource analysis and data statistics are carried out to freshly harvested log cache,
The statistics of same category log cache is contrasted and presented with the statistics before reaching the standard grade, for example,
All to domain name for the domain name hit rate of the big file class of resource suffix of " ykimg.com " is contrasted, can
It is 10% to obtain domain name hit rate of the big file class of domain name resources suffix before rule optimization is cached, should
Domain name hit rate of the big file class of domain name resources suffix after rule optimization is cached is up to 70%.
In above-described embodiment, resource analysis processing module can be according to the ruuning situation of domain name of having reached the standard grade, to hit
Mark, HTTP conditional codes, resource size, resource URL, whether hit the data such as rule and carry out statistical
Analysis；Rule generation component is cached according to the resource depth and designated domain name that can optimize log cache URL,
Automatically generate correspondence caching rule；Certain domain name is analyzed, and according to different resource URL depth point
Also more targetedly, each cached parameters setting is also more reasonable, subsequently for the regular expression normal form not generated
When caching rule according to optimization is cached, domain name can be shortened long with the match time of the caching rule of optimization
Degree, can effectively lift buffer efficiency.
By the domain name hit situation before and after the new caching rule application of contrast, for operator provides more resources
Statistical information.
The Cache cache optimizations instrument of the embodiment of the present invention simultaneously completes to dock with existing network caching server, is lifted
In system while the utilization rate of log cache, it is capable of achieving to carry out specified URL using log cache resource
Count, analyze, caching rule and caching effect of optimization comparing function being automatically generated, using the instrument, operation
Business can independently realize Cache system operations, greatly promote cache optimization operating efficiency.
For above method flow, the embodiment of the present invention additionally provides a kind of caching resource device, these devices
Particular content referring to above method flow, be not repeated herein.
A kind of caching resource device as shown in Figure 4, including：
First acquisition unit 401, for obtaining domain name to be analyzed；
Second acquisition unit 402, specifies in the time period this to treat point for for any domain name to be analyzed, obtaining
The corresponding log cache of analysis domain name；And
Resource analysis unit 403, for extracting first from the corresponding any log cache of the domain name to be analyzed
Class key message；And determine whether the log cache is that can optimize caching day according at least to first kind key message
Will；
Rule generating unit 404, for if it is determined that the log cache then delays this for that can optimize log cache
Deposit the domain name field information input of URL in daily record to the corresponding canonical table of resource depth levels of the URL
Up in formula, the caching rule of the log cache is generated；
Wherein, the corresponding regular expression of resource depth levels of the URL is the money according at least to the URL
The regular expression that cached parameters corresponding to Depth grade and the resource depth levels are write in advance.
Wherein, the function phase of the input interface in first acquisition unit 401 and above-described embodiment is same；Second obtains
The function phase for taking the specified interface of caching server in unit 402 and above-described embodiment is same；Resource analysis list
Unit 403 with above-described embodiment in resource analysis processing module function phase it is same；Rule generating unit 404 with
The function phase of the caching rule generation component in above-described embodiment is same.
Further, second acquisition unit 402 specifically for：
By the specified interface of caching server, read and specify in the time period the corresponding caching of the domain name to be analyzed
Daily record.
Further, the first key message at least includes HTTP conditional codes；
Resource analysis unit 403 specifically for：
If the HTTP conditional codes in first kind key message are to specify HTTP conditional codes and the crucial letter of the first kind
Do not include the matching identification information of the log cache and general caching rule match in breath, then by the log cache
It is defined as that log cache can be optimized.
Further, resource analysis unit 403 is additionally operable to：
If it is determined that the log cache is for that can optimize log cache, then Equations of The Second Kind is extracted from the log cache crucial
Information, Equations of The Second Kind key message at least includes resource suffix identification information and resource size information；
If the resource suffix identification information of the log cache belongs to the identity type of the big file of resource suffix, will
The log cache is categorized into the classification of the big file of resource suffix；Or,
If the resource suffix identification information of the log cache belongs to the identity type of resource suffix small documents, will
The log cache is categorized into the classification of resource suffix small documents；Or,
If the resource size information of the log cache meets the threshold range of large resource file, by the caching day
Will is categorized into the classification of large resource file；Or,
If the resource size information of the log cache meets the threshold range of small resource file, by the caching day
Will is categorized into the classification of small resource file；
Then rule generating unit 404 specifically for：
By the money of the domain name field information input of the URL in any log cache in each classification to the URL
In the corresponding regular expression of Depth grade, the caching rule of the log cache is generated, and stored at such
Not in.
Further, rule generating unit 404 specifically for：
By resource suffix up to the domain of the URL in any log cache in the classification of file and/or large resource file
File-name field information input generates the caching in the corresponding regular expression of resource depth levels of the URL
The caching rule of daily record, and store in the category.
In above-described embodiment, obtain and specify the corresponding log cache of domain name to be analyzed in the time period, treated point from this
First kind key message, such as hit mark, HTTP states are extracted in the corresponding any log cache of analysis domain name
Code, resource size, resource URL, whether with the data such as general caching rule match, and according at least to first
Class key message determines whether the log cache is that can optimize log cache；Further basis can optimize caching day
The resource depth and designated domain name of will URL, automatically generate correspondence caching rule；Certain domain name is divided
Analysis, and it is also more targeted according to the regular expression normal form that different resource URL depth is generated respectively,
Each cached parameters setting is also more reasonable, when subsequently being cached according to the caching rule of optimization, can shorten domain
Name and the match time length of the caching rule of optimization, can effectively lift buffer efficiency.
The present invention is produced with reference to method according to embodiments of the present invention, equipment (system) and computer program
The flow chart and/or block diagram of product is described.It should be understood that can by computer program instructions realize flow chart and
/ or block diagram in each flow and/or the flow in square frame and flow chart and/or block diagram and/
Or the combination of square frame.These computer program instructions to all-purpose computer, special-purpose computer, insertion can be provided
The processor of formula processor or other programmable data processing devices is producing a machine so that by calculating
The instruction of the computing device of machine or other programmable data processing devices is produced for realizing in flow chart one
The device of the function of being specified in individual flow or multiple one square frame of flow and/or block diagram or multiple square frames.
These computer program instructions may be alternatively stored in can guide computer or the treatment of other programmable datas to set
In the standby computer-readable memory for working in a specific way so that storage is in the computer-readable memory
Instruction produce include the manufacture of command device, the command device realization in one flow of flow chart or multiple
The function of being specified in one square frame of flow and/or block diagram or multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices, made
Obtain and series of operation steps is performed on computer or other programmable devices to produce computer implemented place
Reason, so as to the instruction performed on computer or other programmable devices is provided for realizing in flow chart one
The step of function of being specified in flow or multiple one square frame of flow and/or block diagram or multiple square frames.
, but those skilled in the art once know base although preferred embodiments of the present invention have been described
This creative concept, then can make other change and modification to these embodiments.So, appended right will
Ask and be intended to be construed to include preferred embodiment and fall into having altered and changing for the scope of the invention.
Obviously, those skilled in the art can carry out various changes and modification without deviating from this hair to the present invention
Bright spirit and scope.So, if it is of the invention these modification and modification belong to the claims in the present invention and
Within the scope of its equivalent technologies, then the present invention is also intended to comprising these changes and modification.
Claims (10)
1. a kind of resource caching method, it is characterised in that including：
Obtain domain name to be analyzed；
For any domain name to be analyzed, obtain and specify in the time period the corresponding log cache of the domain name to be analyzed；
And
First kind key message is extracted from the corresponding any log cache of the domain name to be analyzed；
Determine whether the log cache is that can optimize log cache according at least to the first kind key message；
If it is determined that the log cache is for that can optimize log cache, then the unified resource in the log cache is positioned
The domain name field information input of device URL in the corresponding regular expression of resource depth levels of the URL,
Generate the caching rule of the log cache；
Wherein, the corresponding regular expression of resource depth levels of the URL is the money according at least to the URL
The regular expression that cached parameters corresponding to Depth grade and the resource depth levels are write in advance.
2. the method for claim 1, it is characterised in that
It is described for any domain name to be analyzed, obtain and specify in the time period domain name to be analyzed corresponding caching day
Will, including：
By the specified interface of caching server, read and specify in the time period the corresponding caching of the domain name to be analyzed
Daily record.
3. the method for claim 1, it is characterised in that first key message at least includes
HTTP HTTP conditional codes；
It is described according at least to the first kind key message, determine the log cache whether be can optimize caching day
Will resource, including：
If the HTTP conditional codes in the first kind key message are to specify HTTP conditional codes and described first
Do not include the matching identification information of the log cache and general caching rule match in class key message, then should
Log cache is defined as that log cache can be optimized.
4. the method for claim 1, it is characterised in that if it is determined that the log cache is can to optimize
Log cache, then also include：
Equations of The Second Kind key message is extracted from the log cache, the Equations of The Second Kind key message at least includes resource
Suffix identification information and resource size information；
If the resource suffix identification information of the log cache belongs to the identity type of the big file of resource suffix, will
The log cache is categorized into the classification of the big file of resource suffix；Or,
If the resource suffix identification information of the log cache belongs to the identity type of resource suffix small documents, will
The log cache is categorized into the classification of the resource suffix small documents；Or,
If the resource size information of the log cache meets the threshold range of large resource file, by the caching day
Will is categorized into the classification of the large resource file；Or,
If the resource size information of the log cache meets the threshold range of small resource file, by the caching day
Will is categorized into the classification of the small resource file；
Then resource depth of the domain name field information input of the URL by the log cache to the URL
In the corresponding regular expression of grade, the caching rule of the log cache is generated, including：
By the money of the domain name field information input of the URL in any log cache in each classification to the URL
In the corresponding regular expression of Depth grade, the caching rule of the log cache is generated, and stored at such
Not in.
5. method as claimed in claim 4, it is characterised in that the URL by the log cache
Domain name field information input in the corresponding regular expression of resource depth levels of the URL, generation should
The caching rule of log cache, including：
By the resource suffix up in any log cache in the classification of file and/or the large resource file
The domain name field information input of URL is raw in the corresponding regular expression of resource depth levels of the URL
Into the caching rule of the log cache, and store in the category.
6. a kind of caching resource device, it is characterised in that including：
First acquisition unit, for obtaining domain name to be analyzed；
Second acquisition unit, for for any domain name to be analyzed, obtaining and specifying the domain to be analyzed in the time period
The corresponding log cache of name；And
Resource analysis unit, closes for extracting the first kind from the corresponding any log cache of the domain name to be analyzed
Key information；And determine whether the log cache is that can optimize caching day according at least to the first kind key message
Will；
Rule generating unit, for if it is determined that the log cache is can to optimize log cache, then by the caching day
Resource depth levels pair of the domain name field information input of the uniform resource locator URL in will to the URL
In the regular expression answered, the caching rule of the log cache is generated；
Wherein, the corresponding regular expression of resource depth levels of the URL is the money according at least to the URL
The regular expression that cached parameters corresponding to Depth grade and the resource depth levels are write in advance.
7. device as claimed in claim 6, it is characterised in that
The second acquisition unit specifically for：
By the specified interface of caching server, read and specify in the time period the corresponding caching of the domain name to be analyzed
Daily record.
8. device as claimed in claim 6, it is characterised in that first key message at least includes
HTTP conditional codes；
The resource analysis unit specifically for：
If the HTTP HTTP conditional codes in the first kind key message are to specify HTTP shapes
Do not include matching for the log cache and general caching rule match in state code and the first kind key message
, then be defined as the log cache that log cache can be optimized by identification information.
9. device as claimed in claim 6, it is characterised in that the resource analysis unit is additionally operable to：
If it is determined that the log cache is for that can optimize log cache, then Equations of The Second Kind is extracted from the log cache crucial
Information, the Equations of The Second Kind key message at least includes resource suffix identification information and resource size information；
If the resource suffix identification information of the log cache belongs to the identity type of the big file of resource suffix, will
The log cache is categorized into the classification of the big file of resource suffix；Or,
If the resource suffix identification information of the log cache belongs to the identity type of resource suffix small documents, will
The log cache is categorized into the classification of the resource suffix small documents；Or,
If the resource size information of the log cache meets the threshold range of large resource file, by the caching day
Will is categorized into the classification of the large resource file；Or,
If the resource size information of the log cache meets the threshold range of small resource file, by the caching day
Will is categorized into the classification of the small resource file；
Then the rule generating unit specifically for：
By the money of the domain name field information input of the URL in any log cache in each classification to the URL
In the corresponding regular expression of Depth grade, the caching rule of the log cache is generated, and stored at such
Not in.
10. device as claimed in claim 9, it is characterised in that the rule generating unit specifically for：
By the resource suffix up in any log cache in the classification of file and/or the large resource file
The domain name field information input of URL is raw in the corresponding regular expression of resource depth levels of the URL
Into the caching rule of the log cache, and store in the category.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510999566.4A CN106921713B (en) | 2015-12-25 | 2015-12-25 | Resource caching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510999566.4A CN106921713B (en) | 2015-12-25 | 2015-12-25 | Resource caching method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106921713A true CN106921713A (en) | 2017-07-04 |
| CN106921713B CN106921713B (en) | 2019-12-06 |
Family
ID=59456083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510999566.4A Active CN106921713B (en) | 2015-12-25 | 2015-12-25 | Resource caching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106921713B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107819837A (en) * | 2017-10-31 | 2018-03-20 | 南京优速网络科技有限公司 | A kind of method and log cache analysis system for lifting buffer service quality |
| CN108704312A (en) * | 2018-04-26 | 2018-10-26 | 网易（杭州）网络有限公司 | The test method and device of fine arts resource |
| CN109145220A (en) * | 2018-09-10 | 2019-01-04 | 北京知道创宇信息技术有限公司 | Data processing method, device and electronic equipment |
| CN109586937A (en) * | 2017-09-28 | 2019-04-05 | 中兴通讯股份有限公司 | A kind of O&M method, equipment and the storage medium of caching system |
| CN110020249A (en) * | 2017-12-28 | 2019-07-16 | 中国移动通信集团山东有限公司 | A kind of caching method, device and the electronic equipment of URL resource |
| CN110401553A (en) * | 2018-04-25 | 2019-11-01 | 阿里巴巴集团控股有限公司 | The method and apparatus of server configuration |
| CN110677270A (en) * | 2018-07-03 | 2020-01-10 | 长春亿阳计算机开发有限公司 | Domain name cacheability analysis method and system |
| WO2022152086A1 (en) * | 2021-01-15 | 2022-07-21 | 华为云计算技术有限公司 | Data caching method and apparatus, and device and computer-readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103825919A (en) * | 2012-11-16 | 2014-05-28 | 中国移动通信集团北京有限公司 | Method, device and system for data resource caching |
| CN104010010A (en) * | 2013-02-25 | 2014-08-27 | 中国移动通信集团北京有限公司 | Internet resource acquisition method, device and cache system |
| CN104079534A (en) * | 2013-03-27 | 2014-10-01 | 中国移动通信集团北京有限公司 | Method and system of implementing HTTP (Hyper Text Transport Protocol) cache |
| CN104111900A (en) * | 2013-04-22 | 2014-10-22 | 中国移动通信集团公司 | Method and device for replacing data in cache |
| CN104426838A (en) * | 2013-08-20 | 2015-03-18 | 中国移动通信集团北京有限公司 | Internet cache scheduling method and system |
-
2015
- 2015-12-25 CN CN201510999566.4A patent/CN106921713B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103825919A (en) * | 2012-11-16 | 2014-05-28 | 中国移动通信集团北京有限公司 | Method, device and system for data resource caching |
| CN104010010A (en) * | 2013-02-25 | 2014-08-27 | 中国移动通信集团北京有限公司 | Internet resource acquisition method, device and cache system |
| CN104079534A (en) * | 2013-03-27 | 2014-10-01 | 中国移动通信集团北京有限公司 | Method and system of implementing HTTP (Hyper Text Transport Protocol) cache |
| CN104111900A (en) * | 2013-04-22 | 2014-10-22 | 中国移动通信集团公司 | Method and device for replacing data in cache |
| CN104426838A (en) * | 2013-08-20 | 2015-03-18 | 中国移动通信集团北京有限公司 | Internet cache scheduling method and system |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109586937A (en) * | 2017-09-28 | 2019-04-05 | 中兴通讯股份有限公司 | A kind of O&M method, equipment and the storage medium of caching system |
| CN107819837A (en) * | 2017-10-31 | 2018-03-20 | 南京优速网络科技有限公司 | A kind of method and log cache analysis system for lifting buffer service quality |
| CN110020249A (en) * | 2017-12-28 | 2019-07-16 | 中国移动通信集团山东有限公司 | A kind of caching method, device and the electronic equipment of URL resource |
| CN110020249B (en) * | 2017-12-28 | 2021-11-30 | 中国移动通信集团山东有限公司 | URL resource caching method and device and electronic equipment |
| CN110401553B (en) * | 2018-04-25 | 2022-06-03 | 阿里巴巴集团控股有限公司 | Server configuration method and device |
| CN110401553A (en) * | 2018-04-25 | 2019-11-01 | 阿里巴巴集团控股有限公司 | The method and apparatus of server configuration |
| US11431669B2 (en) | 2018-04-25 | 2022-08-30 | Alibaba Group Holding Limited | Server configuration method and apparatus |
| CN108704312A (en) * | 2018-04-26 | 2018-10-26 | 网易（杭州）网络有限公司 | The test method and device of fine arts resource |
| CN110677270A (en) * | 2018-07-03 | 2020-01-10 | 长春亿阳计算机开发有限公司 | Domain name cacheability analysis method and system |
| CN110677270B (en) * | 2018-07-03 | 2023-02-28 | 长春亿阳计算机开发有限公司 | Domain name cacheability analysis method and system |
| CN109145220A (en) * | 2018-09-10 | 2019-01-04 | 北京知道创宇信息技术有限公司 | Data processing method, device and electronic equipment |
| CN109145220B (en) * | 2018-09-10 | 2022-03-29 | 北京知道创宇信息技术股份有限公司 | Data processing method and device and electronic equipment |
| WO2022152086A1 (en) * | 2021-01-15 | 2022-07-21 | 华为云计算技术有限公司 | Data caching method and apparatus, and device and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106921713B (en) | 2019-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106921713A (en) | A kind of resource caching method and device | |
| CN105357054B (en) | Website traffic analysis method, device and electronic equipment | |
| CN104869009B (en) | The system and method for website data statistics | |
| CN109118296A (en) | Movable method for pushing, device and electronic equipment | |
| CN106651416A (en) | Analyzing method and analyzing device of application popularization information | |
| CN103729385B (en) | Method and device for automatically updating reports | |
| CN109254901B (en) | A kind of Monitoring Indexes method and system | |
| CN107145556B (en) | Universal distributed acquisition system | |
| CN105871919A (en) | Network application firewall system and realization method thereof | |
| CN106331172A (en) | Method and device for detecting resources for content distribution network | |
| US20190197140A1 (en) | Automation of sql tuning method and system using statistic sql pattern analysis | |
| CN111131070B (en) | Port time sequence-based network traffic classification method and device and storage medium | |
| CN110737645B (en) | Data migration method and system among different systems and related equipment | |
| CN109275045A (en) | Mobile terminal encrypted video ad traffic recognition methods based on DFI | |
| CN111898036A (en) | Behavior data collecting and processing system and method | |
| CN107229628A (en) | The method and device of distributed data base pretreatment | |
| CN109586937A (en) | A kind of O&M method, equipment and the storage medium of caching system | |
| CN106897313B (en) | Mass user service preference evaluation method and device | |
| Liu et al. | Request dependency graph: A model for web usage mining in large-scale web of things | |
| CN108287874B (en) | DB2 database management method and device | |
| CN111538881B (en) | Activity analysis method, equipment and storage medium based on behavior data | |
| CN109033330A (en) | Big data cleaning method, device and server | |
| CN104539452B (en) | A kind of method that statistics Web applications access regional characteristic | |
| CN110489569B (en) | Event processing method and device based on knowledge graph | |
| CN106713374A (en) | DNS-based traffic analysis and optimal traffic scheduling system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |