CN106657196B - Cache content elimination method and cache device - Google Patents

Cache content elimination method and cache device Download PDF

Info

Publication number
CN106657196B
CN106657196B CN201510736056.8A CN201510736056A CN106657196B CN 106657196 B CN106657196 B CN 106657196B CN 201510736056 A CN201510736056 A CN 201510736056A CN 106657196 B CN106657196 B CN 106657196B
Authority
CN
China
Prior art keywords
content
hot
condition
elimination
eliminated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510736056.8A
Other languages
Chinese (zh)
Other versions
CN106657196A (en
Inventor
王剑桥
徐保磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
XFusion Digital Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201510736056.8A priority Critical patent/CN106657196B/en
Publication of CN106657196A publication Critical patent/CN106657196A/en
Application granted granted Critical
Publication of CN106657196B publication Critical patent/CN106657196B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682Policies or rules for updating, deleting or replacing the stored data

Abstract

The embodiment of the invention provides a cache content elimination method and a cache device, relates to the technical field of communication, and can solve the problem that in the prior art, because the historical access amount is high, the content with the current or future access amount possibly low occupies the cache space of a CDN edge node for a long time, the cache space is insufficient. The specific scheme is as follows: the CDN edge node divides the content in the cache into hot content and non-hot content according to the historical access amount, the historical access amount of any hot content is higher than that of any non-hot content, when it is determined that a content elimination process in the cache needs to be started in a current time window, if it is determined that a first content set meeting a first elimination condition exists in the hot content, the to-be-eliminated content in the first content set is eliminated from the hot content, the to-be-eliminated content comprises at least one content, and the first elimination condition is used for determining that the access amount of the content in the hot content is in a descending trend. The embodiment of the invention is used for eliminating the content in the cache.

Description

Cache content elimination method and cache device
Technical Field
The embodiment of the invention relates to the technical field of communication, in particular to a cache content elimination method and a cache device.
Background
When a Content Delivery Network (CDN) edge node caches Content in a CDN system, because the cache space of the CDN edge node is limited, when a stored resource already occupies an excessive cache space of the CDN edge node, some data needs to be eliminated to ensure that there is enough cache space to store new data.
For The video acceleration service CDN, The video with The highest historical visit amount is ranked according to The height of The historical visit amount (L east frequencytly Used, L FU) adopted by The prior art for eliminating The video in The CDN edge node cache, and referring to FIG. 1, The video with The highest historical visit amount is ranked at The head of a L FU queue, and The future visit amount of The historical popular video with The higher historical visit amount is also higher, and The future visit amount of The historical popular video with The lower historical visit amount is also lower, so that The video with The lowest historical visit amount at The tail of a FU L queue is eliminated.
In the above-mentioned elimination mechanism, since videos with high historical access amount are easy to be kept at the front part of the queue, even if the videos are not accessed any more or are accessed very little in the current or future, a long time is required to move to the tail part of the queue, and then the cache is eliminated. In the prior art, videos with the lowest historical access amount in a cache are eliminated, so that the elimination speed of the videos with high historical access amount is low, and the videos with high historical access amount but possibly low current or future access amount cannot be eliminated in time, so that the cache space of a CDN edge node is occupied for a long time, and the cache space is insufficient.
Disclosure of Invention
The embodiment of the invention provides a cache content elimination method and a cache device, which can solve the problem that in the prior art, because the historical access amount is high, the content with the current or future access amount possibly low occupies the cache space of a CDN edge node for a long time, so that the cache space is insufficient.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
the CDN edge node divides the content in the cache into hot content with higher historical access amount and non-hot content with lower historical access amount, and when the cache content elimination process needs to be started, the access amount of the hot content with higher historical access amount is in a descending trend, so that at least one content with possibly lower current or future access amount is eliminated from the hot content, the release speed of the cache space occupied by the hot content with high historical access amount but possibly lower current or future access amount is accelerated, and the cache space is prevented from being occupied for a long time.
With reference to the first aspect, in a first possible implementation manner of the first aspect, when a cache content elimination process needs to be started, the CDN edge node eliminates, from the hot content, at least one content with a lower historical access amount, from the hot content, where the access amount is in a downward trend.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, when a cache content elimination process needs to be started, the CDN edge node directly eliminates, from the cache, content with a sharply decreasing visit rate and a low historical visit rate in the popular content; the method comprises the steps of eliminating the contents with the access quantity in the hot contents which is in the descending trend but not necessarily in the sharp descending trend from the hot contents to the non-hot contents instead of directly eliminating the cache, so that the elimination speed of the hot contents with the access quantity in the descending trend is accelerated, meanwhile, the misoperation that the hot contents with high historical access quantity and the access quantity in the slow descending trend are directly eliminated from the cache is reduced, and the hit rate of the cache is improved.
With reference to the first aspect to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, when a content elimination procedure needs to be started, if there is no content with a downward access amount in the trending content, the CDN edge node eliminates at least one content with a low historical access amount or a long interval between the latest access time and the current time from the cache in the non-trending content.
In a second aspect, a cache apparatus is provided, in which after a dividing unit divides contents in a cache into hot contents with a higher historical access amount and non-hot contents with a lower historical access amount, when a cache content elimination flow needs to be started, an elimination unit eliminates at least one content with a lower current or future access amount from the hot contents with the higher historical access amount by making the access amount of the hot contents with the higher historical access amount in a descending trend.
With reference to the second aspect, in a first possible implementation manner of the second aspect, when a cache content elimination process needs to be started, the elimination unit eliminates at least one content with a lower historical access amount from the hot contents with a decreasing access amount.
With reference to the first possible implementation manner of the second aspect, in a second possible implementation manner of the first aspect, when a cache content elimination flow needs to be started, the elimination unit directly eliminates, from the cache, content with a sharply decreasing visit rate and a low historical visit rate in the hot content; and eliminating the content with the access quantity which is in a descending trend but not necessarily in a sharp descending trend in the hot content from the hot content to the non-hot content instead of directly eliminating the cache.
With reference to the second aspect to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, when a content elimination procedure needs to be started, if there is no content with a downward access amount in the trending content, the elimination unit eliminates at least one content with a lower historical access amount or a longer interval between the latest access time and the current time from the cache in the non-trending content.
For ease of understanding, some of the illustrations of concepts related to the present invention are given as examples for reference, as follows:
caching: the storage space of the CND edge node may be, for example, a storage for high-speed data exchange, and the operation speed of the storage is higher than that of a memory.
Time window: the CDN edge node periodically divides the content in the cache into hot content and non-hot content according to historical access amount, wherein a time period between any two times of division is called a time window.
The current time window: and after the time when the CDN edge node divides the content in the cache into the hot content and the non-hot content according to the historical access amount last time, and the time period before the time when the content in the cache is divided into the hot content and the non-hot content according to the historical access amount next time.
Dividing time: and the CDN edge node divides the content in the cache into hot content and non-hot content according to the historical access amount.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a diagram of an L FU queue structure provided in the prior art;
FIG. 2 is a system architecture diagram according to an embodiment of the present invention;
fig. 3a is a schematic flow chart of a cache content elimination method according to an embodiment of the present invention;
fig. 3b is a schematic flow chart of another cache content elimination method according to an embodiment of the present invention;
fig. 3c is a schematic flow chart of another cache content elimination method according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of another cache content elimination method according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating a method for determining whether to evict a video in a cache according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating an arrangement structure of a hot video according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating a method for determining a cold door code rate according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating a method for determining a cold timestamp according to an embodiment of the present invention;
fig. 9 is a schematic flow chart of another cache content elimination method according to an embodiment of the present invention;
fig. 10 is a schematic flow chart illustrating a cache content eviction method according to an embodiment of the present invention;
fig. 11 is a schematic diagram illustrating a cache content eviction process according to an embodiment of the present invention;
fig. 12a is a schematic structural diagram of a cache apparatus according to an embodiment of the present invention;
fig. 12b is a schematic structural diagram of another cache apparatus according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of another cache apparatus according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
Referring to the system architecture diagram shown in fig. 2, when a CDN user accesses content, a domain name of the content to be accessed is provided by a client browser, and a request for resolving the domain name is sent to a CDN center node. After receiving the domain name resolution request, the CDN central node intelligently resolves the domain name through a domain name resolution and load balancing system, and sends an Internet Protocol (IP) address of a CDN edge node closest to the user to the client. And after receiving the IP address, the browser of the client sends a content access request to the corresponding CDN edge node according to the IP address. If the content to be accessed requested by the client is stored in the cache of the CDN edge node, the content to be accessed is sent to the client, and if the content to be accessed requested by the client is not stored in the cache of the CDN edge node, a content acquisition request is sent to a CDN central node or a content source server to acquire the content to be accessed, and the acquired content to be accessed is sent to the client.
According to the CDN user access process, when the cache of the CDN edge node contains the content to be accessed requested by the client, the CDN edge node can directly return the content to be accessed to the client without returning to the source, so that the time of user request response can be shortened, and the user experience is improved. Because the CDN edge node has a limited storage space, the content stored in the cache needs to be dynamically adjusted according to a certain elimination policy, so that the content frequently accessed by the client is stored in the CDN edge node as much as possible.
As described in the background art, the elimination policy adopted in the prior art causes high historical access amount, but content with possibly low current or future access amount occupies the cache space of the CDN edge node for a long time, so that the cache space is insufficient, and it is difficult to store content that the client accesses currently or will often in the future. To solve the problem, the following embodiments of the present invention mainly divide the content stored in the cache of the cache device into the hot content with high historical access amount and the non-hot content with low historical access amount according to the historical access amount, so that when a cache content elimination process needs to be started within a current time window, at least one content with a downward access amount in the hot content is eliminated as the hot content, so as to solve the problem in the prior art that the content with high historical access amount but possibly low current or future access amount occupies the cache space of the cache device for a long time, which results in insufficient cache space.
The system architecture and the service scenario described in fig. 2 are for more clearly illustrating the technical solution of the embodiment of the present invention, and do not form a limitation on the technical solution provided by the embodiment of the present invention, and it can be known by those skilled in the art that the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems along with the evolution of the network architecture and the appearance of a new service scenario.
The following will be described in detail by way of specific examples.
Example 1
The embodiment of the invention provides a cache content elimination method, which can be any node in a CDN system, such as a CDN edge node, and is introduced by taking the CDN edge node as an example. Referring to fig. 3a, may include:
201. and the CDN edge node divides the content in the cache into hot content and non-hot content according to the historical access amount, wherein the historical access amount of any hot content is higher than that of any non-hot content.
The cache here refers to a data exchange buffer of the CDN edge node. The content in the cache may include various file forms, which are not specifically limited herein, and may be, for example, video, audio, and the like. The CDN edge node may divide the content stored in the cache into top content with a higher historical access amount and non-top content with a lower historical access amount according to the historical access amount, so that the historical access amount of any one top streaming media file is higher than the historical access amount of any one non-top streaming media file, so that when a content elimination flow is started, the content in the top content may be directly eliminated from the cache, or the content in the top content is eliminated to the non-top content, and the content in the non-top content is eliminated from the cache.
For example, if the predetermined number is 30 ten thousand, 30 ten thousand streaming media files with higher historical access amount (60 ten thousand × 50% is 30 ten thousand) can be divided into hot streaming media files, and if the predetermined ratio is 50%, 60 ten thousand streaming media files in the cache of the CDN edge node are totally included in the cache of the CDN edge node, and the remaining 30 ten thousand streaming media files are not limited to hot streaming media files.
202. When the CDN edge node determines that a content elimination process in the cache needs to be started within a current time window, if a first content set meeting a first elimination condition exists in hot content, eliminating the to-be-eliminated content in the first content set from the hot content, wherein the to-be-eliminated content comprises at least one content, and the first elimination condition is used for determining that the access amount of the content in the hot content is in a descending trend.
The determining, by the CDN edge node, that a content elimination process in the cache needs to be started within the current time window may specifically include:
the CDN edge node receives an access request which is sent by a client and used for acquiring the content to be accessed;
the CDN edge node determines that the content to be accessed is not stored in a cache;
when the content to be accessed meets the caching condition, the CDN edge node determines that the residual caching space is smaller than the file size of the content to be accessed;
the caching conditions may include: the historical access amount of the content to be accessed is higher than or equal to a first preset threshold value.
In addition, if the remaining cache space is greater than or equal to the file size of the content to be accessed, the CDN edge node may determine that a content elimination procedure in the cache does not need to be started within the current time window, insert the content to be accessed into the non-popular content, and send the content to be accessed to the client.
When it is determined that the content elimination flow in the cache needs to be started within the current time window, the CDN edge node starts entering the elimination flow to eliminate the content in the cache.
Specifically, the first elimination condition in step 202 may include the following first elimination sub-conditions:
the access amount of the content in the hot content in the current time window is lower than the preset access amount.
Since the preset access amount is an expected access amount which can be achieved in the current time window estimated according to the access conditions of the content in the previous time windows, if the access amount of the content in the hot content in the current time window is lower than the preset access amount, it can be stated that the current access amount of the content is lower and is in a descending trend.
In addition, the first knockout condition can further comprise a second knockout sub-condition, and/or a third knockout sub-condition, and/or a fourth knockout sub-condition;
the second knockout sub-condition comprises: the current time is not in a preset protection period of the content in the hot content;
the third knockout sub-condition comprises: the number of the eliminated cached contents in the hot contents in the current time window is lower than the preset elimination number;
the fourth knockout sub-condition comprises: the ratio of the number of the hot contents with the cache eliminated to the number of the non-hot contents with the cache eliminated in the current time window is lower than the preset elimination ratio.
It should be noted that the specific contents of the first elimination condition for determining that the access amount of the content in the popular content is in the descending trend are only examples, and are not used to limit the scope of the first elimination condition. When the CDN edge node determines that a first content set meeting a first elimination condition exists in the hot content, it may be indicated that the access amount of the content in the first content set is in a downward trend, and the current or future access amount of the content in the first content set may be lower, so that the content to be eliminated in the first content set may be eliminated from the hot content, thereby accelerating the elimination speed of the content with high historical access amount but possibly lower current or future access amount, and avoiding insufficient cache space due to the long-term occupancy of the cache space of the CDN edge node.
It is emphasized that the to-be-eliminated content may be at least one content in the first content set, the to-be-eliminated content is a subset of the first content set, and the contents in the first content set are not necessarily all to-be-eliminated contents. The content range included in the to-be-eliminated content can be set according to actual needs, and is not specifically limited here.
Since the historical access amount also reflects the hot degree of the content to a certain extent, the future access amount of the content with low historical access amount is more likely to be low, and the future access amount of the content with high historical access amount is more likely to be high, at least one content with low historical access amount in the first content set can be preferentially eliminated from the hot content as the to-be-eliminated content, so as to reduce the possibility that the content with high historical access amount and possibly high future access amount is mistakenly eliminated.
For example, the content to be eliminated may be the content with the lowest historical access in the first set of content.
For another example, when the sum of the file sizes of all the contents in the first content set is smaller than or equal to the file size of the content to be accessed, the content to be eliminated may be all the contents in the first content set, and in addition, the content whose file size or the sum of the file sizes in the non-hotline queue is larger than or equal to a first byte number may be eliminated from the cache, where the first byte number is a difference value between the sum of the file sizes of the content to be eliminated and the file size of the content to be accessed; when the sum of the file sizes of all the contents in the first content set is larger than the file size of the contents to be accessed, the obsolete contents may be part of the first content set, for example, one of the contents in the first content set whose file size is larger than or equal to the file size of the contents to be accessed and whose historical access amount is lower, or a plurality of contents whose sum of the file sizes is larger than or equal to the file size of the contents to be accessed and whose historical access amount is lower, so that enough cache space may be made available to store the contents to be accessed.
When the CDN edge node eliminates the to-be-eliminated content in the first content set from the trending content in step 202, it may determine to eliminate the to-be-eliminated content from the trending content into the non-trending content according to the second elimination condition, or directly eliminate the cache from the trending content. The method specifically comprises the following steps:
the CDN edge node determines whether the content in the content to be eliminated meets a second elimination condition or not;
the CDN edge node eliminates the contents meeting the second elimination condition from the contents to be eliminated out of the cache;
and the CDN edge node eliminates the contents which do not meet the second elimination condition in the contents to be eliminated from the hot contents into the non-hot contents, and eliminates at least one content in the non-hot contents out of the cache.
And the second elimination condition is used for determining that the access quantity of the content in the content to be eliminated is in a sharp descending trend. The second knockout condition can include at least one of a fifth knockout sub-condition, a sixth knockout sub-condition, and a seventh knockout sub-condition.
Wherein the fifth eliminating sub-condition comprises: the file type of the content in the to-be-eliminated content is a preset file type;
the sixth knockout sub-condition comprises: the code rate of the content in the to-be-eliminated content is a cold door code rate value;
the seventh knockout sub-condition comprises: and the time stamp of the content in the to-be-eliminated content is the cold time stamp.
The preset file type in the second elimination condition can be set to be a type with strong timeliness according to needs, after a certain time, the access amount of the content will be remarkably reduced, the cold threshold rate is a code rate which is counted out according to the historical access condition of the content and is rarely used by the client, and the cold threshold timestamp is a timestamp which is counted out according to the historical access condition of the content and is rarely used by the client, so that the access amount of the content to be eliminated meeting the second elimination condition may be in a sharp decline trend, namely the current or future access amount of the part of the content to be eliminated may be particularly low, and therefore the CDN edge node can directly eliminate the content to be eliminated meeting the second elimination condition for caching. According to the cache content elimination method provided by the embodiment of the invention, the part of the contents to be eliminated, of the hot contents with higher historical access amount and with the sharply decreased access amount, is directly eliminated from the cache, so that the release speed of the cache space occupied by the hot contents with high historical access amount but sharply decreased current or future access amount is increased, and the cache space of the CDN edge node is prevented from being occupied for a long time.
If the second elimination condition at least includes the fifth elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method may further include:
acquiring a uniform resource locator UR L of the content in the to-be-eliminated content;
and requesting a CDN center node to acquire a type field for describing the file type of the content in the to-be-eliminated content according to UR L.
If the second elimination condition includes at least a sixth elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method may further include:
acquiring code rate values carried in UR L L corresponding to all accessed contents in an access log;
and determining that the code rate value with the access quantity lower than a second preset threshold value is the cold door code rate value.
If the second elimination condition includes at least a seventh elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method may further include:
UR L L which acquires all accessed contents in the access log carries time stamps;
and determining the time stamp with the access amount lower than a third preset threshold value as the cold time stamp.
It should be noted that the specific content of the second elimination condition for determining that the access amount of the content in the to-be-eliminated content is in a sharp downward trend is merely an example, and is not used to limit the protection scope.
If the to-be-eliminated content does not satisfy the second elimination condition, it may be stated that the access amount of the to-be-eliminated content is in a decreasing trend, but is not necessarily in a sharply decreasing trend, that is, the current or future access amount of the part of the to-be-eliminated content that does not satisfy the second elimination condition is lower, but may not be particularly low, so the CDN edge node may eliminate the part of the to-be-eliminated content that does not satisfy the second elimination condition from the hot content to the non-hot content, instead of directly eliminating the cache, and eliminate at least one of the non-hot content from the cache.
According to the cache content elimination method provided by the embodiment of the invention, the part of the to-be-eliminated content with the access quantity in the hot content with high historical access quantity in the descending trend but not necessarily in the sharp descending trend is eliminated from the hot content to the non-hot content, so that the release speed of the cache space occupied by the content with high historical access quantity and low current or future access quantity is accelerated, the cache space of the CDN edge node is prevented from being occupied for a long time as much as possible, and the cache space is saved.
It is worth noting that in the method provided by the embodiment of the present invention, when the content to be eliminated meets the second elimination condition and the access amount is in a sharp decline trend, the CDN edge node directly eliminates the content to be eliminated that meets the second elimination condition out of the cache; when the content to be eliminated does not meet the second elimination condition and the access amount does not always show a sharp decline trend, only the content to be eliminated which does not meet the second elimination condition is eliminated from the hot content to the non-hot content instead of directly eliminating the content out of the cache, so that the elimination speed of the hot content with the access amount showing a decline trend is accelerated, meanwhile, the misoperation of directly eliminating the hot content with higher historical access amount but with a slow decline trend of the access amount out of the cache is reduced, and the hit rate of the cache is improved.
In addition, referring to fig. 3b, the method provided in the embodiment of the present invention may further include:
203. when the CDN edge node determines that a content elimination process in the cache needs to be started within the current time window, if it is determined that the first content set meeting the first elimination condition does not exist in the hot content, the CDN edge node eliminates at least one content in the non-hot content out of the cache.
When at least one of the non-popular contents is eliminated from the cache, the CDN edge node may specifically eliminate a low-popularity content of the non-popular contents from the cache, where the low-popularity content may include a low historical access amount of the content, or a long time interval between a latest access time of the content and a current time.
Further, after the above process, the CDN edge node may further continue to determine whether the remaining cache space is smaller than the file size of the content to be accessed; if the remaining cache space is larger than or equal to the file size of the content to be accessed, the content to be accessed can be inserted into the non-hot content, and the content to be accessed is sent to the client, so that the elimination process is completed, and the next elimination process is waited to enter; if the remaining cache space is smaller than the file size of the content to be accessed, the CDN edge node determines that the elimination process needs to be started again within the current time window to continue to eliminate the content in the cache until the remaining cache space is greater than or equal to the file size of the content to be accessed.
In addition, referring to fig. 3c, the method provided in the embodiment of the present invention may further include:
204. and when the content elimination process in the cache does not need to be started in the current time window, the CDN edge node inserts the content to be accessed into the non-hot content and sends the content to be accessed to the client.
In addition, the method provided by the embodiment of the invention can further comprise the following steps:
the CDN edge node periodically divides the content in the cache into hot content and non-hot content according to the historical access amount at a preset dividing moment, wherein a time period between the two dividing moments can be a time window. Since the access amount of the content is constantly changing, the CDN edge node periodically divides the content in the cache into the hot content and the non-hot content by constantly dividing the content at a preset dividing time, so that the dividing condition of the hot content and the non-hot content can be kept consistent with the basic trend of the real-time access amount of the content.
The division time may be set to a period of time in which the access amount is small, for example, 00:00:00 to 4:00:00 in the morning. Of course, as above, the interval between adjacent divided time may be a period T, for example, the period T may be 24 hours, and the divided time may be the time 00:00:00 of each day; the time interval between adjacent dividing time points may also be other time intervals set according to actual needs, for example, the CDN edge node may set the time interval between the dividing time points to be shorter when the access volume change speed of the content is faster, and set the time interval between the dividing time points to be longer when the access volume change speed of the content is slower, so that the dividing condition of the hot content and the non-hot content is substantially dynamically consistent with the real-time access volume of the content.
In the embodiment of the present invention, an optional implementation manner is that the hot contents form a hot queue, and the contents in the hot queue are sequentially arranged from the head of the hot queue to the tail of the hot queue according to the sequence of the historical access volumes from high to low; if it is determined that a first content set meeting a first elimination condition exists in the trending content, eliminating the to-be-eliminated content in the first content set from the trending content comprises:
determining whether the contents in the hot queue meet a first elimination condition or not in sequence according to the sequence from the tail of the hot queue to the head of the hot queue;
and if determining that a first content set meeting the first elimination condition exists in the hot content, eliminating at least one content close to the tail of the hot queue in the first content set from the hot content.
It should be noted that, because the to-be-eliminated content is usually at least one content with a lower historical access amount in the first content set that meets the first elimination condition, when the contents in the hot queue are sequentially arranged from the head of the hot queue to the tail of the hot queue according to the sequence of the historical access amounts from high to low, at least one content in the first content set that is close to the tail of the hot queue is the at least one content with a lower historical access amount in the first content set. It can be seen that the contents in the hot queue are sequentially arranged from the head of the hot queue to the tail of the hot queue according to the sequence of the historical access volumes from high to low, and the contents to be eliminated in the first content set can be quickly determined.
Of course, the contents of the hot queue may be arranged according to other rules, such as sorting according to the order of the access amount in the current time window.
In addition, because the number of the contents in the hot queue is usually large, for example, tens of thousands of contents or hundreds of thousands of contents, and when the contents in the hot queue are sorted according to the high-low order of the historical access amount, the front-back order adjustment of the contents in the current time window is small, and the resource overhead is small; if the sequencing is performed according to the high-low sequence of the access amount in the current time window, the access amount of the content in the current time window may change in real time, so that the sequence of the content in the hot queue needs to be frequently adjusted, and thus the resource overhead is increased.
In the embodiment of the present invention, an optional implementation manner is that non-hot contents form a non-hot queue, and the contents in the non-hot queue are sequentially arranged from the head of the non-hot queue to the tail of the non-hot queue in an order from high to low of a historical access amount, or in an order from short to long of a time interval between a latest access time and a current time;
the eliminating of the content not meeting the second elimination condition from the hot content to the non-hot content comprises:
eliminating the contents which do not meet the second elimination condition from the hot queue to the head of the non-hot queue;
culling at least one of the non-trending content out of the cache comprises:
and eliminating at least one content at the tail part of the non-hot queue from the cache.
The historical visit amount and the time interval between the latest visit moment and the current moment can reflect the popularity of the content, which can be called the popularity of the content, and the higher the historical visit amount is, the higher the popularity is; alternatively, the shorter the time interval between the latest access time and the current time, the higher the heat, that is, the contents in the non-hot queue are arranged from the head of the non-hot queue to the tail of the non-hot queue in the order of the heat from high to low. The CDN edge node eliminates the contents which do not meet the second elimination condition in the contents to be eliminated from the hot queue to the head of the non-hot queue, and eliminates at least one content with lower heat at the tail of the non-hot queue out of the cache, so that the elimination speed of the contents which do not meet the second elimination condition in the contents to be eliminated from the non-hot queue is lower.
The value is emphasized that, because the cache content eliminating method provided by the embodiment of the present invention does not directly eliminate the content with the lowest historical access amount from the cache, the content with low historical access amount but high current or future access amount is not easily eliminated from the cache due to low historical access amount, and thus can be stored in the cache for a long time, so that the hit rate of the cache is improved, the bandwidth of returning the source of the content to the CDN central node or the content source server is reduced, and the user experience is improved.
The embodiment of the invention provides a cache content elimination method, after dividing the content in a cache into hot content and non-hot content, if a first content set meeting a first elimination condition exists in the hot content, the access amount of the content in the first content set has a decreasing trend, the current or future access amount of the content in the first content set may be low, and thus the content to be eliminated in the first content set is eliminated from the trending content, the content to be eliminated is the subset of the first content set, so that the release speed of the cache space occupied by the hot content with high historical access amount but possibly low current or future access amount is accelerated, the hot content with high historical access amount but possibly low current or future access amount is solved, the problem of insufficient cache space is caused by long-term occupation of the cache space of CDN edge nodes.
Example 2
On the basis of embodiment 1, an embodiment of the present invention provides a method for eliminating a cache video, where contents in a cache are used as videos, hot videos in a hot queue are sequentially arranged from a head of the hot queue to a tail of the hot queue according to a sequence from high historical access amounts to low historical access amounts, non-hot videos in a non-hot queue are sequentially arranged from the head of the non-hot queue to the tail of the non-hot queue according to a sequence from high historical access amounts to low historical access amounts, and a video to be eliminated is a video with the lowest historical access amount in a first video set. The embodiment of the present invention will be mainly described in detail with respect to the part not described in embodiment 1, and with reference to fig. 4, the embodiment may include:
301. and the CDN edge node divides the videos in the cache into hot videos and non-hot videos according to the historical access amount, wherein the historical access amount of any hot video is higher than that of any non-hot video.
For step 301, reference may be made to the detailed description of step 201 in embodiment 1 above.
It should be noted that, in the embodiment of the present invention, the content in the cache is a video, and the video may be a complete video file or a fragment of the complete video file, which is not limited herein. The hot videos in the hot queue are sequentially arranged from the head of the hot queue to the tail of the hot queue according to the sequence of the historical access amount from high to low, and the non-hot videos in the non-hot queue are sequentially arranged from the head of the non-hot queue to the tail of the non-hot queue according to the sequence of the historical access amount from high to low. The head and the tail of the hot queue are opposite, one end of the hot queue for storing the hot video with high historical access amount can be the head of the hot queue, and one end of the hot queue for storing the hot video with low historical access amount can be the tail of the hot queue. Similar to the hot queue, the head and tail of the non-hot queue are also opposite.
In the embodiment of the present invention, a CDN edge node periodically divides a video in a cache into a hot video and a non-hot video, and the division time is a time 00:00:00 of each day. Wherein, 00:00:00 to 23:59:59 of each day can be called a time window, 00:00:00 of the day to 23:59:59 of the day can be called a current time window, and the length of the time window is 24 hours of the corresponding period.
302. The CDN edge node determines whether a video elimination process in the cache needs to be started within a current time window.
Specifically, referring to fig. 5, determining, by the CDN edge node, that a video elimination process in the cache needs to be started within the current time window may include:
1-1, receiving an access request for acquiring a video to be accessed, which is sent by a client, by a CDN edge node.
1-2, the CDN edge node determines whether a video to be accessed requested by a client is stored in a cache.
And 1-3, if so, the CDN edge node sends the video to be accessed in the cache to the client.
If the to-be-accessed video requested to be accessed by the client is stored in the cache of the CDN edge node, the to-be-accessed video does not need to be acquired from the CDN central node, and the to-be-accessed video in the cache can be directly sent to the client.
1-4, if not, the CDN edge node determines whether the video to be accessed meets the caching condition.
If the to-be-accessed video requested to be accessed by the client is not stored in the cache of the CDN edge node, the to-be-accessed video needs to be acquired from the CDN central node, and whether the to-be-accessed video meets the cache condition is further determined. The caching conditions may include: the historical access amount of the video to be accessed is higher than or equal to a first preset threshold value. When the historical access amount of the video to be accessed is higher than or equal to a first preset threshold value, the video to be accessed is considered to be popular, and the video to be accessed needs to be stored in a cache, so that the cache hit rate of the CDN edge node is improved, and the back source bandwidth of the video to be accessed, which is obtained by the CDN edge node from a CDN center node and a CDN center node from an origin server, is reduced. The first preset threshold may be set according to specific situations, and may be 10, for example.
1-5, if the video to be accessed meets the caching condition, determining whether the residual caching space is smaller than the file size of the video to be accessed by the CDN edge node.
1-6, if the remaining cache space is smaller than the file size of the video to be accessed, the CDN edge node determines that a video elimination process in the cache needs to be started in the current time window.
1-7, if the residual cache space is larger than or equal to the file size of the video to be accessed, determining that a video elimination process in the cache does not need to be started in the current time window by the CDN edge node, inserting the video to be accessed into the non-popular video, and sending the video to be accessed to the client.
303. When the CDN edge node determines that a video elimination process in the cache needs to be started in the current time window, whether the hot videos in the hot queue meet a first elimination condition or not is determined in sequence according to the sequence from the tail of the hot queue to the head of the hot queue.
And when the video elimination process in the cache needs to be started in the current time window, the CDN edge node starts to enter the elimination process. Specifically, the CDN edge node may sequentially determine whether the top videos in the top queue satisfy the first elimination condition according to an order from the tail of the top queue to the head of the top queue, that is, according to an order from the top video with the lowest historical access amount to the top video with the highest historical access amount in the top queue. The method includes the steps that videos meeting first elimination conditions in hot queues determined by CDN edge nodes for the first time are videos with the lowest historical visit quantity in first video sets meeting the first elimination conditions in the hot queues.
For example, referring to the schematic structural diagram of the hot queue shown in fig. 6, a CDN edge node may first determine whether a hot video n located at the tail of the hot queue meets a first elimination condition, and if the hot video n does not meet the first elimination condition, the CDN edge node determines whether a hot video n-1 meets the first elimination condition, so as to sequentially determine whether hot videos in the hot queue meet the first elimination condition, until it is first determined that a certain hot video meets the first elimination condition, it is no longer determined whether the subsequent hot video meets the first elimination condition, and step 304 is performed. The CDN edge node sequentially determines whether the hot videos in the hot queue meet the first elimination condition according to the sequence from the hot video with the lowest historical access amount to the hot video with the highest historical access amount in the hot queue, so that the hot videos meeting the first elimination condition in the hot queue determined for the first time are the videos with the lowest historical access amount in the first video set meeting the first elimination condition. Therefore, the videos in the hot queue are arranged according to the historical access amount, and the video with the lowest historical access amount in the first video set meeting the first elimination condition can be quickly determined, so that the processing efficiency of the CDN edge node is improved; otherwise, after determining all the first video sets meeting the first elimination condition in the hot queue, the video with the lowest historical access amount may need to be determined from the first video sets, thereby reducing the processing efficiency.
If no hot video satisfying the first elimination condition appears until the head of the hot queue, it may be determined that none of the hot videos in the hot queue satisfies the first elimination condition, and proceed to step 307.
The first elimination condition may be a first elimination sub-condition, or a combination of at least one of a second elimination sub-condition, a third elimination sub-condition, and a fourth elimination sub-condition with the first elimination sub-condition.
The access amount of the video in the hot door video in the current time window is lower than the preset access amount.
Specifically, since the preset access amount is an expected access amount that can be achieved in the current time window estimated according to the previous access condition of the video, if the access amount of the video in the hot door video in the current time window is lower than the preset access amount, it can be stated that the current access amount of the video is lower and is in a downward trend. For example, if the current time window is from 00:00:00 of the current day to 23:59:59 of the current day, and the time is divided into 00:00: 00:00 of the current day, the determining, by the CDN edge node, whether any of the hit videos in the hit queue satisfies the first culling condition may include:
(1) and if the first elimination condition only comprises the first elimination sub-condition, in the current time window, when the visit quantity of the hot video is lower than the preset visit quantity, the hot video meets the first elimination condition. For example, if the preset access amount is 10, from 00:00:00 to the current time of the day, the first elimination condition is satisfied by top videos with access amounts lower than 10 times, and the first elimination condition is not satisfied by top videos with access amounts higher than or equal to 10 times. The preset access amount corresponding to different videos may be the same or different.
(2) And if the first elimination condition only comprises the first elimination sub-condition and the second elimination sub-condition, the popular video meets the first elimination condition when the visit volume of the popular video in the current time window is lower than the preset visit volume and the current time is not in the video preset protection period in the popular video. For example, when the hit video is a movie that is just on line, a protection period may be preset for the hit video, for example, the preset protection period includes a quarter of the current time window, and the preset access amount is 10, then the hit video does not satisfy the second culling sub-condition and thus does not satisfy the first culling condition on the day 00:00:00 to 05:59:59 on the day, so that the determination of the access amount may not be performed on the day 00:00: 00:00 to 05:59: 59; after the current day 05:59:59, if the visit rate of the popular video is higher than or equal to 10 times from the current day 00:00:00 to the current moment, the popular video simultaneously satisfies the first eliminating sub-condition and the second eliminating sub-condition, so that the first eliminating condition is satisfied; and after 05:59:59, if the visit amount of the top video is less than 10 times from 00:00:00 to the current time of day, the top video satisfies the second culling sub-condition but does not satisfy the first culling sub-condition, and thus does not satisfy the first culling condition.
(3) If the first elimination condition only comprises the first elimination sub-condition and the third elimination sub-condition, the visit quantity of the hot videos in the current time window is lower than the preset visit quantity, and when the quantity of the videos which are eliminated from the cache in the hot videos in the current time window is lower than the preset elimination quantity, the hot videos meet the first elimination condition. The preset elimination quantity can be the approximate quantity of videos in hot videos counted by the current network each day, and the videos are eliminated. For example, if the preset culling quantity is 5 ten thousand, from 00:00:00 to the current time of the day, when the culled quantity of the top video is higher than or equal to 5 ten thousand, the top video does not satisfy the third culling sub-condition; when the number of rejected popular videos is less than 5 ten thousand, the popular video meets the third rejection sub-condition, and when the popular video also meets the first rejection sub-condition, the popular video meets the first rejection condition.
(4) And if the first elimination condition only comprises the first elimination sub-condition and the fourth elimination sub-condition, the hit rate of the hot videos in the current time window is lower than the preset hit rate, and the hot videos meet the first elimination condition when the ratio of the number of the hot videos eliminated from the cache to the number of the non-hot videos eliminated from the cache in the current time window is lower than the preset elimination ratio. The preset elimination ratio can be calculated according to the current network, and the video with the approximate proportion in the hot video every day can be eliminated. For example, if the preset elimination ratio is 1:2, from 00:00:00 to the current time of the day, when the elimination quantity ratio of the hot video with the cache eliminated to the non-hot video with the cache eliminated is higher than or equal to 1:2, the hot video does not satisfy the fourth elimination sub-condition; when the ratio of the elimination quantity of the hot videos with the cache eliminated to the non-hot videos with the cache eliminated is lower than 1:2, the hot videos meet the fourth elimination sub-condition, and when the hot videos also meet the first elimination sub-condition, the hot videos meet the first elimination condition.
It should be noted that the first elimination condition may also be a combination of two or more of the above sub-conditions, which is not described herein again.
304. If a first video set meeting a first elimination condition exists in the hot video, the CDN edge node determines whether the video to be eliminated in the first video set meets a second elimination condition; if yes, go to step 305; if not, go to step 306.
In the embodiment of the invention, the video to be eliminated is the video with the lowest historical visit amount in the first video set meeting the first elimination condition. When the CDN edge node determines whether the hot videos in the hot queue meet the first elimination condition in sequence according to the sequence from the tail part to the head part of the hot queue, the hot videos in the hot queue are sequenced from the head part of the hot queue in sequence according to the sequence from high historical access amount to low historical access amount, so that when the CDN edge node determines that a certain video meets the first elimination condition for the first time, the video is the video which is closest to the tail part of the hot queue in the first video set meeting the first elimination condition, is also the video with the lowest historical access amount in the first video set, and is also the video to be eliminated.
Because the historical visit quantity is higher and the visit quantity of the videos to be eliminated meeting the first elimination condition is in a descending trend, the videos can be eliminated from the popular videos so as to accelerate the elimination speed. However, the video to be eliminated is eliminated from the hot queue to the non-hot queue, or the video to be eliminated is directly eliminated from the cache, and the determination needs to be made according to whether the video to be eliminated meets the second elimination condition. And the second elimination condition is used for determining that the visit quantity of the video to be eliminated is in a sharp descending trend. If the to-be-eliminated video satisfies the second elimination condition, the process may proceed to step 305, and if the to-be-eliminated video does not satisfy the second elimination condition, the process may proceed to step 306.
The second elimination condition for determining that the access amount of the video to be eliminated is in a sharp decline trend may specifically include at least one of a fifth elimination sub-condition, a sixth elimination sub-condition, and a seventh elimination sub-condition. For example, determining, by the CDN edge node, whether the to-be-culled video satisfying the first culling condition in the hit queue satisfies the second culling condition may include:
(1) when the video to be eliminated meets the fifth eliminating sub-condition, the timeliness of the video to be eliminated can be proved to be strong, concentrated access can occur only in a short period of time, and then the access amount can be reduced sharply, so that the video to be eliminated can be considered to be eliminated from a cache.
If the second elimination condition at least comprises a fifth elimination sub-condition, before the CDN edge node determines whether the video to be eliminated meets the second elimination condition, the CDN edge node may obtain UR L of the video to be eliminated from the top queue, and request the CDN center node to obtain a Category table corresponding to the video to be eliminated according to UR L of the video to be eliminated, the CDN center node searches for the Category table in its CMS database and returns it to the CDN edge node, the Category table includes a type field, that is, a paretid field, that has a correspondence with a file type, and the CDN edge node may determine the file type of the video to be eliminated according to the paretid field.
For example, the structure of the Category table can be found in table 1:
TABLE 1
Name (R) Type (B) Key with a key body Brief introduction to the drawings
id int Main key Identification
name varchar Name (R)
parentid int Class of father
remark varchar Remarks for note
For example, the mapping relationship between the Parentid field and the file type can be seen in table 2:
TABLE 2
Parentid File type
1000 Film
1101 News
1102 Finance and economics
1103 Sports
1104 Subject for special use
1105 Method of making
1106 Interview
1107 Comprehensive art
1108 Music
1109 Drama (drama)
1110 Foreign language
1200 Advertising
For an example, a Category table corresponding to a certain video can be referred to as table 3:
TABLE 3
Figure BDA0000837049460000201
If the second elimination condition at least includes a fifth elimination sub-condition, after determining the file type of the video to be eliminated, the CDN edge node may further determine whether the file type of the video to be eliminated is a file type with strong timeliness. Specifically, if the file type of the video to be eliminated is any one of the preset file types, the CDN edge node determines that the video to be eliminated satisfies the fifth elimination sub-condition, so as to satisfy the second elimination condition; otherwise, the CDN edge node determines that the video to be eliminated does not satisfy the fifth elimination sub-condition, so that the second elimination condition is not satisfied.
(2) The method includes the steps of obtaining a code rate value of a video to be eliminated, determining whether the video to be eliminated meets a second elimination condition when the code rate value of the video to be eliminated is any one of the code rate values of the cold gates, and if the second elimination condition only includes a sixth elimination sub-condition, determining that the video to be eliminated meets the second elimination condition when the code rate value of the video to be eliminated is any one of the code rate values of the cold gates.
In addition, if the second elimination condition at least includes a sixth elimination sub-condition, before the CDN edge node determines whether the video to be eliminated satisfies the second elimination condition, the method may further include determining a cold gate rate value, which is shown in fig. 7, and specifically may include:
2-1, establishing a code rate statistic list by the CDN edge node, wherein each item in the code rate statistic list comprises a code rate value and a corresponding access amount, and the initial length of the code rate statistic list is 0;
2-2, obtaining a non-counted uniform resource locator UR L of the video from the stored access log by the CDN edge node, and obtaining a code rate value according to a code rate value field carried in UR L;
2-3, determining whether the obtained code rate value exists in a code rate statistic list by the CDN edge node;
2-4, if yes, adding 1 to the access amount corresponding to the code rate value in the code rate statistic list by the CDN edge node;
2-5, if not, inserting the code rate value into the tail of the code rate statistic list by the CDN edge node, and setting the corresponding access amount as 1;
2-6, determining whether the access log is counted by the CDN edge node;
2-7, if not, the CDN edge node continues to obtain an unmetered UR L from the access log;
and 2-8, if so, determining that the code rate value with the access amount lower than a second preset threshold value is the cold door code rate value by the CDN edge node.
The rate value field corresponding to the rate value in UR L may be Quality L events field, for example, if UR L of a video is:
http://news.cntv.cn/2015/02/10/VIDE1423528743144611.shtml/QualityLevels(900000)/Fragments(video=200200000),
then "Quality L updates (900000)" in UR L is Quality L updates field, and the corresponding bitrate value is 900000 bits/sec.
Or, in step 2-8, the CDN edge node may also sequence all code rate values according to the access amount of the code rate value, and determine the code rate value with a lower access amount of a preset number as the cold gate code rate value.
(3) When the to-be-eliminated video meets the seventh eliminating sub-condition, the timestamp of the to-be-eliminated video can be described as a cold timestamp which is not frequently accessed by the client, for example, the to-be-eliminated video is a header or a trailer of a movie, so that the to-be-eliminated video can be considered to be eliminated from the cache.
In addition, if the second elimination condition at least includes a seventh elimination sub-condition, before the CDN edge node determines whether the video satisfies the second elimination condition, the method provided in the embodiment of the present invention may further include determining a cold timestamp, which is shown in fig. 8, and specifically may include:
3-1, creating a timestamp statistical list by the CDN edge node, wherein each item in the timestamp statistical list comprises a timestamp and a corresponding visit amount, and the initial length of the timestamp statistical list is 0;
3-2, the CDN edge node obtains a non-counted uniform resource locator UR L of the video from the stored access log, and obtains a timestamp according to a timestamp field carried in UR L;
3-3, determining whether the acquired timestamp exists in a timestamp statistical list by the CDN edge node;
3-4, if yes, adding 1 to the access amount corresponding to the timestamp in the timestamp statistical list by the CDN edge node;
3-5, if not, inserting the timestamp into the tail of the timestamp counting list by the CDN edge node, and setting the corresponding access amount as 1;
3-6, determining whether the access log is counted by the CDN edge node;
3-7, if not, the CDN edge node continues to obtain an unmetered UR L from the access log;
and 3-8, if so, determining that the time stamp with the access amount lower than a third preset threshold value is the cold door time stamp by the CDN edge node.
The timestamp field in UR L corresponding to the timestamp may be a Fragments field, for example, if UR L of a video is:
http://news.cntv.cn/2015/02/10/VIDE1423528743144611.shtml/QualityLevels(900000)/Fragments(video=200200000),
then "Fragments (200200000)" in UR L is a Fragments field, the unit of the timestamp is 10-7 seconds, i.e. 0.1 microsecond, the corresponding timestamp is 20020000 microseconds, i.e. the video is a video slice of a complete video, and the start time of the video is the 20.02 th second of the complete video.
Or, in step 3-8, the CDN edge node may also sort all the timestamps according to the access amounts of the timestamps, and determine the timestamps with a lower access amount in a preset number as the cold timestamp.
It should be noted that the second elimination condition may also be a combination of two or more of the above sub-conditions, which is not described herein again.
305. And the CDN edge node eliminates the video to be eliminated from the cache.
And if the to-be-eliminated video meeting the first elimination condition also meets the second elimination condition, directly eliminating the to-be-eliminated video from the cache by the CDN edge node.
It should be noted that, in the cache video elimination method provided by the embodiment of the present invention, the part of the to-be-eliminated video with the sharply decreased access amount in the top-ranked video with the higher historical access amount is directly eliminated from the cache, so that the release speed of the cache space occupied by the top-ranked video with the high historical access amount but with the sharply decreased current or future access amount is increased, and the cache space of the CDN edge node is prevented from being occupied for a long time.
306. And the CDN edge node eliminates the video to be eliminated from the hot queue to the head of the non-hot queue and eliminates at least one video positioned at the tail part of the non-hot queue out of the cache.
In the embodiment of the present invention, since the videos in the non-hot queue are sequentially arranged from the head of the non-hot queue to the tail of the non-hot queue in the order from high to low in the historical access amount, the video at the tail of the non-hot queue is the video with the lowest historical access amount in the non-hot queue.
Therefore, if the to-be-eliminated video does not satisfy the second elimination condition, the CDN edge node may eliminate the to-be-eliminated video to the head of the non-hit queue, and eliminate at least one video with a lower historical access amount located at the tail of the non-hit queue from the cache. Because the non-hot queue eliminates the videos from the tail part with lower heat degree and the elimination speed of the videos at the head part of the non-hot queue is lower, the to-be-eliminated videos which have higher historical access but do not meet the second elimination condition in the hot queue are eliminated to the head part of the non-hot queue and cannot be eliminated out of the cache quickly, thereby reducing the probability that the hot videos are mistakenly eliminated out of the cache and improving the hit rate of the cache.
It should be noted that, in the cache video elimination method provided in the embodiment of the present invention, the to-be-eliminated video whose access amount is in a descending trend but not necessarily in a sharp descending trend in the top videos with high historical access amounts is eliminated from the top videos to the non-top videos, so that the release speed of the cache space occupied by the videos with high historical access amounts but with low current or future access amounts is increased, and the cache space of the CDN edge node is prevented from being occupied for a long time as much as possible, thereby saving the cache space.
It is worth noting that in the method provided by the embodiment of the present invention, when the video to be eliminated meets the second elimination condition and the access amount is in a sharp decline trend, the CDN edge node directly eliminates the video to be eliminated that meets the second elimination condition out of the cache; when the video to be eliminated does not meet the second elimination condition and the access amount does not always show a sharp decline trend, only the video to be eliminated which does not meet the second elimination condition is eliminated from the hot video to the non-hot video instead of directly eliminating the video from the cache, so that the elimination speed of the hot video with the access amount showing a decline trend is accelerated, meanwhile, the misoperation of directly eliminating the hot video with higher historical access amount but with the access amount showing a slow decline trend from the cache is reduced, and the hit rate of the cache is improved.
307. If the CDN edge node determines that the first video set meeting the first elimination condition does not exist in the hot video, the CDN edge node eliminates at least one video positioned at the tail part of the non-hot queue out of the cache.
If any hot video in the hot queue does not meet the first elimination condition, the CDN edge node may eliminate at least one video located at the tail of the non-hot queue from the cache.
The cache video elimination method provided by the embodiment of the invention does not directly eliminate the video with the lowest historical access amount from the cache, so that the video with low historical access amount but high current or future access amount is not easily eliminated from the cache due to low historical access amount, and can be stored in the cache for a long time, thereby improving the hit rate of the cache, reducing the source returning bandwidth of the video acquired from the CDN edge node to the CDN central node or the CDN central node to the video source server, and improving the user experience.
Further, after step 301-; if the residual cache space is larger than or equal to the file size of the video to be accessed, the video to be accessed can be inserted into the non-hot video, and the video to be accessed is sent to the client, so that the elimination process is completed, and the next elimination process is waited to enter; if the remaining cache space is smaller than the file size of the video to be accessed, the CDN edge node continues to eliminate the video in the cache until the remaining cache space is greater than or equal to the file size of the video to be accessed.
308. And when determining that the video elimination process in the cache does not need to be started in the current time window, the CDN edge node inserts the video to be accessed into the non-hot video and sends the video to be accessed to the client.
When it is determined that the video elimination process in the cache does not need to be started within the current time window, because the historical access amount of the video to be accessed is generally not higher than that of the video in the cache, the CDN edge node may insert the video to be accessed into the tail of the non-hit queue, so that the order of the video in the non-hit queue and the historical access amount is kept consistent.
In addition, referring to fig. 9, after the CDN edge node divides the video in the cache into the hot video and the non-hot video at the preset dividing time in step 301, the method provided in the embodiment of the present invention may further include:
309. if the CDN edge node determines that the current time is the next divided time, continue to step 301.
And if the CDN edge node determines that the current moment is the next dividing moment, dividing the video in the cache into a hot video and a non-hot video according to the historical access amount again.
For example, when the time interval between the preset dividing moments is a period T, the CDN edge node may determine, after each elimination process is finished, whether a time interval between a current moment and a dividing moment at which the CDN edge node divides a video in a cache into a top video and a non-top video is greater than or equal to the period T; if so, determining that the current moment is the next division moment, and dividing the video in the cache into hot videos and non-hot videos according to the historical access amount again; if not, continuously determining whether the video in the cache needs to be eliminated or not so as to wait for entering the next elimination flow. According to the scheme, whether the current time is the next division time needs to be detected after the elimination process is finished every time, so that the cost is high.
Or, the CDN edge node may trigger to detect whether the current elimination process is finished when a time interval between the current time and a division time at which the video in the cache is divided into the hot video and the non-hot video last time is greater than or equal to the period T and thus the next division time is reached; if the current video is ended, dividing the video in the cache into hot video and non-hot video according to the historical access amount; if not, after the current elimination process is finished, dividing the videos in the cache into hot videos and non-hot videos according to the historical access amount again.
The embodiment of the invention provides a cache video elimination method, wherein after a CDN edge node divides videos in a cache into a hot queue and a non-hot queue, if videos to be eliminated with the lowest historical access amount in a first video set with the access amount in a descending trend in the hot queue meet a second elimination condition, the videos to be eliminated with the access amount in a sharp descending trend are eliminated from the cache, so that the release speed of a cache space occupied by the hot videos with high historical access amount but sharply reduced at present or in future can be accelerated, and the cache space of the CDN edge node is prevented from being occupied for a long time; if the video to be eliminated does not meet the second elimination condition, the access quantity of the video to be eliminated does not necessarily fall, so that the video to be eliminated can be eliminated to the non-hit queue, and at least one video with lower historical access quantity at the tail of the non-hit queue is eliminated from the cache, so that the elimination speed of the hit video with lower access quantity can be accelerated, the misoperation of directly eliminating the hot video with higher historical access quantity but slowly falling access quantity from the cache can be reduced, and the hit rate of the cache is improved.
Example 3
On the basis of embodiment 2, referring to fig. 10, an embodiment of the present invention provides another cache video elimination method, which mainly describes an interaction process among a CDN edge node, a CDN center node, a client, and an origin server, and details of the parts not described in embodiment 2 are described below, specifically as follows:
401. and the CDN edge node divides the videos in the cache into hot videos and non-hot videos according to the historical access amount, wherein the historical access amount of any hot video is higher than that of any non-hot video.
Specifically, step 401 may refer to the description in step 201.
402. The client sends a video access request to the video source server, the video access request including UR L for the video, and the video source server provides video acceleration services through the CDN.
For example, the video access request for requesting access to a certain video sent by the client to the video source server may include:
201.106.77.103--[02/Jul/2014:12:04:58+0000]"GEThttp://sscdn.clarovideo.com/multimediav81/plataforma_vod/ISM/201301/WMP4H01538MTSS_full/WMP4H01538MTSS_full.ism/QualityLevels(1105000)/Fragments(video=3563560000)HTTP/1.1"10583064"-""Mozilla/5.0(Windows NT 6.1;WOW64)AppleWebKit/537.36(KHTML,like Gecko)Chrome/35.0.1916.153Safari/537.36OPR/22.0.1471.70""-"
wherein, the meaning of each field from front to back is as follows:
201.106.77.103: % client, IP address of client;
the first "-": % ident, unique designation of viewer;
the second "-": % authuser, name provided at the time of browser authentication;
[02/Jul/2014:12:04:58+0000 ]: % data, the time when the CDN edge node receives the video access request;
GET: % method, request method;
http:// sscdn. clarovideo. com/multimedia 81/platoforma _ vod/ISM/201301/WMP4H01538MTSS _ full/WMP4H01538MTSS _ full. ISM/Quality L events (1105000)/Fragments (video ═ 3563560000): resource,% resource, UR L for video;
HTTP/1.1: % protocol, protocol and version information;
105: % status, HTTP status code;
83064: % response size, the actual number of bytes transferred for this request, in bytes, without an HTTP header;
the third "-": % refer, the page viewed by the viewer prior to accessing the page, here null;
"Mozilla/5.0(Windows NT 6.1; WOW64) AppleWebKit/537.36(KHTM L, likeGecko) Chrome/35.0.1916.153Safari/537.36 OPR/22.0.1471.70" "-", agent, browser information of client, OS information, etc.
403. The CDN central node redirects the client to a CDN edge node.
After the CDN central node redirects the client to the CDN edge node, the CDN edge node may receive a video access request from the client and send a video requested for access to the client through the CDN edge node.
404. The CDN edge node receives an access request of a client and determines whether a video to be accessed requested by the client is stored in a cache, if so, the step 405 is executed, and if not, the step 406 is executed.
After receiving a video access request sent by a client, a CDN edge node parses UR L of the video from the video access request, and calculates a hash value of UR L, where, for example, the hash value parsed by the CDN edge node according to UR L may be 3365827339883413968.
405. The CDN edge node sends the video to be accessed stored in the cache to the client, and proceeds to step 402.
And if the video to be accessed requested by the client is stored in the CDN edge node, directly sending the video to be accessed stored in the cache to the client.
406. And the CDN edge node requests the CDN center node to acquire the video to be accessed.
If the to-be-accessed video requested by the client is not stored in the CDN edge node, the to-be-accessed video may be requested to be acquired from the CDN center node.
For example, corresponding to the video access request sent by the client in step 402, the video acquisition request sent by the CDN edge node to the CDN central node may be:
189.247.157.203--[02/Jul/2014:12:05:00+0000]"GET http://sscdn.clarovideo.com/multimediav00/plataforma_vod/ISM/201301/WMP4H01538MTSS_full/WMP4H01538MTSS_full.ism/QualityLevels(1105000)/Fragments(video=3563560000)HTTP/1.1"10583064"-""curl/7.19.0(x86_64-suse-linux-gnu)libcurl/7.19.0OpenSSL/0.9.8j zlib/1.2.3 libidn/1.10""-"
the meaning of each field in the video obtaining request is similar to that of the video access request in step 402, except that the requesting party in the video obtaining request is a CDN edge node, and the requesting party in the video access request is a client.
407. The CDN central node acquires the video to be accessed from the source server and sends the video to be accessed to the CDN edge node, or the CDN central node sends the video to be accessed in the cache of the CDN central node to the CDN edge node.
After receiving a video acquisition request sent by a CDN edge node for acquiring the video to be accessed, the CDN central node analyzes UR L from the video acquisition request, calculates the hash value of the UR L, searches whether UR L of the video with the same hash value exists in a cache according to the hash value, if yes, the CDN central node directly sends the video to be accessed stored in the cache to the CDN edge node, and if not, the CDN central node acquires the video to be accessed from an origin server and sends the video to be accessed to the CDN edge node.
408. The CDN edge node determines whether the video to be accessed meets a caching condition; if the cache condition is not satisfied, go to step 409; if the caching condition is satisfied, go to step 410.
Illustratively, the caching condition is that the historical access amount of the video is higher than or equal to a first preset threshold value of 10. the CDN edge node first obtains a historical access list in the CDN edge node, the list records the historical access times of each video, then searches for a video with the same hash value from the historical access list according to the hash value 3365827339883413968 of the to-be-accessed video UR L, and obtains the historical access times of the video.
409. The CDN edge node sends the video to be accessed to the client, and proceeds to step 402.
410. The CDN edge node determines whether the residual cache space is smaller than the file size of the video to be accessed; if yes, go to step 411; if not, go to step 422.
If the video to be accessed meets the cache condition, the CDN edge node needs to store the video to be accessed in the cache, so that the CDN edge node can further determine whether the remaining cache space is sufficient to store the file size of the video to be accessed. Illustratively, the CDN edge node may analyze, according to a video access request sent by the client, that the file size of the video to be accessed that the client requests to access is 83064bytes, and the CDN edge node may determine whether the remaining cache space is smaller than the file size of the video to be accessed 83064 bytes.
411. The CDN edge node determines that a video elimination process in the cache needs to be started in the current time window.
If the remaining cache space of the CDN edge node is smaller than the file size of the video to be accessed 83064bytes, the remaining cache space is not enough to store the video to be accessed, and the CDN edge node determines that the video in the cache needs to be eliminated.
As can be seen from the above, the process of determining whether to eliminate the video in the CDN edge node cache in step 402-411 may refer to step 302 in embodiment 2.
412. When the CDN edge node determines that the videos in the cache need to be eliminated, the CDN edge node sequentially determines whether the videos in the hot queue meet a first elimination condition according to the sequence from the tail of the hot queue to the head of the hot queue.
For a detailed description of step 412, refer to step 303. If it is determined that one popular video meets the first elimination condition for the first time, the CDN edge node determines that the popular video is a to-be-eliminated video with the lowest historical access amount in the first video set that meets the first elimination condition.
After step 303 and before step 304 in embodiment 2, the method may further include:
413. if a first video set meeting a first elimination condition exists in the top videos, the CDN edge node obtains UR L of the videos to be eliminated in the first video set.
Specifically, the process of determining, by the CDN edge node, a video to be eliminated in the first video set may refer to step 304 in embodiment 2.
414. And the CDN edge node requests the category list corresponding to the video to be eliminated from the CDN central node according to the obtained UR L of the video to be eliminated.
415. And the CDN central node sends the category list corresponding to the video to be eliminated to the CDN edge node.
416. And the CDN edge node receives a category list corresponding to the video to be eliminated sent by the CDN center node, and determines the file type of the video to be eliminated according to the type field in the category list.
In the above steps 413, 414, and 416, a specific process of determining, by the CDN edge node, the file type of the video to be eliminated according to the category table may refer to the description in the step 304 in the above embodiment 2, and details are not described here.
417. And the CDN edge node determines the code rate value and the time stamp of the video according to the obtained UR L of the video to be eliminated.
The CDN edge node may determine the code rate value of the video according to the Quality L events field carried in UR L of the video to be eliminated, and determine the timestamp of the video to be eliminated according to the timestamp field Fragments field carried in UR L of the video to be eliminated, which may specifically refer to the detailed description in step 304 in embodiment 2.
418. The CDN edge node determines whether the video to be eliminated meets a second elimination condition or not; if yes, go to step 419; if not, go to step 420.
When the second elimination condition includes a fifth elimination sub-condition, a sixth elimination sub-condition, and a seventh elimination sub-condition, if the file type of the video to be eliminated is any one of the preset file types so as to satisfy the fifth elimination sub-condition, and the code rate value of the video to be eliminated is any one of the cold gate code rate values so as to satisfy the sixth elimination sub-condition, and the timestamp of the video to be eliminated is any one of the cold gate timestamps so as to satisfy the seventh elimination sub-condition, the CDN edge node may determine that the video to be eliminated satisfies the second elimination condition.
It should be noted that before step 418, the CDN edge node further needs to determine a cold gate code rate and a cold gate timestamp, and the specific process may refer to the detailed description in step 304 in embodiment 2, which is not described herein again.
419. The CDN edge node evicts the video to be evicted from the cache and continues to step 410.
420. The CDN edge node culls the video to be culled from the hot queue to the head of the non-hot queue, culls at least one video located at the tail of the non-hot queue out of the cache, and continues to step 410.
421. If the CDN edge node determines that the first video set satisfying the first elimination condition does not exist in the top videos, the CDN edge node eliminates at least one video located at the tail of the non-top queue out of the cache, and proceeds to step 410.
422. The CDN edge node determines that a video elimination process in the cache does not need to be started within the current time window, inserts the video to be accessed into the tail of the non-hot queue, sends the video to be accessed to the client, and continues to enter step 402.
Wherein, the steps 419-422 can be referred to the detailed description of the steps 305-308 in the above embodiment.
Fig. 11 shows a schematic diagram of the cache video elimination process according to an embodiment of the present invention.
Furthermore, after step 401, the method provided by the implementation of the present invention may further include:
423. if the CDN edge node determines that the current time is the next divided time, then step 401 is entered.
For a detailed description of step 423, refer to step 309 in embodiment 2 above.
Consistent with embodiment 2, in the cache video elimination method provided in the embodiment of the present invention, after dividing a video in a cache into a hit queue and a non-hit queue, if a to-be-eliminated video with the lowest historical access amount in a first video set with a downward access amount in the hit queue meets a second elimination condition, the to-be-eliminated video with a steeply downward access amount is eliminated from the cache, so that a release speed of a cache space occupied by the hit video with a high historical access amount but a steep decrease in current or future access amount can be increased, and the cache space of the CDN edge node is prevented from being occupied for a long time; if the video to be eliminated does not meet the second elimination condition, the access quantity of the video to be eliminated does not necessarily fall, so that the video to be eliminated can be eliminated to the non-hit queue, and at least one video with lower historical access quantity at the tail of the non-hit queue is eliminated from the cache, so that the elimination speed of the hit video with lower access quantity can be accelerated, the misoperation of directly eliminating the hot video with higher historical access quantity but slowly falling access quantity from the cache can be reduced, and the hit rate of the cache is improved.
Example 4
An embodiment of the present invention provides a cache device using the cache content elimination method provided in embodiments 1 to 3, and referring to fig. 12a, the cache device may include:
the dividing unit 111 may be configured to divide the content in the cache into popular content and non-popular content according to the historical access amount, where the historical access amount of any popular content is higher than that of any non-popular content.
The eliminating unit 112 may be configured to, when it is determined that a content eliminating process in the cache needs to be started within the current time window, eliminate, if it is determined that a first content set meeting a first eliminating condition exists in the trending content, content to be eliminated in the first content set from the trending content, where the content to be eliminated includes at least one content, and the first eliminating condition is used to determine that an access amount of the content in the trending content is in a downward trend.
The content to be eliminated can be at least one content with lower historical access in the first content set meeting the first elimination condition. In the embodiment of the present invention, if the caching device determines that the first content set meeting the first elimination condition exists in the hot content, it may indicate that the access amount of the content in the first content set is in a downward trend, and the current or future access amount of the content in the first content set may be lower, so that the content to be eliminated in the first content set may be eliminated from the hot content, thereby increasing the elimination speed of the content with high historical access amount but possibly lower current or future access amount, and avoiding that the caching device occupies the caching space for a long time, thereby resulting in insufficient caching space.
Wherein, the elimination unit 112 may be specifically configured to:
determining whether the content in the content to be eliminated meets a second elimination condition;
eliminating the contents meeting the second elimination condition from the contents to be eliminated out of the cache;
and eliminating the contents which do not meet the second elimination condition from the hot contents to the non-hot contents in the contents to be eliminated, and eliminating at least one content from the non-hot contents out of the cache.
Culling unit 112 may also be used to:
and if the first content set meeting the first elimination condition does not exist in the hot content, eliminating at least one content in the non-hot content from the cache.
It should be noted that, with the cache device provided in the embodiment of the present invention, when the content to be eliminated satisfies the second elimination condition and the access amount is in a sharp decline trend, the content to be eliminated satisfying the second elimination condition is directly eliminated from the cache; when the content to be eliminated does not meet the second elimination condition and the access amount does not always show a sharp decline trend, only the content to be eliminated which does not meet the second elimination condition is eliminated from the hot content to the non-hot content instead of directly eliminating the content out of the cache, so that the elimination speed of the hot content with the access amount showing a decline trend is accelerated, meanwhile, the misoperation of directly eliminating the hot content with higher historical access amount but with a slow decline trend of the access amount out of the cache is reduced, and the hit rate of the cache is improved.
In addition, referring to fig. 12b, an interface unit 113 may be further included, configured to receive an access request sent by the client to obtain the content to be accessed.
Furthermore, the elimination unit 112 may be further specifically configured to:
determining that the content to be accessed received by the interface unit 113 is not stored in the cache;
when the content to be accessed meets the caching condition, determining that the residual caching space is smaller than the file size of the content to be accessed;
wherein, the caching condition comprises:
the historical access amount of the content to be accessed is higher than or equal to a first preset threshold value.
Culling unit 112 may also be used to:
when determining that the content elimination process in the cache does not need to be started in the current time window, inserting the content to be accessed into the non-hot content;
the interface unit 113 may also be used to send content to be accessed to the client.
Wherein, the first eliminating condition may include a first eliminating sub-condition:
the access amount of the content in the hot content in the current time window is lower than the preset access amount.
Further, the first elimination condition may further include a second elimination sub-condition, and/or a third elimination sub-condition, and/or a fourth elimination sub-condition;
the second knockout sub-condition comprises: the current time is not in a preset protection period of the content in the hot content;
the third knockout sub-condition comprises: the number of the eliminated cached contents in the hot contents in the current time window is lower than the preset elimination number;
the fourth knockout sub-condition comprises: the ratio of the number of the hot contents with the cache eliminated to the number of the non-hot contents with the cache eliminated in the current time window is lower than the preset elimination ratio.
The second knockout condition may include at least one of a fifth knockout sub-condition, a sixth knockout sub-condition, and a seventh knockout sub-condition;
the fifth eliminating sub-condition comprises: the file type of the content in the to-be-eliminated content is a preset file type;
the sixth knockout sub-condition comprises: the code rate of the content in the to-be-eliminated content is a cold door code rate value;
the seventh knockout sub-condition comprises: and the time stamp of the content in the to-be-eliminated content is the cold time stamp.
If the second elimination condition includes at least a fifth elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the elimination unit 112 may be further configured to:
acquiring a uniform resource locator UR L of the content in the to-be-eliminated content;
and requesting a CDN center node to acquire a type field for describing the file type of the content in the to-be-eliminated content according to UR L.
If the second elimination condition includes at least a sixth elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the elimination unit 112 may be further configured to:
acquiring code rate values carried in UR L L corresponding to all accessed contents in an access log;
and determining that the code rate value with the access quantity lower than a second preset threshold value is the cold door code rate value.
If the second elimination condition includes at least a seventh elimination sub-condition, before determining whether the content in the content to be eliminated satisfies the second elimination condition, the elimination unit 112 is further configured to:
UR L L which acquires all accessed contents in the access log carries time stamps;
and determining the time stamp with the access amount lower than a third preset threshold value as the cold time stamp.
In the embodiment of the present invention, an optional implementation manner is that the hot contents form a hot queue, and the contents in the hot queue are sequentially arranged from the head of the hot queue to the tail of the hot queue according to the sequence of the historical access volumes from high to low; culling unit 112 may be specifically configured to:
determining whether the contents in the hot queue meet a first elimination condition or not in sequence according to the sequence from the tail of the hot queue to the head of the hot queue;
and if determining that a first content set meeting the first elimination condition exists in the hot content, eliminating at least one content close to the tail of the hot queue in the first content set from the hot content.
In the embodiment of the present invention, an optional implementation manner is that the content to be eliminated may be the content with the lowest historical access amount in the first content set.
In the embodiment of the present invention, an optional implementation manner is that the contents in the non-hot queue are sequentially arranged from the head of the non-hot queue to the tail of the non-hot queue according to an order from high to low of the historical access amount, or according to an order from short to long of the time interval between the latest access time and the current time; culling unit 112 may be specifically configured to:
eliminating the contents which do not meet the second elimination condition from the hot queue to the head of the non-hot queue;
the culling unit 112 may also be specifically configured to:
and eliminating at least one content at the tail part of the non-hot queue from the cache.
The embodiment of the present invention provides a cache device, which may specifically be a CDN edge node, where after the cache device divides content in a cache into hot content and non-hot content, if a first content set meeting a first elimination condition exists in the hot content, the access amount of the content in the first content set has a decreasing trend, the current or future access amount of the content in the first content set may be low, and thus the content to be eliminated in the first content set is eliminated from the trending content, the to-be-eliminated content is a subset of the first content set, so that the release speed of the cache space occupied by the hot content with high historical access amount but possibly low current or future access amount is increased, and the problem that the cache space is insufficient due to the fact that the cache space of the cache device is occupied for a long time due to the fact that the hot content with high historical access amount but possibly low current or future access amount is solved.
Example 5
The embodiment of the present invention provides a cache device, which is exemplified by a CDN edge node 100. Fig. 13 is another schematic structural diagram of a cache apparatus according to an embodiment of the present invention, which employs a general-purpose computer system structure, and program codes for executing the scheme of the present invention are stored in a memory and controlled by a processor to execute. The device for handling grants comprises a bus, a processor (502), a memory (504), and a communication interface (506).
A bus may include a path that transfers information between the various components of a computer.
Processor 502 may be a general purpose Central Processing Unit (CPU), microprocessor, Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to control the execution of programs in accordance with the present invention. The memory or memories included in the computer system may be a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, or a disk storage device. These memories are connected to the processor via a bus.
The communication interface 506 may use any transceiver or the like to communicate with other devices or communication networks, such as ethernet, Radio Access Network (RAN), wireless local area network (W L AN), etc.
Memory 504, such as RAM, holds an operating system and programs that implement aspects of the present invention. The operating system is a program for controlling the operation of other programs and managing system resources. Program code for implementing aspects of the present invention is stored in memory and controlled for execution by the processor.
The program stored in the memory 504 is for instructing the processor to perform a cache content eviction method comprising: dividing the content in the cache into popular content and non-popular content according to the historical access amount, wherein the historical access amount of any popular content is higher than that of any non-popular content;
when it is determined that a content elimination process in a cache needs to be started in a current time window, if it is determined that a first content set meeting a first elimination condition exists in the popular content, eliminating to-be-eliminated content in the first content set from the popular content, wherein the to-be-eliminated content comprises at least one content, and the first elimination condition is used for determining that the access amount of the content in the popular content is in a descending trend.
It can be understood that a cache device of this embodiment may be used to implement all functions in the method embodiments described in embodiments 1, 2, and 3, and a specific implementation process thereof may refer to the related description of the above method embodiments, which is not described herein again.
The embodiment of the present invention provides a cache device, which may specifically be a CDN edge node, where after the CDN edge node divides content in a cache into hot content and non-hot content, if a first content set meeting a first elimination condition exists in the hot content, the access amount of the content in the first content set has a decreasing trend, the current or future access amount of the content in the first content set may be low, and thus the content to be eliminated in the first content set is eliminated from the trending content, the content to be eliminated is the subset of the first content set, so that the release speed of the cache space occupied by the hot content with high historical access amount but possibly low current or future access amount is accelerated, the hot content with high historical access amount but possibly low current or future access amount is solved, the problem of insufficient cache space is caused by long-term occupation of the cache space of CDN edge nodes.
In several embodiments provided in the present application, it should be understood that the disclosed caching apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be physically included alone, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (25)

1. A cache content eviction method is characterized by comprising the following steps:
dividing the content in the cache into popular content and non-popular content according to the historical access amount, wherein the historical access amount of any popular content is higher than that of any non-popular content;
when it is determined that a content elimination process in a cache needs to be started in a current time window, if it is determined that a first content set meeting a first elimination condition exists in the popular content, eliminating to-be-eliminated content in the first content set from the popular content, wherein the to-be-eliminated content comprises at least one content, and the first elimination condition is used for determining that the access amount of the content in the popular content is in a descending trend; the first eliminating condition comprises a first eliminating sub-condition: the access quantity of the content in the hot content in the current time window is lower than the preset access quantity;
wherein the eliminating of the to-be-eliminated content in the first content set from the trending content comprises:
determining whether the content in the to-be-eliminated content meets a second elimination condition, wherein the second elimination condition is used for determining that the access amount of the content in the to-be-eliminated content is in a sharp decline trend;
eliminating the contents meeting the second elimination condition from the contents to be eliminated out of the cache;
and eliminating the contents which do not meet the second elimination condition in the contents to be eliminated from the hot contents into the non-hot contents, and eliminating at least one content in the non-hot contents from the cache.
2. The method of claim 1, further comprising:
and if the first content set meeting the first elimination condition does not exist in the popular content, eliminating at least one content in the non-popular content from the cache.
3. The method according to claim 1, wherein the determining that a content elimination procedure in the cache needs to be started within the current time window specifically comprises:
receiving an access request for acquiring content to be accessed, which is sent by a client;
determining that the content to be accessed is not stored in a cache;
when the content to be accessed meets the caching condition, determining that the residual caching space is smaller than the file size of the content to be accessed;
wherein the caching condition comprises:
the historical access amount of the content to be accessed is higher than or equal to a first preset threshold value.
4. The method of claim 3, further comprising:
and when determining that the content elimination process in the cache does not need to be started in the current time window, inserting the content to be accessed into the non-popular content, and sending the content to be accessed to the client.
5. The method of claim 1, wherein the first knockout condition further comprises a second knockout sub-condition, and/or a third knockout sub-condition, and/or a fourth knockout sub-condition;
the second knockout sub-condition comprises: the current moment is not in a preset protection period of the content in the hot content;
the third knockout sub-condition comprises: the number of the eliminated and cached contents in the hot contents in the current time window is lower than the preset elimination number;
the fourth knockout sub-condition comprises: the ratio of the number of the hot contents with the cache eliminated to the number of the non-hot contents with the cache eliminated in the current time window is lower than the preset elimination ratio.
6. The method of claim 1, wherein the second knockout condition comprises at least one of a fifth knockout sub-condition, a sixth knockout sub-condition, and a seventh knockout sub-condition;
the fifth eliminating sub-condition comprises: the file type of the content in the to-be-eliminated content is a preset file type;
the sixth knockout sub-condition comprises: the code rate of the content in the to-be-eliminated content is a cold door code rate value;
the seventh knockout sub-condition comprises: and the time stamp of the content in the to-be-eliminated content is a cold time stamp.
7. The method of claim 6, wherein the second elimination condition comprises at least the fifth elimination sub-condition, and wherein before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method further comprises:
acquiring a uniform resource locator UR L of the content in the to-be-eliminated content;
and requesting a CDN central node to acquire a type field for describing the file type of the content in the to-be-eliminated content according to the UR L.
8. The method of claim 6 or 7, wherein the second elimination condition comprises at least a sixth elimination sub-condition, and before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method further comprises:
acquiring UR L corresponding to all accessed contents in an access log, wherein the UR L carries a code rate value;
and determining that the code rate value with the access quantity lower than a second preset threshold value is the cold door code rate value.
9. The method of claim 6 or 7, wherein the second elimination condition comprises at least a seventh elimination sub-condition, and before determining whether the content in the content to be eliminated satisfies the second elimination condition, the method further comprises:
acquiring UR L of all accessed contents in an access log, wherein UR L carries a time stamp;
and determining the time stamp with the access amount lower than a third preset threshold value as the cold time stamp.
10. The method of claim 1, wherein the hot contents form a hot queue, and the contents in the hot queue are arranged from the head of the hot queue to the tail of the hot queue in sequence from high to low of historical access amount; if it is determined that a first content set meeting a first elimination condition exists in the popular content, eliminating the to-be-eliminated content in the first content set from the popular content comprises:
sequentially determining whether the contents in the hot queue meet the first elimination condition according to the sequence from the tail of the hot queue to the head of the hot queue;
and if determining that a first content set meeting a first elimination condition exists in the hot contents, eliminating at least one content close to the tail part of the hot queue in the first content set from the hot contents.
11. The method of any of claims 1, 4-7, and 10, wherein the content to be eliminated is content with the lowest historical access in the first set of content.
12. The method according to claim 1, wherein the non-hot contents form a non-hot queue, and the contents in the non-hot queue are arranged from the head of the non-hot queue to the tail of the non-hot queue in the order of historical access amount from high to low, or in the order of time interval between the latest access time and the current time from short to long; the eliminating, from the trending content, the content that does not satisfy the second elimination condition in the to-be-eliminated content to the non-trending content includes:
eliminating the contents which do not meet the second elimination condition in the contents to be eliminated from the hot queue to the head of the non-hot queue;
the evicting at least one of the non-trending content from the cache comprises:
and eliminating at least one content at the tail part of the non-hot queue out of the cache.
13. A cache apparatus, comprising:
the dividing unit is used for dividing the content in the cache into hot content and non-hot content according to the historical access amount, wherein the historical access amount of any hot content is higher than that of any non-hot content;
the content elimination unit is used for eliminating to-be-eliminated contents in the first content set from the hot contents if it is determined that a first content set meeting a first elimination condition exists in the hot contents when it is determined that a content elimination flow in a cache needs to be started in a current time window, wherein the to-be-eliminated contents comprise at least one content, and the first elimination condition is used for determining that the access amount of the content in the hot contents is in a descending trend; the first eliminating condition comprises a first eliminating sub-condition: the access quantity of the content in the hot content in the current time window is lower than the preset access quantity;
wherein the elimination unit is specifically configured to:
determining whether the content in the to-be-eliminated content meets a second elimination condition, wherein the second elimination condition is used for determining that the access amount of the content in the to-be-eliminated content is in a sharp decline trend;
eliminating the contents meeting the second elimination condition from the contents to be eliminated out of the cache;
and eliminating the contents which do not meet the second elimination condition in the contents to be eliminated from the hot contents into the non-hot contents, and eliminating at least one content in the non-hot contents from the cache.
14. The apparatus of claim 13, wherein the culling unit is further configured to:
and if the first content set meeting the first elimination condition does not exist in the popular content, eliminating at least one content in the non-popular content from the cache.
15. The apparatus according to claim 13, further comprising an interface unit, configured to receive an access request sent by a client to obtain the content to be accessed;
the elimination unit is specifically configured to:
determining that the content to be accessed received by the interface unit is not stored in a cache;
when the content to be accessed meets the caching condition, determining that the residual caching space is smaller than the file size of the content to be accessed;
wherein the caching condition comprises:
the historical access amount of the content to be accessed is higher than or equal to a first preset threshold value.
16. The apparatus of claim 15, wherein the culling unit is further configured to:
when determining that a content elimination process in a cache does not need to be started in a current time window, inserting the content to be accessed into the non-popular content;
the interface unit is further configured to send the content to be accessed to the client.
17. The apparatus of claim 13, wherein the first knockout condition further comprises a second knockout sub-condition, and/or a third knockout sub-condition, and/or a fourth knockout sub-condition;
the second knockout sub-condition comprises: the current moment is not in a preset protection period of the content in the hot content;
the third knockout sub-condition comprises: the number of the eliminated and cached contents in the hot contents in the current time window is lower than the preset elimination number;
the fourth knockout sub-condition comprises: the ratio of the number of the hot contents with the cache eliminated to the number of the non-hot contents with the cache eliminated in the current time window is lower than the preset elimination ratio.
18. The apparatus of claim 13, wherein the second knockout condition comprises at least one of a fifth knockout sub-condition, a sixth knockout sub-condition, and a seventh knockout sub-condition;
the fifth eliminating sub-condition comprises: the file type of the content in the to-be-eliminated content is a preset file type;
the sixth knockout sub-condition comprises: the code rate of the content in the to-be-eliminated content is a cold door code rate value;
the seventh knockout sub-condition comprises: and the time stamp of the content in the to-be-eliminated content is a cold time stamp.
19. The apparatus of claim 18, wherein the second elimination condition comprises at least the fifth elimination sub-condition, and wherein, prior to determining whether content in the content to be eliminated satisfies the second elimination condition, the elimination unit is further configured to:
acquiring a uniform resource locator UR L of the content in the to-be-eliminated content;
and requesting a CDN central node to acquire a type field for describing the file type of the content in the to-be-eliminated content according to the UR L.
20. The apparatus of claim 18 or 19, wherein the second elimination condition comprises at least a sixth elimination sub-condition, and wherein, prior to determining whether content in the content to be eliminated satisfies the second elimination condition, the elimination unit is further configured to:
acquiring UR L corresponding to all accessed contents in an access log, wherein the UR L carries a code rate value;
and determining that the code rate value with the access quantity lower than a second preset threshold value is the cold door code rate value.
21. The apparatus of claim 18 or 19, wherein the second elimination condition comprises at least a seventh elimination sub-condition, and wherein, prior to determining whether content in the content to be eliminated satisfies the second elimination condition, the elimination unit is further configured to:
acquiring UR L of all accessed contents in an access log, wherein UR L carries a time stamp;
and determining the time stamp with the access amount lower than a third preset threshold value as the cold time stamp.
22. The apparatus of claim 13, wherein the hot contents form a hot queue, and the contents in the hot queue are arranged from the head of the hot queue to the tail of the hot queue in sequence from high to low of the historical access amount; the elimination unit is specifically configured to:
sequentially determining whether the contents in the hot queue meet the first elimination condition according to the sequence from the tail of the hot queue to the head of the hot queue;
and if determining that a first content set meeting a first elimination condition exists in the hot contents, eliminating at least one content close to the tail part of the hot queue in the first content set from the hot contents.
23. The apparatus of any of claims 13, 16-19, and 22, wherein the content to be eliminated is content with a lowest historical access amount in the first set of content.
24. The apparatus according to claim 13, wherein the non-hot contents form a non-hot queue, and the contents in the non-hot queue are arranged from the head of the non-hot queue to the tail of the non-hot queue in the order of historical access amount from high to low, or in the order of time interval between the latest access time and the current time from short to long; the elimination unit is specifically configured to:
eliminating the contents which do not meet the second elimination condition in the contents to be eliminated from the hot queue to the head of the non-hot queue;
the elimination unit is further specifically configured to:
and eliminating at least one content at the tail part of the non-hot queue out of the cache.
25. A cache apparatus, comprising: a processor, a memory; wherein the memory has a computer readable program stored therein; the processor is configured to execute the method of any one of claims 1-12 by executing a program in the memory.
CN201510736056.8A 2015-11-02 2015-11-02 Cache content elimination method and cache device Active CN106657196B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510736056.8A CN106657196B (en) 2015-11-02 2015-11-02 Cache content elimination method and cache device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510736056.8A CN106657196B (en) 2015-11-02 2015-11-02 Cache content elimination method and cache device

Publications (2)

Publication Number Publication Date
CN106657196A CN106657196A (en) 2017-05-10
CN106657196B true CN106657196B (en) 2020-07-24

Family

ID=58811024

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510736056.8A Active CN106657196B (en) 2015-11-02 2015-11-02 Cache content elimination method and cache device

Country Status (1)

Country Link
CN (1) CN106657196B (en)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107577618B (en) * 2017-09-13 2020-09-29 武大吉奥信息技术有限公司 Three-path balanced cache elimination method and device
CN109542613A (en) * 2017-09-22 2019-03-29 中兴通讯股份有限公司 Distribution method, device and the storage medium of service dispatch in a kind of CDN node
CN107786668B (en) * 2017-11-09 2020-06-12 成都知道创宇信息技术有限公司 Weight caching website method based on CDN (content delivery network)
CN109995836B (en) 2017-12-29 2021-12-03 华为技术有限公司 Cache decision method and device
CN108173974B (en) * 2018-03-01 2021-02-12 南京邮电大学 HCModel internal cache data elimination method based on distributed cache Memcached
CN111104528B (en) * 2018-10-29 2023-05-16 浙江宇视科技有限公司 Picture acquisition method and device and client
CN109511009B (en) * 2018-12-07 2021-03-30 北京交通大学 Video online cache management method and system
CN109587528B (en) * 2018-12-24 2021-06-29 中国移动通信集团江苏有限公司 Method, device, equipment and medium for distributing CDN resources
CN111372095B (en) * 2018-12-25 2023-06-23 深圳市茁壮网络股份有限公司 Method and device for calculating heat
CN111506524B (en) * 2019-01-31 2024-01-30 华为云计算技术有限公司 Method and device for eliminating and preloading data pages in database
CN110022482B (en) * 2019-03-05 2021-07-27 咪咕视讯科技有限公司 Video playing starting method, video service system and storage medium
CN111159066A (en) * 2020-01-07 2020-05-15 杭州电子科技大学 Dynamically-adjusted cache data management and elimination method
CN111597259B (en) * 2020-05-12 2023-04-28 北京爱奇艺科技有限公司 Data storage system, method, device, electronic equipment and storage medium
CN111818142A (en) * 2020-06-28 2020-10-23 深圳市高德信通信股份有限公司 CDN cache clearing management system
CN114422522B (en) * 2020-10-13 2024-02-13 贵州白山云科技股份有限公司 Cache distribution method, device, medium and equipment
CN113382302B (en) * 2021-05-27 2022-12-27 北京达佳互联信息技术有限公司 Video determination method, device, server and storage medium
CN113590031B (en) * 2021-06-30 2023-09-12 郑州云海信息技术有限公司 Cache management method, device, equipment and computer readable storage medium
CN113949921A (en) * 2021-08-31 2022-01-18 上海二三四五网络科技有限公司 Control method and control device for short video cache cleaning
CN114390099B (en) * 2022-01-12 2023-06-02 中国联合网络通信集团有限公司 Content distribution method and edge server

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102595207A (en) * 2012-01-10 2012-07-18 深圳市同洲视讯传媒有限公司 Content distribution system, method and device
CN102819586A (en) * 2012-07-31 2012-12-12 北京网康科技有限公司 Uniform Resource Locator (URL) classifying method and equipment based on cache
CN103312776A (en) * 2013-05-08 2013-09-18 青岛海信传媒网络技术有限公司 Method and device for caching contents of videos by edge node server
CN104239233A (en) * 2014-09-19 2014-12-24 华为技术有限公司 Cache managing method, cache managing device and cache managing equipment
CN104750715A (en) * 2013-12-27 2015-07-01 中国移动通信集团公司 Data elimination method, device and system in caching system and related server equipment
CN104935648A (en) * 2015-06-03 2015-09-23 北京快网科技有限公司 High-cost-performance CDN system, and file pre-push and fragment buffer memory methods

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102595207A (en) * 2012-01-10 2012-07-18 深圳市同洲视讯传媒有限公司 Content distribution system, method and device
CN102819586A (en) * 2012-07-31 2012-12-12 北京网康科技有限公司 Uniform Resource Locator (URL) classifying method and equipment based on cache
CN103312776A (en) * 2013-05-08 2013-09-18 青岛海信传媒网络技术有限公司 Method and device for caching contents of videos by edge node server
CN104750715A (en) * 2013-12-27 2015-07-01 中国移动通信集团公司 Data elimination method, device and system in caching system and related server equipment
CN104239233A (en) * 2014-09-19 2014-12-24 华为技术有限公司 Cache managing method, cache managing device and cache managing equipment
CN104935648A (en) * 2015-06-03 2015-09-23 北京快网科技有限公司 High-cost-performance CDN system, and file pre-push and fragment buffer memory methods

Also Published As

Publication number Publication date
CN106657196A (en) 2017-05-10

Similar Documents

Publication Publication Date Title
CN106657196B (en) Cache content elimination method and cache device
US20150039593A1 (en) Pre-delivery of content to a user device
US7860993B2 (en) Streaming media content delivery system and method for delivering streaming content
US20020092026A1 (en) Method and apparatus for broadcast delivery of content to a client-side cache based on user preferences
CN109299144B (en) Data processing method, device and system and application server
US8745262B2 (en) Adaptive network content delivery system
US9015269B2 (en) Methods and systems for notifying a server with cache information and for serving resources based on it
US9906595B2 (en) Content source discovery
CN109600437B (en) Downloading method of streaming media resource and cache server
CN109190070B (en) Data processing method, device and system and application server
US20120054295A1 (en) Method and apparatus for providing or acquiring the contents of a network resource for a mobile device
WO2005045708A1 (en) Method for prioritizing segments of multimedia content in a proxy cache
US11032394B1 (en) Caching techniques
CN109033462B (en) Method and system for determining low frequency data items in a storage device for large data storage
Gessert et al. Quaestor: Query web caching for database-as-a-service providers
KR101066872B1 (en) System and method for content delivery using cache server, and cache server thereof
CN107659626B (en) Temporary metadata oriented separation storage method
CN107682281B (en) SDN switch and application management method thereof
EP3274844B1 (en) Hierarchical cost based caching for online media
WO2018153236A1 (en) Method and apparatus for accelerating dynamic resource access based on api request, medium, and device
CN112019451A (en) Bandwidth allocation method, debugging network element, local cache server and computing equipment
EP3207457B1 (en) Hierarchical caching for online media
CN113507522A (en) Method and system for improving hit rate of PCDN (Primary Contourlet distribution) network requests
CN113612823A (en) Cache duration adjusting method and device for content distribution network and storage medium
CN112533004B (en) Program information publishing method, system, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20211222

Address after: 450046 Floor 9, building 1, Zhengshang Boya Plaza, Longzihu wisdom Island, Zhengdong New Area, Zhengzhou City, Henan Province

Patentee after: Super fusion Digital Technology Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd.

TR01 Transfer of patent right