WO2019057212A1

WO2019057212A1 - Method, apparatus and device for scheduling service within cdn node, and storage medium

Info

Publication number: WO2019057212A1
Application number: PCT/CN2018/107387
Authority: WO
Inventors: 陈伟
Original assignee: 中兴通讯股份有限公司
Priority date: 2017-09-22
Filing date: 2018-09-25
Publication date: 2019-03-28
Also published as: CN109542613A

Abstract

A method, apparatus and device for scheduling a service within a content distribution network (CDN) node, and a storage medium. The method comprises: when receiving a request message for acquiring a content resource, extracting, from the request message, a uniform resource locator of the content resource (110); according to the uniform resource locator, determining an identification (ID) of the content resource (120); and according to the ID of the content resource, and according to a preset selection algorithm, selecting, from among all cache servers within the CDN node, a root cache server used for acquiring the content resource (130).

Description

Method, device, device and storage medium for service scheduling in CDN node

The present application claims the priority of the Chinese Patent Application, filed on Sep. 22, 2011, the filing date of

Technical field

The present disclosure relates to the field of wireless mobile communications, for example, to a content delivery network (CDN) intra-node intra-service scheduling method, apparatus, device, and storage medium.

Background technique

With the rapid development of the Internet, the CDN system has to carry more and more content and more and more storage content, including Android market, Apple market application download (some applications exceed 500M), massive video online playback, And 4096 × 2160 pixel resolution (4K resolution), virtual reality (Virtual Reality, VR) and other high-definition video gradually increased.

The above business development puts forward the following higher requirements for scheduling design and storage efficiency in CDN nodes, including:

1. The scheduling of multiple cache servers in a CDN node must have high performance: the traditional algorithm based on file distribution centralized record update can not meet such requirements, because the content distribution index of Server Load Balancing (SLB) is the same. The content storage of multiple cache servers is consistent, and a high data consistency algorithm is designed to efficiently process the index of massive content (the latency of SLB scheduling is very low, such as less than 1ms).

2. The number of levels of content of multiple cache servers in a CDN node is increased: in the traditional sense, a large file is generally a single file, and one or more cache servers exist in the file content. However, since Apple's large-scale adoption of the HyperText Transfer Protocol Live Streaming (HLS) protocol, many video content is stored in the form of fragmented files. In the past, a movie recorded an index on the content storage address, and now based on the score. The method record number of the slice can be 300 times of the original. If the record mode is based on the centralized content index, the load of the SLB and the content consistency of the content in the SLB and the cache server are greatly increased.

Summary of the invention

The present disclosure provides a CDN intra-node service scheduling method and apparatus, in order to solve the problem that the load balancing load in the CDN node is high, the load balancing, and the cache server are difficult to maintain data consistency in the mass content scenario. Equipment and storage media.

In an embodiment, the CDN intra-node service scheduling method provided by the present disclosure is applied to a cache server, including:

When receiving the request message for acquiring the content resource, extracting a uniform resource locator of the content resource in the request message;

Determining an identifier (Identity, ID) of the content resource according to the uniform resource locator;

And selecting, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.

In an embodiment, the present disclosure further provides a CDN intra-node service scheduling apparatus, including: a receiving module, a first computing module, and a second computing module:

The receiving module is configured to: when receiving the request message for acquiring the content resource, extract a uniform resource locator of the content resource in the request message;

The first calculating module is configured to determine an ID of the content resource according to the uniform resource locator;

The second calculating module is configured to select, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.

In an embodiment, the present disclosure also provides a CDN intra-node service scheduling device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, the computer program being The processor performs the above-described CDN intra-node service scheduling method when executed by the processor.

In an embodiment, the present disclosure further provides a storage medium on which a CDN intra-node service scheduler is stored, and the CDN intra-node service scheduler is implemented by the processor to implement the above-mentioned CDN intra-node service. Scheduling method.

In an embodiment, the storage medium is a computer readable storage medium.

DRAWINGS

1 is a schematic flowchart of a method for allocating a service scheduling in a CDN node according to an embodiment of the present disclosure;

2 is a schematic structural diagram of a service scheduling allocation device in a CDN node according to an embodiment of the present disclosure;

3 is a schematic diagram showing the structure of a CDN node and its relationship with an external network element;

4 is a schematic flowchart of a service scheduling allocation method in a CDN node in Example 1;

5 is a schematic flowchart of a service scheduling allocation method in a CDN node in Example 2;

FIG. 6 is a schematic structural diagram of a service scheduling distribution device in a CDN node according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure will be described below with reference to the drawings.

The present disclosure provides a CDN intra-node service scheduling method and apparatus, in order to solve the problem that the load balancing load in the CDN node is high, the load balancing, and the cache server are difficult to maintain data consistency in the mass content scenario. Apparatus and Storage Media The following disclosure is described with reference to the accompanying drawings. The embodiments described herein are merely illustrative of the disclosure and are not limiting of the disclosure.

In an embodiment, a CDN node can be understood as a logical concept rather than a physical entity. A CDN node may include load balancing, a cluster of cache servers, and one or more auxiliary modules (eg, alarm modules) that implement non-core functions. The cache server cluster may include one or more cache servers, that is, when one cache server has limited capabilities, multiple cache servers may be employed, and the functions of the multiple cache servers are the same as one cache server.

According to an embodiment of the present disclosure, a method for scheduling a service in a CDN node is provided, which is applied to a cache server cache. FIG. 1 is a schematic flowchart of a service scheduling allocation method in a CDN node according to an embodiment of the present disclosure. The CDN intra-node service scheduling method of the disclosed embodiment includes:

Step 110: When receiving the request message for acquiring the content resource, extract the uniform resource locator of the content resource in the request message.

As an implementation manner of the embodiment of the present disclosure, before receiving the request message for acquiring the content resource, the cache server further includes: the SLB receiving the Transmission Control Protocol (TCP) link establishment request sent by the user, according to the TCP connection situation. (including but not limited to the number of connections, the frequency of requests, the number of active connections, etc.), load balancing user requests to a cache server (assumed to be A).

Step 120: Determine an ID of the content resource according to the uniform resource locator.

In an embodiment, the determining the ID of the content resource according to the uniform resource locator includes: extracting domain name information in the uniform resource locator; and using a preset according to the uniform resource locator The content matching rule calculates the content resource information; and obtains the ID of the content resource according to the domain name information and the content resource information. That is, the user uniform resource locator (URL) request, according to a preset content matching rule (such as a regular expression) to extract the content ID, the following formula can be used: content resource ID = regular match (URL, the Regular expression for domain name content resources).

Step 130: Select, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.

In an embodiment, selecting, according to the ID of the content resource, all the cache servers in the CDN node according to a preset selection algorithm, including: using a preset consistency hash algorithm, in the A root cache server that obtains the ID mapping of the content resource in a collection of all cache servers in the CDN node. That is, according to the consistent HASH algorithm, the ID of the content resource is uniquely mapped to one of the back-end cache servers, which is called the root cache server (assumed to be B), and the following formula can be adopted: the root cache server ID of the content resource ID. = Consistent hash algorithm (content resource ID, cache server collection).

As an implementation manner of the embodiment of the present disclosure, after selecting a root cache server for acquiring the content resource, the method further includes:

Determining whether the cache server and the root cache server are the same server;

If the cache server and the root cache server are the same server, searching the cache server, acquiring the content resource, and performing a request response;

And if the cache server is not the same server as the root cache server, sending the request message to the root cache server, and performing a request response according to the content resource fed back by the root cache server.

In an embodiment, the searching the cache server to obtain the content resource comprises: sending the request message to a superior node or a source station when the content server is not in the cache server, to obtain The content resource and cache the content resource. That is, when the cache server and the root cache server are the same server, and when the content resource is not in the cache server, the source is returned to the upper node or the source station to obtain the content resource, and the cache is obtained. The content resource. In the CDN intra-node service scheduling method of the embodiment of the present disclosure, the method further includes: when the cache server and the root cache server are the same server, and when the content resource is included in the cache server, Content resources are sent directly to the user.

In an embodiment, the superior node can be understood as a back-end server, ie, a database server, and the source station can be understood as a location where the data source of the CDN is stored.

In an embodiment, the first CDN node, the second CDN node, and the third CDN node are included, and when the content resource requested by the terminal does not exist in the first CDN node, the second CDN node is searched, and when the second CDN node is also If there is no content resource requested by the terminal, the third CDN node may be searched. If the content resource requested by the terminal does not exist in the third CDN node, the content resource is obtained from the source station, and the source station is the source of the content resource. If the content resource requested by the terminal exists in the third CDN node, the content resource is provided to the terminal, and the node that wants the content resource to the next-level CDN node is the upper node, that is, the second CDN node is the upper node of the first CDN node. The third CDN node is a superior node of the second CDN node.

In an embodiment of the present disclosure, when the cache server and the root cache server are not the same server, sending the request message to the root cache server, and according to the feedback of the root cache server Content resources, request response. That is, when the cache server and the root cache server are not the same server, the source is returned to the root cache server, and the request response is performed according to the content resource fed back by the root cache server. In the CDN intra-node service scheduling method of the embodiment of the present disclosure, the method further includes: when the source-to-root cache server still cannot obtain the content resource, continue to return the source to the upper-level node or the source station to obtain the content resource, And caching the content resource to the root cache server. In the CDN intra-node service scheduling method of the embodiment of the present disclosure, the method further includes: when the cache server and the root cache server are not the same server, but the content server is included in the cache server, The content resource is sent directly to the user.

In the CDN intra-node service scheduling method of the embodiment of the present disclosure, the method further includes: counting the number of times the content resource is accessed, and determining whether the content resource is hot content according to the number of times the content resource is accessed; The content resource is hot content, and when the hot content fed back by the root cache server is received, the hot content is cached.

For example, the number of times the content resource is accessed is counted, and whether the content resource is hot content is calculated according to the number of times the content resource is accessed, and the following manner may be adopted:

Method 1: Calculate whether the content is a hotspot according to the time dimension and the ID dimension of the content resource.

Method 2: Calculate the content access traffic according to the time dimension and the ID dimension of the content resource, and calculate whether the content is hot.

In an embodiment, for a small file (file size is less than 2M), it is implemented according to mode 1. For a large file (file size is greater than or equal to 2M), it is implemented according to mode 2.

In an embodiment, the following formula is used to calculate whether the content is hot content: whether it is a hot content=hot content calculation algorithm (domain name, content ID, number of services, service flow, time).

In an embodiment of the present disclosure, step 130 may also be represented as the following steps:

(1) When the cache server A has a content resource, the content resource is directly returned to the user;

(2) When the cache server A has no content resources, if A and B are the same cache server, proceed to step (3);

If A and B are different cache servers, there are two cases: the first case is: when the content resource is non-hotspot content: the A proxy returns the source to B, and the content is not cached when the content passes the A; The situation is: when the content resource is hot content: when the content passes A, the content is cached.

(3) Return the source to the superior node/source station and cache this content to B.

The CDN intra-node service scheduling method provided by the embodiment of the present disclosure can fully utilize the algorithm mode without recording and synchronizing the content in one or more cache server indexes when performing internal content scheduling of the CDN node, thereby greatly reducing the SLB scheduling load. It avoids the problems caused by inconsistent content index, unsynchronized synchronization, and SLB abnormality. At the same time, it can improve the load balance of one or more cache servers in the CDN node, reduce the overheating of a cache server due to hot spots, and avoid the traditional adoption. The content consistency record is generated by the scheme of accurate content index recording in the mass content scene.

In an embodiment, the embodiment of the present disclosure further provides a CDN intra-node service scheduling apparatus, which is applied to a cache server. FIG. 2 is a schematic structural diagram of a service scheduling distribution apparatus in a CDN node according to an embodiment of the present disclosure, as shown in FIG. 2 The CDN intra-node service scheduling apparatus of the embodiment of the present disclosure includes a receiving module 20, a first computing module 22, and a second computing module 24:

The receiving module 20 is configured to: when receiving the request message for acquiring the content resource, extract the uniform resource locator of the content resource in the request message;

The first calculating module 22 is configured to determine an ID of the content resource according to the uniform resource locator;

The second calculating module 24 is configured to select, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.

In an embodiment, in the CDN intra-node service scheduling apparatus of the present disclosure, the content proxy return source module 26 is further included:

The content proxy back to the source module 26 is configured to determine whether the cache server and the root cache server are the same server;

In an embodiment, in the CDN intra-node service scheduling apparatus of the present disclosure, the content proxy back source module 26 is configured to:

When the content resource is not in the cache server, the request message is sent to a superior node or a source station to obtain the content resource, and the content resource is cached.

In an embodiment, the CDN in-node service scheduling apparatus of the present disclosure further includes a content hotspot statistics module 28 and a content cache module 30:

The content hotspot statistic module 28 is configured to count the number of times the content resource is accessed, and calculate whether the content resource is hot content according to the number of times the content resource is accessed;

The content caching module 30 is configured to cache the hot content when the content resource is hot content and when receiving the hot content fed back by the root cache server.

In an embodiment, in the CDN intra-node service scheduling apparatus of the present disclosure, the first computing module 22 is configured to:

Extracting domain name information in the uniform resource locator;

Calculating content resource information by using a preset content matching rule according to the uniform resource locator;

And obtaining an ID of the content resource according to the domain name information and the content resource information.

In an embodiment, in the CDN intra-node service scheduling apparatus of the present disclosure, the second computing module 24 is configured to:

The root cache server of the ID mapping of the content resource is obtained in a set of all cache servers in the CDN node by using a preset consistency hash algorithm.

FIG. 3 is a schematic diagram showing the structure of a CDN node and its relationship with an external network element. As shown in FIG. 3, the CDB and the N cache servers (N is a positive integer greater than or equal to 1) are included in the CDN node, wherein a cache server is included as an example, and the content deduplication module is included in the cache server ( The first calculation module 22 and the second calculation module 24), the content hotspot statistics module 28, the content proxy return source module 26, the content cache module 30, and the content replication module are included.

The CDN intra-node service scheduling apparatus provided by the embodiment of the present disclosure can fully utilize an algorithm manner without recording and synchronizing content actually in one or more cache server indexes when performing internal content scheduling of a CDN node, thereby greatly reducing the SLB scheduling load. It avoids the problems caused by inconsistent content index, unsynchronized synchronization, and SLB abnormality. At the same time, it can improve the load balance of one or more cache servers in the CDN node, reduce the overheating of a cache server due to hot spots, and avoid the traditional adoption. The content consistency record is generated by the scheme of accurate content index recording in the mass content scene.

For a more detailed description of the CDN intra-node service scheduling method of the embodiment of the present disclosure, Example 1 and Example 2 are given. FIG. 4 is a schematic flowchart of a service scheduling allocation method in a CDN node in Example 1, as shown in FIG. 4, including the following step:

step one:

The user 211 initiates a TCP link establishment request to the SLB of the CDN node, and the SLB receives the user to send a TCP link establishment request, and loads the user request to the cache server 231 according to the TCP connection situation (for example, the number of connections, the request frequency, and the number of active connections).

Step two:

The cache server 231 receives the Hypertext Transfer Protocol (HTTP) information that the user 211 initiates to acquire the content resource request from the CDN node, including the request URI, and the domain name Domain. E.g:

https://ss0.bdstatic.com/5aV1bjqh_Q23odCf/static/superman/img/logo/bd_logo1_31bdc765.png

Step three:

The cache server 231 calculates a unique ID of the content resource corresponding to the request according to the URI request, the domain name information, and the pre-configured regular expression of the corresponding content resource of the domain name. For example, ss0.bdstatic.com/5aV1bjqh_Q23odCf/static/superman/img/logo/bd_logo1_31bdc765.png is used as the key KEY. The unique ID of the content resource is calculated by the message digest algorithm MD5 as fb6667636553c773d7d3fd13bc6d4d4e.

The cache server 231 determines whether the content of the ID is fb6667636553c773d7d3fd13bc6d4d4e is hot content by counting the number of accesses in the content unit time and the predefined hotspot calculation algorithm. Assume that the content whose ID is fb6667636553c773d7d3fd13bc6d4d4e is non-hotspot content.

The cache server 231 uses the consistency hash algorithm to calculate the root cache server ID of the content as cacheid3 by using the fb6667636553c773d7d3fd13bc6d4d4e, cacheid1, cacheid2, and cacheid3, that is, the root cache server is the cache server 231.

Step four:

When the cache server 231 does not cache the content of the ID fb6667636553c773d7d3fd13bc6d4d4e, the upper CDN node or the source station 241 acquires the content resource in real time and returns the content resource to the user 211. At the same time, the cache server 231 also caches the content resources locally.

FIG. 5 is a schematic flowchart of a method for allocating a service scheduling in a CDN node in the example 2, as shown in FIG. 5, including the following steps:

step one:

The user 211 initiates a TCP link establishment request to the SLB of the CDN node, and the SLB receives the TCP connection request from the user, and load balances the user request to the cache server 232 according to the TCP connection situation (for example, the number of connections, the request frequency, and the number of active connections).

Step two:

The cache server 232 receives the user 211 to initiate a request for acquiring a content resource to the CDN node, including requesting a URI, requesting HTTP information such as a domain name. E.g:

Step three:

The cache server 232 calculates a unique ID of the content resource corresponding to the request according to the URI request, the domain name information, and the pre-configured regular expression of the corresponding content resource of the domain name. For example, use ss0.bdstatic.com/5aV1bjqh_Q23odCf/static/superman/img/logo/bd_logo1_31bdc765.png to do KEY. The unique ID of the content resource calculated by MD5 is fb6667636553c773d7d3fd13bc6d4d4e.

The cache server 232 calculates the content of the ID as fb6667636553c773d7d3fd13bc6d4d4e as the hot content by the hot content statistics.

The cache server 232 uses the consistency hash algorithm to calculate the root cache server of the content as the cache server 231 by calculating the information such as the cache server ID list and the like by fb6667636553c773d7d3fd13bc6d4d4e.

Step four:

The cache server 232 initiates a proxy return request to the cache server 231.

Step five:

When the cache server 231 does not cache the content of the ID bb6667636553c773d7d3fd13bc6d4d4e, the upper CDN node or the source station 241 acquires the content resource in real time, and returns the content resource to the cache server 231. At the same time, the cache server 231 also caches the content locally.

Step six:

The cache server 232 returns the content data to the user and caches the content fb6667636553c773d7d3fd13bc6d4d4e locally.

FIG. 6 is a schematic structural diagram of a service scheduling distribution device in a CDN node according to an embodiment of the present disclosure. The present disclosure also provides a CDN intra-node service scheduling device, including a memory 610 and a processor 620 having stored thereon a computer program executable on the processor 620, the computer program being processed The CDN intra-node service scheduling method as described in the embodiment of the present disclosure is implemented when the device is executed.

In an embodiment, the present disclosure provides a computer readable storage medium having a CDN intra-node service scheduler stored thereon, the CDN intra-node service scheduler being implemented by a processor to implement the above Service scheduling method within a CDN node.

The CDN intra-node service scheduling method, apparatus, device and computer readable storage medium provided by the embodiments of the present disclosure can fully utilize an algorithm manner without recording and synchronizing content actually in one or more caches when performing internal content scheduling of a CDN node. The server index greatly reduces the SLB scheduling load, avoids the problems caused by content index inconsistency, unsynchronized synchronization, and SLB exceptions. At the same time, it can improve the load balance of one or more cache servers in the CDN node and reduce the sudden hotspots. This causes a cache server to overheat and circumvents the difficulty of content consistency caused by the traditional scheme of using accurate content index records in massive content scenarios.

Claims

A content distribution network CDN node service scheduling method, applied to a cache server, comprising:

When receiving the request message for acquiring the content resource, extracting a uniform resource locator of the content resource in the request message;

Determining, according to the uniform resource locator, an identifier ID of the content resource;

And selecting, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.
The method of claim 1, after obtaining the root cache server configured to acquire the content resource, further comprising:

Determining whether the cache server and the root cache server are the same server;

If the cache server and the root cache server are the same server, searching the cache server, acquiring the content resource, and performing a request response;

And if the cache server is not the same server as the root cache server, sending the request message to the root cache server, and performing a request response according to the content resource fed back by the root cache server.
The method of claim 2, wherein the finding the cache server to obtain the content resource comprises:

When the content resource is not in the cache server, the request message is sent to a superior node or a source station to obtain the content resource, and the content resource is cached.
The method of claim 2 further comprising:

Counting the number of times the content resource is accessed, and calculating whether the content resource is hot content according to the number of times the content resource is accessed;

When the content resource is hot content, and when receiving the hot content fed back by the root cache server, the hot content is cached.
The method of claim 1, wherein the determining the ID of the content resource according to the uniform resource locator comprises:

Extracting domain name information in the uniform resource locator;

Calculating content resource information by using a preset content matching rule according to the uniform resource locator;

And obtaining an ID of the content resource according to the domain name information and the content resource information.
The method of claim 1, wherein selecting, according to the ID of the content resource, all cache servers in the CDN node according to a preset selection algorithm, including:

The root cache server of the ID mapping of the content resource is obtained in a set of all cache servers in the CDN node by using a preset consistency hash algorithm.
A content distribution network CDN node service scheduling device includes: a receiving module, a first computing module, and a second computing module:

The receiving module is configured to: when receiving the request message for acquiring the content resource, extract a uniform resource locator of the content resource in the request message;

The first calculating module is configured to determine an identifier ID of the content resource according to the uniform resource locator;

The second calculating module is configured to select, according to the ID of the content resource, a root cache server for acquiring the content resource in all cache servers in the CDN node according to a preset selection algorithm.
The apparatus of claim 7 further comprising a content agent return source module;

The content proxy back to the source module is configured to determine whether the cache server and the root cache server are the same server;

If the cache server and the root cache server are the same server, searching the cache server, acquiring the content resource, and performing a request response;

And if the cache server is not the same server as the root cache server, sending the request message to the root cache server, and performing a request response according to the content resource fed back by the root cache server.
A content distribution network CDN intra-node service scheduling device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the computer program being implemented by the processor The CDN intra-node service scheduling method according to any one of claims 1 to 6.
A storage medium storing a CDN intra-node service scheduler, wherein the CDN intra-node service scheduler is executed by a processor to implement the CDN intra-node service scheduling according to any one of claims 1 to 6. method.