WO2012152824A1 - Method for managing the infrastructure of a content distribution network service in an isp network and such an infrastructure - Google Patents
Method for managing the infrastructure of a content distribution network service in an isp network and such an infrastructure Download PDFInfo
- Publication number
- WO2012152824A1 WO2012152824A1 PCT/EP2012/058527 EP2012058527W WO2012152824A1 WO 2012152824 A1 WO2012152824 A1 WO 2012152824A1 EP 2012058527 W EP2012058527 W EP 2012058527W WO 2012152824 A1 WO2012152824 A1 WO 2012152824A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- cdn
- region
- dns
- content
- per
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 239000008186 active pharmaceutical agent Substances 0.000 claims abstract description 37
- 238000005192 partition Methods 0.000 claims description 23
- 238000001824 photoionisation detection Methods 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 8
- 238000007726 management method Methods 0.000 claims description 5
- 230000003993 interaction Effects 0.000 description 24
- 239000000306 component Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 238000012805 post-processing Methods 0.000 description 4
- 230000000644 propagated effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004366 reverse phase liquid chromatography Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101150012579 ADSL gene Proteins 0.000 description 1
- 102100020775 Adenylosuccinate lyase Human genes 0.000 description 1
- 108700040193 Adenylosuccinate lyases Proteins 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/508—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement
- H04L41/509—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement wherein the managed service relates to media content delivery, e.g. audio, video or TV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/535—Tracking the activity of the user
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/029—Location-based management or tracking services
Definitions
- the present invention generally relates, in a first aspect, to a method for managing the infrastructure of a Content Distribution Network (CDN) service in an ISP network, and more particularly to a method further comprising using a CDN manager to administer one or more regions in an operating business.
- CDN Content Distribution Network
- a second aspect of the invention relates to an infrastructure of a CDN service in an ISP network comprising a CDN manager implementing the method of the first aspect.
- CDNs Content Delivery Networks
- CDNs are a network of computers that contain copies of data, placed at various points in the network to speed up access to data for clients throughout the network [1 ].
- the key idea here is to ensure that end users don't all go to the central sever to access data. Rather, the end user accesses a copy of the data close by so as to avoid causing network bandwidth and computation bottleneck at the central server.
- CDNs have taken off in a big way in the last few years. CDNs can dynamically distribute content to different points of the edge network and manage network load and optimize capacity per customer. CDNs can offer content availability in the face of power, network and hardware outages.
- Content delivery networks have been around for over a decade. There are several architectures for building and operating such networks. Akamai for instance, has several tens of thousands of servers spread across many remote PoPs. Akamai uses a hierarchy of DNS servers to identify the content servers that can best serve the requested content.
- Web caches are also used to store the most popular content on servers that have the most number of client requests. Web caches reduce server load and improves the end user response time for content that is stored in the cache.
- the customer Under Amazon Cloudfront, the customer first creates an S3 bucket that will serve as the origin server for the content. The customer then places the content in the S3 bucket and makes the content publicly readable. As a next step, the CDN customer creates a cloudfront distribution and the infrastructure generates a unique distribution ID and a cloudfront domain name. The CDN customer can then embed the URLs in their website so that when an end user requests content, cloudfront will receive the appropriate URL that will be used to serve the content.
- Cloudfront offers HTTP for serving basic files and RTMP streaming of media files. Cloudfront also allows a CDN customer to use their custom domain names and use of origin servers located at the customer's premises.
- Limelight uses a small number of large datacenters connected by a high speed, high capacity optical network. These datacenters are strategically located to connect with access networks at multiple locations worldwide.
- Limelight uses a DNS redirection mechanism that maps requests to an end point in one of its datacenters. For instance, a DNS query of www.shrek3.com is first resolved to drmwrks.vo.llnwd.net via the public DNS infrastructure [2]. This is then resolved to a specific end point IP address in a datacenter by Limelight's private DNS infrastructure.
- Limelight also allows customers to directly use Limelight hostnames on their websites. For example, customers downloading movies from Amazon Unbox may find that their request is answered by Amazon-xxx.vo.llnwd.net (where xxx is an integer) [2].
- Limelight and Amazon Cloudfront have a small number of large datacenters that connect via a global network.
- Highwinds Network's architecture has many small globally distributed datacenters connect via a global network.
- a server in the datacenter (of the hosting CDN network) closest to the requesting client is identified to deliver the content.
- building such large and overloaded datacenters involve significant capital expenditure and yet, don't fully utilize the flexibility offered by existing network infrastructure.
- P2P based solutions use unreliable bandwidth at end users, who have different resource constraints depending on the application they run on their machines to attempt to deliver content reliably. Reliable content delivery is impossible on a large scale in such a scenario.
- the present invention provides, in a first aspect, a method for managing the infrastructure of a Content Distribution Network service in an ISP network, which, contrary to known proposals, comprises using a CDN manager to administer one or more regions in an operating business, or OB by means of one or more Application Programming Interfaces, APIs, hosted by the CDN manager.
- a second aspect of the invention relates to an infrastructure of a CDN service in an ISP network that comprises a CDN manager hosting one or more APIs for implementing the method of the first aspect of the invention according to several embodiments.
- Point of Presence A point-of-presence is an artificial demarcation or interface point between two communication entities. It is an access point to the Internet that houses servers, switches, routers and call aggregators. ISPs typically have multiple PoPs.
- ISP DNS Resolver Residential users connect to an ISP. Any request to resolve an address is sent to a DNS resolver maintained by the ISP. The ISP DNS resolver will send the DNS request to one or more DNS servers within the ISP's administrative domain.
- a bucket is a logical container for a customer.
- a bucket either makes a link between origin server URL and CDN URL or it may contain the content itself (that is ftp-ed into the bucket at the entryPoint).
- An end point will replicate files from the origin server to files in the bucket.
- Each file in a bucket may be mapped to exactly one file in the origin server.
- a bucket has several attributes associated with it. These attributes are of two kinds (i) file-system based (much like the attributes of a file under Unix) and (ii) for content distribution, e.g. duration that a piece of content is valid, geo- blocking of content etc. Mechanisms are also in place to ensure that new versions of the content at the origin server get pushed to the bucket at the endpoints and old versions are removed.
- a customer may create as many buckets as she wants.
- a bucket is really a directory that contains media files.
- a bucket may contain sub-directories and files within those sub-directories.
- Geo-location It is the identification of real-world geographic location of an Internet connected device.
- the device may be a computer, mobile device or an appliance that allows for connection to the Internet for an end user.
- the IP-address geo-location data can include information such as country, region, city, zip code, latitude / longitude of a user.
- Consistent Hashing This method provides hash-table functionality in such a way that adding or removing a slot does not significantly alter the mapping of keys to slots. Consistent hashing is a way of distributing requests among a large and changing population of web servers. The addition of removal of a web server does not significantly alter the load on the other servers.
- MD5 In cryptography, MD5 is a widely used cryptographic function with a 128- bit hash value. MD5 is widely used to test the integrity of the files. MD5 is typically expressed as a hexadecimal number.
- OB Operating Business
- An OB is an arbitrary geographic area in which the TID's CDN is installed. An OB may operate in more than one region. A region is an arbitrary geographic area and may represent a country, or part of a country or even a set of countries. An OB may consist of more than one region. An OB may be composed of one or more ISPs. An OB has exactly one instance of DNS and Topology Server.
- PID Partition ID: This is merely a mapping of IP prefixes at the AS level to integers. This is a one-to-one mapping. This is a very coarse partitioning of IP prefixes.
- DNS Domain Name Service
- DNS is a service that translates domain names into IP addresses. DNS resolution is hierarchical. If one DNS server does not know how to translate a domain name, it asks other DNS servers starting with the root DNS server until it finds the DNS server that can resolve the domain name.
- CDN Content Delivery Network
- This refers to a system of nodes (or computers) that contain copies of customer content that is stored and placed at various points in a network (or public Internet).
- content is replicated at various points in the network, bandwidth is better utilized throughout the network and users have faster access times to content. This way, the origin server that holds the original copy of the content is not a bottleneck.
- URL Uniform Resource Locator
- URL redirection is also known as URL forwarding.
- a page may need redirection if (1 ) its domain name changed, (2) creating meaningful aliases for long or frequently changing URLs (3) spell errors from the user when typing a domain name (4) manipulating visitors etc.
- a typical redirection service is one that redirects users to the desired content.
- a redirection link can be used as a permanent address for content that frequently changes hosts (much like DNS).
- Video on Demand These are systems that allow users to select and watch video streams. These systems may stream content via a set-top-box, a computer (allowing for content to be viewed in real-time) or allow the file to be downloaded before viewing.
- HTTP Hypertext Transfer Protocol
- ISP Internet Service Provider
- ISP Internet Service Provider
- the ISP may use dial-up, DSL, cable modem or high speed interconnects to do this.
- LRU Least Recently Used It is an example of replacement algorithm or a replacement policy. LRU discards the least recently used items first. Often used to pre- place caches, in this invention, it is used this to replace files on a disk.
- API Application Programming Interface
- Web APIs or web service
- An A record is a 32 bit IPv4 address. It is used to map hostnames to an IP address of the host.
- An NS (Name Server) record delegates a region to use a given authoritative name server.
- Figure 1 shows a typical OB setup with all components of the CDN infrastructure of the second aspect of the invention. Note that the log processing and CDN monitoring infrastructure may be operated on a separate network.
- FIG. 2 shows a deployment of the service provider's CDN.
- Each Operating Business (OB) runs a CDN that connects to the Internet backbone, according to an embodiment of the invention.
- Figure 3 shows the interaction between CDN customer and the service provider's CDN infrastructure on uploading content, according to the method of the first aspect of the invention.
- Figure 4 illustrates an example that shows all the interactions in a service provider's CDN infrastructure. More exactly, this figure shows all the interactions of a CDN customer, end users and interactions between the components of the CDN infrastructure, as per an embodiment of the method and infrastructure of the invention.
- FIG. 5 shows a DNS resolution in a service provider's CDN infrastructure as per an embodiment of the first aspect of the invention.
- the DNS resolution entails two DNS queries and one HTTP redirect.
- the service provider's CDN infrastructure of the second aspect of the invention comprises several embodiments.
- An embodiment includes a CDN manager hosting one or more APIs for implementing the method of the first aspect of the invention.
- Other elements of the CDN infrastructure are: entry point (or publishing point), end point, tracker, origin server, topology server, live splitter, DNS server and log server.
- entry point or publishing point
- end point tracker
- origin server topology server
- live splitter live splitter
- DNS server live splitter
- the elements of a CDN service according to the method and deployment scenario of the CDN infrastructure of the invention may be deep within an ISP network. This allows the ISP to host content servers (or end points) at the edge and as close as possible to the end users as needed. For instance, end points could be hosted at same location as the DSLAMs.
- the goal of a CDN is to move content from origin servers to end points that are close to requesting end users. Thus, content may be served with lowest delay to requesting end users.
- the method of the first aspect of the invention further comprises different CDN elements interacting with one another according to the several embodiments of the invention for deployment of a CDN infrastructure.
- the CDN infrastructure of the second aspect and also described in the method of the first aspect comprises of CDN elements that are part of a typical OB deployment scenario.
- Other embodiments of the invention are: the deployment of CDN elements in a region, a multiple OB deployment scenario for entities of a CDN service.
- Another embodiment of the method of the invention comprises using the topology server to describe the topology at an OB level for creating an abstraction of the topology in a region and/or an abstraction for connecting two regions within the same OB.
- the method comprises providing steps for interaction between different types of CDN elements, the CDN customer (content provider or owner) and end-users (requesting content).
- each of the CDN elements is described first.
- the CDN design is described.
- a deployment scenario of an OB and how the deployment may be extended to multiple OBs is also detailed.
- the description includes detailing the actions resulting from a CDN customer uploading content to the service provider's CDN and actions resulting from an end user requesting content hosted by the service provider's CDN.
- Entry Point Any CDN service provider's customer may interact with the CDN infrastructure solely via the entry point.
- the entry point runs a web services interface to create / update their CDN accounts and to create / delete and update buckets for their account.
- Each bucket has associated with it meta-data of two kinds, Unix like file-system meta-data and content distribution meta-data.
- the customer may define bucket-level meta-data when creating a bucket.
- a CDN customer may upload content file together with file-level meta-data.
- a customer may create two kinds of buckets: VoD for Video on demand content and Live to stream live content.
- VoD content a CDN customer has two options for uploading data. The customer can either upload files into the bucket or give URLs to the files that point to the CDN customer's website. The data is then downloaded for processing by the CDN publishing point. Subsequently, the downloaded file is moved to another directory for post-processing.
- the CDN customer merely gives the URL of the live stream as a meta-data parameter to the live bucket.
- the CDN infrastructure is responsible for getting the original content stream and distributing it to the CDN end users.
- the CDN manager hosts the Content Manager API, the DNS API and the Network Topology API (all APIs are on this server). There is one instance of the CDN manager for the entire CDN.
- the CDN manager may reside at one of the publishing points or in a separate server.
- the CDN manager may reside in OB0 or any one of the OBs.
- End Point An end point is the entity that manages communication between requesting end users and the CDN infrastructure. It is essentially a custom HTTP server.
- the end points maintain a list of buckets/files that are available with other end points in the same datacenter. This allows end points in the same datacenter to get content from one another when possible and go to the origin server for the requested content when necessary.
- the end points also have a list of all files and buckets in the OB. Thus, they are also responsible for geo-blocking requests to the CDN infrastructure, discarding frivolous content requests, performing authentication on requested content if deemed necessary by content owners.
- the end points also play a central role in DNS resolution since it has both the consistent hash ring and the regions database to identify the region of the originating request and issue a HTTP redirect to the request to the appropriate region if necessary.
- Tracker The tracker is the key entity that enables intelligence and coordination of the CDN infrastructure. In order to do this, a tracker maintains (1 ) information about content at each end point through consistent hashing and (2) collects resource usage statistics periodically from each end point. As part of this, the tracker maintains information like number of outbound bytes, number of inbound bytes, number of active connections for each bucket, size of content being served etc.
- Tracker assists in communication between end points by identifying neighbourhoods of end points allowing them to share data between one another rather than going to the origin server to retrieve data each time.
- the tracker also helps end points synchronize with other entities of the CDN infrastructure: change in bucket metadata and geodb with the CDN manager, regions database between with the TLD DNS server, pidlocdb and cost-matrix with the topology server.
- Origin Server This is the element of the service provider's CDN infrastructure that contains the master copy of the data. Any end point that does not have a copy of the data can request it from the origin server. A CDN customer does not have access to the origin server.
- the service provider's CDN infrastructure moves data from the publishing server to the origin server after performing sanity-checks on the downloaded data (MD5 of data etc.).
- the origin server also maintains a block level integrity of the downloaded data.
- Origin server is the property of buckets in the service provider's CDN. Every file in a bucket has associated with it, an origin server URL. Files from the CDN customer are first uploaded to the publishing point and then to origin server after postprocessing.
- the Topology server provides a topology aware service that has information about the topology of the OB in which it operates. Every OB can specify their routing policies between IP prefixes at the topology server. The topology server maintains information about the transit networks and cost of traversing links as well.
- the topology server helps the tracker select the path of least cost (as defined by the OB policy) in the event that there are multiple paths between a content source and a requesting end user.
- Live Splitter When the customer creates a bucket for a "live" channel (or event), it is passed to the live splitter.
- the live splitter serves as an origin server for a live stream.
- the live splitter is responsible for getting a live-stream, splitting a live stream and using a segmenter to create a playlist.
- the playlist generated is sent to end points that can serve live content.
- the live-splitter is limited only by the available bandwidth.
- DSN Server The DNS server provides the name resolution within the CDN infrastructure.
- the root DNS forwards a query for t-cdn.net to the DNS server that is authoritative for the top-level domain (TLD).
- TLD DNS server resolves queries for DNS server authoritative for the second-level domain.
- Log Server HTTP log files from all end points are sent to this server. These log files are then processed by the processing infrastructure at the log server for real-time display and accounting for the CDN customers.
- the entities that are part of the CDN deployment are described. First, the elements that are part of a typical single OB deployment are identified. Next, an example with multiple OBs is detailed. The global entities in the CDN are also identified and detailed.
- FIG. 1 shows a typical deployment of an OB, according to an embodiment of the invention.
- This deployment assumes one region in the OB.
- the deployment consists of a tracker, a DNS server that is authoritative for an OB, a publishing point (comprising both the CDN manager and the Content Publisher), a live splitter to handle live content, a topology server an origin server to hold content uploaded by the CDN customers, a log server to hold logs from all end points and a number of end points that serve content to the requesting end users.
- the log server feeds into the reporting infrastructure of the OB.
- the log processing/reporting may occur in a separate network and is part of an OB deployment.
- the TLD for the CDN may also reside within an OB.
- TLD DNS subnets are independent. They are not considered as part of the OB's topology and never subject to geo-blocking.
- An OB must have exactly one topology server.
- An OB may deploy the CDN in more than one region (within the same OB).
- a region must have exactly one tracker and DNS server authoritative for that region.
- Two OBs may have the same region ID.
- Region IDs are not globally unique.
- a region may contain more than one subnet.
- a subnet belongs to exactly one region.
- both subnets can belong to different regions and OBs.
- the smaller subnet takes precedence in determining what sub-region it belongs to. That is, if subnet A contains subnet B, and A and B belong to regions R A and R B , then any IP from subnet B will belong only to region B, and not to region A.
- a default region 0 is defined for every OB. An IP address not in any IP subnet of the OB is assigned to region 0.
- the TLD DNS server is responsible for forwarding end user requests to the appropriate OB DNS server.
- an OB may have as many trackers and DNS servers as the number of regions in an OB.
- the DNS servers of an OB are authoritative only in the region in which they operate (one DNS server per region).
- Figure 2 shows a deployment of the CDN with three OBs. Each deployment runs as an independent CDN.
- Links marked 0 are backbone links that connect the OBs with large, core routers in the Internet.
- the labels on OB1 are explained as follows:
- Label 1 is the publishing point (also called the publishing server). This is where the CDN customer creates buckets, uploads content. The content is moved to the origin server (label 1 a) after post processing and sanity checks.
- the DNS server (after one HTTP re-direction) forwards the request to the tracker at the OB (label 1 b).
- the tracker forwards the request to the end point (label 1 c) that can best serve the request.
- the request for content arrives directly from the end user, if the end point has the content, it will serve the content. If not, the end point will fetch the content from the origin server (label 1 d) before serving the request.
- the CDN infrastructure also contains a global element, the TLD DNS server. It can either sit independently or it can reside within one of the OBs.
- the CDN Manager resides with the publishing server in one of the OBs (since there is one instance of the CDN Manager for the CDN).
- Every OB must set a default region 0.
- An OB administrator is responsible for defining the range of IPs under administration of each of its regions. Since no OB will define the whole range of IPs, the remaining IP prefixes are assigned to region 0 (region zero).
- Region zero is a default global region. It has a tracker to coordinate all of the end points deployed in this region. Any request from an IP that is not part of any of the regions will be served by an end point from the default region.
- This global region is a separate entity altogether. This global region is dimensioned so that it can handle requests and has enough capacity to serve content requests from all parts of the world where regions are not properly defined. API Definitions
- the method and deployment scenario of the CDN infrastructure of the invention provides for the use of a CDN manager hosting all the APIs for the CDN.
- This allows a CDN service provider for an OB to use the APIs for content management, DNS service when defining a region and Network Topology when defining the underlying network.
- the CDN manager provides Web APIs (or web service) based on REST style communication with defined request and response messages. Said APIs are described next for some embodiments.
- This API provides all necessary methods to manage content that the CDN will deliver.
- the delivery is based on the concept of an intelligent bucket, a logical container with appropriate meta-data.
- the meta-data for a bucket is composed of two parts: file- system level meta-data (as in a Unix file-system) and content distribution metadata.
- VoD bucket for video streaming or static file delivery
- Live bucket for live streaming
- the VoD bucket defines meta-data for authentication type, distribution bitrate, geo-blocking, content pre-positioning, whitelist / blacklist, etc.
- the Live stream on the other hand defines meta-data for source URL of the live stream, the live-splitter, geo-blocking, whitelist / blacklist, etc.
- the API defines methods for creating, deleting, retrieving and updating bucket meta-data.
- the content management API defines file level meta-data updates for each file in a bucket.
- the file level meta-data override the bucket level meta-data.
- the API defines methods for listing files in a bucket, adding files to and deleting files from a bucket. Some of the meta-data parameters that may be over-written are: enabled, startdate (valid from), enddate (valid until), bandwidth (delivery rate), geoloc (for geo-blocking), etc.
- the DNS entry point is part of the CDN Manager in the form of a DNS manager.
- the DNS manager provides a DNS API for communication between a CDN / OB administrator and the CDN infrastructure.
- the namespace is divided into OBs.
- An OB may have multiple regions.
- the regions are each assigned a region ID (an integer).
- the DNS API is used to assign names and regions, performing geo- location and to translate names in the name space.
- the TLD DNS server is responsible of the geo-localization, translating names in the CDN name-space: (a) bucket names will always be geo-localized using the regions database. For example, b10.t-cdn.net will become b10.1 1 1.t-cdn.net for a user in region 1 1 1. (b) Named hosts (i.e., webcachel ), will be geo-localized if they are present in the names database. If the name host is not present in the names database, the name will be resolved as an alias for the region 0. The TLD DNS is in the default region 0. IP address space that does not belong to any region in any OB defaults to region 0. The TLD DNS is responsible for forwarding end user requests to the appropriate OB DNS servers.
- (a) bucket names will always be geo-localized using the regions database. For example, b10.t-cdn.net will become b10.1 1 1.t-cdn.net for a user in region 1 1 1.
- Named hosts i.e., webcachel
- the API defines a method for creating entities in an OB or a region in an OB, a GET on the names method to list all the entities in an OB.
- the API also defines methods that use a regions code to retrieve, update and delete information about a network element (tracker, DNS server, topology server etc.).
- the API is useful in creating a region, retrieving the list of current regions, retrieving information about the region, updating region information (e.g. subnets that belong to a region) and deleting a region.
- the API also defines methods to retrieve the regions database (all IP prefixes in a region from a regional DNS server) and the entire database of regions (all IP prefixes in all regions) from the TLD DNS server.
- the tracker executes the last two API calls.
- the network topology API allows the specification of a logical network topology graph from the point of view of a network operator, by acting as an interface with the Topology Server.
- the API allows the operators to abstract physical connections into logical partitions containing subnets.
- the partitions are linked to one another with edges that have weights associated with them. These weights indicate the cost of transferring data on the link between the two partitions.
- Subnets A subnet object has the following attributes: subnet and mask.
- the API defines methods to create a new subnet, delete an existing subnet and retrieve a collection of subnets.
- the API also defines methods to add subnets to a partition ID (PID), delete subnets from a PID, and retrieve all subnets that belong to a PID.
- a method to retrieve the entire partition - subnet pairs list is also defined.
- Partitions The API defines methods to create a partition object that returns a PID, return a collection of partitions, retrieve information about (subnets in) a PID, update / create a PID and delete a PID.
- a network edge defines the cost of transporting one unit of data between two PIDs (or simply, the weight of an edge).
- the API defines methods to get the costmatrix (a matrix that defines the cost between every partition pair), create, update and retrieve the cost between two PIDs. As a default, the cost between two PIDs is set to a large value.
- the API also defines methods to retrieve all edges with a given origin PID, retrieve all edges to a destination PID.
- the topology described at a Topology Server can extend far beyond its OB boundaries. Only the OB point of view of the network topology is described. As an example, an OB in Spain creates a partition that defines all the subnets in USA while another OB in Argentina creates partitions that defines all of the subnets of USA.
- the APIs described above that are hosted at the CDN manager are used to add and delete regions in an OB.
- the CDN manager also hosts the DNS manager that is the entry point to TLD
- DNS server This allows the DNS manager to remotely manage the SQL database at the TLD DNS server.
- An OB administrator assigns a region ID for a new region in an OB. Once all the elements of a region within an OB are configured, the DNS API is invoked. This call to the DNS API associates the new region with the TLD DNS server. This allows the TLD DNS server to recursively resolve a request destined to a region in an OB.
- the DNS API is invoked. This has the effect of (i) deleting the region from the TLD DNS server, (ii) deleting all the NS records and A records of the region just removed and (iii) deleting all IP address ranges that belong to the deleted region.
- the interactions consist of three parts: (a) A CDN customer's interaction with the service provider's CDN when adding content, (b) interaction between all of the components of the CDN infrastructure and (c) how does a CDN serve content when an end user makes a request for content hosted by the CDN.
- Figure 3 shows the sequence diagram of interactions between CDN customer and the CDN infrastructure and between various elements of the CDN infrastructure on uploading content.
- the entry point is the sole point of communication for a CDN customer with a service provider's CDN infrastructure.
- a CDN customer can use her account to create, remove and edit buckets (a directory or folder that holds the CDN customer's content files).
- a CDN customer must first create a content bucket within the CDN infrastructure.
- the bucket has a globally unique id that helps the CDN service provider to identify a CDN customer.
- the CDN manager as shown in Figure 3 provides this unique bucket id.
- customers create a bucket associate appropriate meta-data to the bucket and upload content to the bucket, a requests. xml file is created. This file (requests. xml) contains a list of files that need to be placed in a bucket as well as metadata associated with each file.
- a CDN customer may add content by any one of two methods: upload content or give the IP address of the CDN customer's origin server from where the CDN infrastructure can download the content.
- the content is downloaded at the publishing point. Before the content is transferred to the origin server, the integrity of content is checked.
- the customer can either upload the files themselves to the publishing server or point to the files as being in a server at the customer's site.
- the CDN infrastructure will pull the content from the CDN customer's server. Since the XML also contains meta-data for each file in the bucket and is set by the customer, it ensures that the customer has full control over the content.
- the CDN customer may also assign some file-level meta-data and / or override some of the bucket meta-data. This has the effect of creating a requests. xml file.
- the requests. xml file defines meta-data of the uploaded file(s) in the bucket.
- a monitoring process looks for the existence of requests. xml file. Once it detects that all files referenced in the xml are present (and have the right size), the files are moved to another directory for post- processing. At this time, the content publisher is invoked.
- the content publisher checks the uploaded files for content integrity. Once the integrity of the content is checked against the meta-data defined by a CDN customer, a block-level checksum for each uploaded file is created. This ensures that the master copy of the residing in the origin server of the CDN is always valid.
- the request. xml file is detected at the publishing point, it is read, and if the media files are uploaded, they are moved to the origin server after checking for their integrity. On the other hand, if the URL of the media files point to a server in the customer's premises, these files are downloaded to the publishing point and then to origin server of the service provider's CDN infrastructure. All files are now under the control of the CDN infrastructure but all meta-data control over individual files and buckets is with the CDN customer. This ensures that the CDN customer has full control over the duration for which content is valid and also the geographic location in which the content may be shown. The meta-data allows a CDN customer to add, remove and update content file(s).
- the monitoring process After processing the files, the monitoring process generates a responses. xml file in the same directory. This notifies the CDN customer if the content uploaded to the publishing point was successfully ingested. In addition, the CDN customer gets the CDN URL(s) of the uploaded content.
- the content publisher moves the uploaded files together with the block level integrity check to the origin server. Any end point requesting content will make the request to the origin server. The same holds if the content is downloaded from the origin sever by any of the CDN end points that serve the content to a requesting CDN customer.
- the end points have a list of buckets and files together with block level checksum information from the CDN manager (via the tracker).
- the end point contacts the origin server.
- the origin server (or other end points in the same datacenter) serve the requested content.
- the receiving end point verifies the block level integrity of the downloaded file. Once verified, the end point serves the content to the requesting end user(s).
- the CDN infrastructure consists of a variety of components - tracker, topology server, end points, publishing server, origin server and entry point.
- a CDN customer can interact with the service provider's CDN infrastructure only at the publishing server.
- the communication between each of the CDN entities occurs via RPCs.
- the RPCs may take any of the following formats: XML, binary, json, REST API call etc. with HTTP as a transport mechanism.
- the geo-location database, geodb (and any changes in the geodb) are propagated to the tracker periodically.
- the geodb information is sent to the end points to exercise geo-blocking.
- the trackers keep an open connection with the topology server.
- the tracker fetches information about (1 ) partitions (or subnets), called pidlocdb and (2) cost-matrix (called costmatrix) between partitions (or subnets).
- the topology server has a global view across geographically distributed trackers in an OB. ISPs can define their routing policies via the topology server.
- the topology server can recommend certain routes to the tracker over others depending on cost of sending data across the transit networks, IGP or BGP policies and the current load on the links. This information is contained in the cost-matrix between partitions.
- the tracker receives a file that contains information about regions (called regionsdb) from the TLD DNS server. This regions information is passed to the end point periodically or when updated. This information is useful for an end point for (1 ) performing geo-blocking and (2) determining the region of an originating request.
- the end point If the region of the originating request is not the same as that of the end point, the end point returns a HTTP 302 while encoding the region as part of the URL (using regionsdb obtained from the TLD DNS server).
- the tracker synchronizes with the end points regularly.
- the tracker updates the end points with the following information regularly: (1 ) list of all buckets meta-data, (2) pidlocdb, a table of PIDs, (3) regionsdb, information about regions in the CDN, (4) geodb, the geolocation database, (5) costmatrix, the cost of transferring data between two PIDs and (6) neighbourhood IP list.
- Any change in a customer's bucket meta-data is propagated to the tracker.
- the change in bucket meta-data is also propagated to the end points in a matter of seconds. This ensures that any change a CDN customer makes to her files in a content bucket has an almost immediate effect on its distribution.
- the tracker collects detailed usage (cpu, disk, connection count) and file statistics (bytes transferred inbound/outbound) at each end point. It uses this information to load-balance connections across endpoints after determining the end points that have the requested content upon executing consistent hashing on the requested file.
- the tracker also helps the end point keep an updated list of all its neighbours.
- An end point can request the tracker to update its list of neighbours.
- the tracker sends a list of neighbouring IP addresses and their file statistics. This allows an end point to determine which of the end points in its neighbourhood is best positioned to send the content requested (rather than go to the origin server to retrieve the content) by an end user.
- End points in the same datacenter inform their neighbours of the availability of newly downloaded content. Thus, other end points download the content (if needed) from end point(s) in the same datacenter that have the content thereby reducing the load on the origin server. This is referred to as peer-to-peer communication (since end points in the same datacenter are involved).
- label 1 corresponds to the CDN customer's interaction with service provider's CDN infrastructure.
- Labels 1 a and 1 b correspond to data being downloaded to the origin server of the CDN infrastructure.
- DNS Domain Name Service
- DNS resolution occurs transparently regardless of the application program a user runs. DNS resolution occurs in the following steps: (1 ) the requesting program sends an address resolution request to the DNS resolver in the local operating system. If the local cache contains the answer, the resolver returns the value in the cache to the requesting program. (2) If the cache does not contain the answer, the resolver will send the request to the designated DNS server of the organization. For most home users, the ISP to which the machine connects provides the DNS server (or resolver). (3) The DNS resolver of the ISP will attempt to resolve the IP address first in its local cache or request the root DNS server. So, the IP address received by any of the TLD is not the IP address of the client, but that of the DNS resolver of the ISP.
- Any DNS resolver is configured with known address of root servers.
- a query to one of the root servers will return the server authoritative for the top-level domain (TLD).
- TLD top-level domain
- the root DNS server will return the server authoritative for .net domain.
- the ISP DNS resolver queries the TLD DNS server for the .net domain to resolve t-cdn.net domain.
- the .net nameserver responds with the IP address of server authoritative for t-cdn.net domain.
- t-cdn.net first infers 87.t-cdn.net to be an alias for 87. g. t-cdn.net. So, a query for the g. t-cdn.net domain will be performed.
- the authoritative DNS server for the t-cdn.net domain resolves the sub- zone of ISP DNS resolver.
- the DNS server authoritative for the second-level domain responds with the sub-zone of the client as "es" (for Spain).
- DNS server resolves the IP address in the sub-zone to be es.t-cdn.net.
- the DNS server es.t-cdn.net is the authoritative DNS server for the "es" sub-zone.
- the ISP DNS resolver will then attempt to resolve 87.es.t- cdn.net by querying the authoritative DNS server in the sub-zone.
- the authoritative DNS server in the sub-zone has a list of all the end points. It forwards the request to any one of the end points using either a round-robin scheme or a best effort geo-location scheme that tries to match the request with the closest end point from ⁇ End Point 1 , End Point 2, End Point 3, ... , End Point N ⁇ .
- End point 2 receives the request.
- the end point performs a fine-grained geo-location on the requested IP address and identifies the location as BCN. This implies that an end point in BCN datacenter may be best suited to serve content to the requesting end user.
- End point 2 performs a consistent hash on the requested URL (here,
- the HTTP response is returned to the End User.
- the ISP DNS resolver forwards the address resolution request from the client to the DNS resolver in the "es" sub-zone.
- End Point 2 and End Point N ⁇ as end points that can best serve the request.
- the tracker takes into account current load at the end points and identifies end point 3 as being best suited to serve content.
- the response from the tracker identifying end point 3 is returned to the authoritative DNS resolver in the "es" sub-zone.
- the ISP DNS resolver returns the response to the End User.
- the end point 3 will attempt to get the content from any one of the other neighbourhood end points in the same datacenter (if the content already exists in one of the end points). Alternately, the end point 3 will revert to the origin server to get the requested content and serve the end user. Before serving the content, the end point may also communicate with an authentication server at the content owner if specified in the meta-data of the bucket. In summary, the whole resolution process entailed two queries: one query to the DNS server in the second level domain t-cdn.net to identify the datacenter closest to the requesting client. This step involves URL rewriting and HTTP redirect message from one of the end point in the CDN infrastructure that is chosen to address the query.
- a DNS query is made to identify the end point in the datacenter closest to the client that can best serve content.
- the DNS request is forwarded to the tracker that uses consistent hashing to identify the content and then uses statistical knowledge of the end points to return a set of end points (in order, with the first end point best positioned and so on) to best serve the content to the requesting end user.
- DNS server authoritative for t-cdn.net domain may also be authoritative for the sub-zone "es".
- the DNS server detects that and will attempt to send the DNS query to one of the endpoints to first resolve the closest datacenter that may best serve the request.
- This invention provides the following features:
- the solution is designed to run on commodity hardware. In addition, the solution obviates the need for expensive load balancers and any high-end network elements.
- the invention allows the CDN service provider to add trackers and end points as needed, making the CDN infrastructure of the invention very scalable.
- the invention shows how the CDN infrastructure may add additional OBs easily.
- the CDN architecture has a small number of global entities. A number of other entities are deployed on a per-OB basis.
- the global entity, the CDN manager is responsible for the following: a. Assigning a globally unique bucket id. A CDN customer may have as many buckets as desired. b. Assign globally unique OB ids. Thus, the CDN manager is invoked when creating a new OB.
- the CDN incorporates the latest network cost in identifying end points that are best located to serve content to requesting end users based on least network cost.
- the CDN infrastructure of this invention does not need very large data centres for hosting.
- the end points may be added in small datacenters as needed.
- the datacenter can be at a block level or city level or at the region level.
- the CDN of this invention is designed for ISPs.
- the end points may be placed at the same location as the DSLAMs.
- the physical location of DSLAMs makes it easy for content to be located close to the end users.
- the tracker behaves as a file-level load-balancer for the entry points obviating the need for other hardware to behave as a load-balancer.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
BR112013028989A BR112013028989A2 (en) | 2011-05-12 | 2012-05-09 | method for administering the infrastructure of a content distribution network service in an isp network, and infrastructure of a content distribution network service in an isp network |
EP12722699.1A EP2707997A1 (en) | 2011-05-12 | 2012-05-09 | Method for managing the infrastructure of a content distribution network service in an isp network and such an infrastructure |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ESP201130761 | 2011-05-12 | ||
ES201130761A ES2410654B1 (en) | 2011-05-12 | 2011-05-12 | SYSTEM AND METHOD FOR MANAGING THE INFRASTRUCTURE OF A NETWORK DISTRIBUTION NETWORK SERVICE IN AN ISP NETWORK |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012152824A1 true WO2012152824A1 (en) | 2012-11-15 |
Family
ID=46147425
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2012/058527 WO2012152824A1 (en) | 2011-05-12 | 2012-05-09 | Method for managing the infrastructure of a content distribution network service in an isp network and such an infrastructure |
Country Status (6)
Country | Link |
---|---|
EP (1) | EP2707997A1 (en) |
AR (1) | AR086342A1 (en) |
BR (1) | BR112013028989A2 (en) |
CL (1) | CL2013003223A1 (en) |
ES (1) | ES2410654B1 (en) |
WO (1) | WO2012152824A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2747379A1 (en) | 2012-12-19 | 2014-06-25 | Telefonica S.A. | A distributed health-check method for web caching in a telecommunication network |
EP3123320A4 (en) * | 2014-03-28 | 2018-04-04 | Amazon Technologies Inc. | Implementation of a service that coordinates the placement and execution of containers |
US20180367375A1 (en) * | 2015-06-25 | 2018-12-20 | Orange | Notification method relative to at least one operation implemented by a device forming a node of a network |
US10320882B2 (en) | 2017-08-29 | 2019-06-11 | At&T Intellectual Property I, L.P. | Uniform resource locator discovery and tracking for managing sponsored data |
CN110933184A (en) * | 2019-12-17 | 2020-03-27 | 广州市百果园信息技术有限公司 | Resource publishing platform and resource publishing method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040172466A1 (en) * | 2003-02-25 | 2004-09-02 | Douglas Christopher Paul | Method and apparatus for monitoring a network |
US20070055764A1 (en) * | 2002-04-09 | 2007-03-08 | Dilley John A | Method and system for tiered distribution in a content delivery network |
EP1821487A1 (en) * | 2006-02-21 | 2007-08-22 | Microsoft Corporation | Topology management in peer-to-peer content distribution clouds |
US20110078327A1 (en) * | 2009-09-30 | 2011-03-31 | Prime Networks (Hong Kong) Limited | Content delivery utilizing multiple content delivery networks |
US20110078230A1 (en) * | 2009-09-25 | 2011-03-31 | Emilio Sepulveda | Method and system for providing a cdn with granular quality of service |
-
2011
- 2011-05-12 ES ES201130761A patent/ES2410654B1/en not_active Withdrawn - After Issue
-
2012
- 2012-05-09 BR BR112013028989A patent/BR112013028989A2/en not_active IP Right Cessation
- 2012-05-09 WO PCT/EP2012/058527 patent/WO2012152824A1/en active Application Filing
- 2012-05-09 EP EP12722699.1A patent/EP2707997A1/en not_active Withdrawn
- 2012-05-10 AR ARP120101653A patent/AR086342A1/en not_active Application Discontinuation
-
2013
- 2013-11-11 CL CL2013003223A patent/CL2013003223A1/en unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070055764A1 (en) * | 2002-04-09 | 2007-03-08 | Dilley John A | Method and system for tiered distribution in a content delivery network |
US20040172466A1 (en) * | 2003-02-25 | 2004-09-02 | Douglas Christopher Paul | Method and apparatus for monitoring a network |
EP1821487A1 (en) * | 2006-02-21 | 2007-08-22 | Microsoft Corporation | Topology management in peer-to-peer content distribution clouds |
US20110078230A1 (en) * | 2009-09-25 | 2011-03-31 | Emilio Sepulveda | Method and system for providing a cdn with granular quality of service |
US20110078327A1 (en) * | 2009-09-30 | 2011-03-31 | Prime Networks (Hong Kong) Limited | Content delivery utilizing multiple content delivery networks |
Non-Patent Citations (3)
Title |
---|
C. HUANG; J. LI; ANGELA WANG; K.W. ROSS: "Understanding Hybrid CDN-P2P: Why Limelight Needs its Own Red Swoosh", NOSSDAV, 2008 |
CHRISTIAN TIMMERER: "An Integrated Management Supervisor for Advanced IPTV Terminal (AIT)", 91. MPEG MEETING; 18-1-2010 - 22-1-2010; KYOTO; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. M17136, 25 January 2010 (2010-01-25), XP030045726 * |
MASATOSHI KAWARASAKI NTT JAPAN: "Q.C Metadata Baseline Document (Draft)", ITU-T DRAFTS ; STUDY PERIOD 2001-2004, INTERNATIONAL TELECOMMUNICATION UNION, GENEVA ; CH, vol. Study Group 16 ; C/16, 20 May 2003 (2003-05-20), pages 1 - 13, XP017506093 * |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2747379A1 (en) | 2012-12-19 | 2014-06-25 | Telefonica S.A. | A distributed health-check method for web caching in a telecommunication network |
WO2014095501A1 (en) * | 2012-12-19 | 2014-06-26 | Telefonica, S.A. | A distributed health-check method for web caching in a telecommunication network |
US9948741B2 (en) | 2012-12-19 | 2018-04-17 | Telefonica, S.A. | Distributed health-check method for web caching in a telecommunication network |
EP3123320A4 (en) * | 2014-03-28 | 2018-04-04 | Amazon Technologies Inc. | Implementation of a service that coordinates the placement and execution of containers |
US10218633B2 (en) | 2014-03-28 | 2019-02-26 | Amazon Technologies, Inc. | Implementation of a service that coordinates the placement and execution of containers |
US20180367375A1 (en) * | 2015-06-25 | 2018-12-20 | Orange | Notification method relative to at least one operation implemented by a device forming a node of a network |
US10320882B2 (en) | 2017-08-29 | 2019-06-11 | At&T Intellectual Property I, L.P. | Uniform resource locator discovery and tracking for managing sponsored data |
CN110933184A (en) * | 2019-12-17 | 2020-03-27 | 广州市百果园信息技术有限公司 | Resource publishing platform and resource publishing method |
CN110933184B (en) * | 2019-12-17 | 2022-08-23 | 广州市百果园信息技术有限公司 | Resource publishing platform and resource publishing method |
Also Published As
Publication number | Publication date |
---|---|
ES2410654A2 (en) | 2013-07-02 |
ES2410654B1 (en) | 2014-05-21 |
EP2707997A1 (en) | 2014-03-19 |
AR086342A1 (en) | 2013-12-04 |
CL2013003223A1 (en) | 2014-08-01 |
ES2410654R1 (en) | 2013-10-07 |
BR112013028989A2 (en) | 2017-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2708013B1 (en) | A method for DNS resolution of content requests in a CDN service | |
US20140215059A1 (en) | Method and a tracker for content delivery through a content delivery network | |
US11277371B2 (en) | Content routing in an IP network | |
US10706029B2 (en) | Content name resolution for information centric networking | |
US9712422B2 (en) | Selection of service nodes for provision of services | |
US20140304386A1 (en) | Routing client requests | |
US20200186993A1 (en) | Anycast manifest retrieval, unicast content retrieval | |
KR20140035385A (en) | Combined cdn reverse proxy and an edge forward proxy with secure connections | |
Braun et al. | Service-centric networking extensions | |
US20190166223A1 (en) | Content delivery network (cdn) for uploading, caching and delivering user content | |
US20140149548A1 (en) | Method for content delivery in a content distribution network | |
WO2012152824A1 (en) | Method for managing the infrastructure of a content distribution network service in an isp network and such an infrastructure | |
EP2708011B1 (en) | System and method for content distribution internetworking | |
EP2400749B1 (en) | Access network controls distributed local caching upon end-user download | |
Tyson | A middleware approach to building content-centric applications | |
WO2013140097A1 (en) | Selection of a network entity for the provision of a digital content | |
JACOBS-BURTON | CROSS REFERENCE TO RELATED APPLICATIONS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12722699 Country of ref document: EP Kind code of ref document: A1 |
|
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2013003223 Country of ref document: CL |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012722699 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112013028989 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112013028989 Country of ref document: BR Kind code of ref document: A2 Effective date: 20131111 |