US20150074187A1 - Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows - Google Patents

Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows Download PDF

Info

Publication number
US20150074187A1
US20150074187A1 US14/542,866 US201414542866A US2015074187A1 US 20150074187 A1 US20150074187 A1 US 20150074187A1 US 201414542866 A US201414542866 A US 201414542866A US 2015074187 A1 US2015074187 A1 US 2015074187A1
Authority
US
United States
Prior art keywords
server
data packet
packet
region
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/542,866
Inventor
Gregory Fletcher
Andrew F. Champagne
Daniel Stodolsky
Michael M. Afergan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Akamai Technologies Inc
Original Assignee
Akamai Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US11/323,342 priority Critical patent/US7660296B2/en
Priority to US12/701,718 priority patent/US8170017B2/en
Priority to US13/459,749 priority patent/US8891522B2/en
Application filed by Akamai Technologies Inc filed Critical Akamai Technologies Inc
Priority to US14/542,866 priority patent/US20150074187A1/en
Publication of US20150074187A1 publication Critical patent/US20150074187A1/en
Assigned to AKAMAI TECHNOLOGIES, INC. reassignment AKAMAI TECHNOLOGIES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: STODOLSKY, DANIEL, AFERGAN, MICHAEL M., CHAMPAGNE, ANDREW F., FLETCHER, GREGORY
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04JMULTIPLEX COMMUNICATION
    • H04J3/00Time-division multiplex systems
    • H04J3/16Time-division multiplex systems in which the time allocation to individual channels within a transmission cycle is variable, e.g. to accommodate varying complexity of signals, to vary number of channels transmitted
    • H04J3/1605Fixed allocated frame structures
    • H04J3/1611Synchronous digital hierarchy [SDH] or SONET
    • H04J3/1617Synchronous digital hierarchy [SDH] or SONET carrying packets or ATM cells
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. local area networks [LAN], wide area networks [WAN]
    • H04L12/46Interconnection of networks
    • H04L12/4633Interconnection of networks using encapsulation techniques, e.g. tunneling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L29/00Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1/00 - H04L27/00
    • H04L29/12Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1/00 - H04L27/00 characterised by the data terminal
    • H04L29/12009Arrangements for addressing and naming in data networks
    • H04L29/12047Directories; name-to-address mapping
    • H04L29/12056Directories; name-to-address mapping involving standard directories and standard directory access protocols
    • H04L29/12066Directories; name-to-address mapping involving standard directories and standard directory access protocols using Domain Name System [DNS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L29/00Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1/00 - H04L27/00
    • H04L29/12Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1/00 - H04L27/00 characterised by the data terminal
    • H04L29/12009Arrangements for addressing and naming in data networks
    • H04L29/1233Mapping of addresses of the same type; Address translation
    • H04L29/12339Internet Protocol [IP] address translation
    • H04L29/1249NAT-Traversal
    • H04L29/12509NAT-Traversal for reachability, e.g. inquiring the address of a correspondent behind a NAT server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/50Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer
    • H04L41/5077Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer wherein the managed service relates to simple transport services, i.e. providing only network infrastructure
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/50Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer
    • H04L41/508Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer based on type of value added network service under agreement
    • H04L41/5096Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer based on type of value added network service under agreement wherein the managed service relates to distributed or central networked applications
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements or network protocols for addressing or naming
    • H04L61/15Directories; Name-to-address mapping
    • H04L61/1505Directories; Name-to-address mapping involving standard directories or standard directory access protocols
    • H04L61/1511Directories; Name-to-address mapping involving standard directories or standard directory access protocols using domain name system [DNS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements or network protocols for addressing or naming
    • H04L61/20Address allocation
    • H04L61/2007Address allocation internet protocol [IP] addresses
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements or network protocols for addressing or naming
    • H04L61/25Network arrangements or network protocols for addressing or naming mapping of addresses of the same type; address translation
    • H04L61/2503Internet protocol [IP] address translation
    • H04L61/256Network address translation [NAT] traversal
    • H04L61/2567Network address translation [NAT] traversal for reachability, e.g. inquiring the address of a correspondent behind a NAT server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/42Protocols for client-server architectures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/50Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer
    • H04L41/5003Managing service level agreement [SLA] or interaction between SLA and quality of service [QoS]
    • H04L41/5009Determining service level performance, e.g. measuring SLA quality parameters, determining contract or guarantee violations, response time or mean time between failure [MTBF]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • H04L43/08Monitoring based on specific metrics
    • H04L43/0823Errors
    • H04L43/0829Packet loss
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • H04L43/08Monitoring based on specific metrics
    • H04L43/0852Delays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • H04L43/16Arrangements for monitoring or testing packet switching networks using threshold monitoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements or network protocols for addressing or naming
    • H04L61/60Details
    • H04L61/6068Internet protocol [IP] addresses subnets

Abstract

The present invention leverages an existing content delivery network infrastructure to provide a system that enhances performance for any application that uses the Internet Protocol (IP) as its underlying transport mechanism. An overlay network comprises a set of edge nodes, intermediate nodes, and gateway nodes. This network provides optimized routing of IP packets. Internet application users can use the overlay to obtain improved performance during normal network conditions, to obtain or maintain good performance where normal default BGP routing would otherwise force the user over congested or poorly performing paths, or to enable the user to maintain communications to a target server application even during network outages.

Description

  • This application is a continuation of Ser. No. 11/323,342, filed Dec. 30, 2005, now U.S. Pat. No. 7,660,296.
  • BACKGROUND OF THE INVENTION
  • 1. Technical Field
  • The present invention relates generally to data packet transport and routing over the Internet.
  • 2. Brief Description of the Related Art
  • The public Internet is increasingly being used by enterprises for a variety of mission-critical applications such as transactions for e-commerce, inter-office connectivity over virtual private networks (VPNs), and most recently, for web services as a new paradigm for developing distributed applications. The current Border Gateway Protocol (BGP) based Internet routing infrastructure, however, is inadequate to support the reliability and performance needs of these applications. In particular, Internet routing, largely determined by the BGP protocol, has several weaknesses. First, BGP uses a metric known as shortest AS (Autonomous System) path length to determine a next hop for a packet. FIG. 1A shows that BGP will route data from Network A destined for Network D directly, because the AS path length is one. This is not always desirable, as it has been shown that BGP is slow to converge. Thus, if a link between two networks becomes unavailable, it can take seconds to several minutes before all relevant routers become aware and can route around the problem. During this time, packets will be lost. Furthermore, as illustrated in FIG. 1B, peering policies may dictate that a network should not accept packets from another network; BGP cannot efficiently route around this problem. Another problem is that different Internet applications require different characteristics (e.g., minimal loss, latency, or variability in latency) of an end-to-end connection for optimal performance. BGP makes no effort to route for quality of service and has no notion of any of these metrics. As illustrated in FIG. 2, BGP will choose a route (between Networks A and D) with larger latency than alternative routes.”
  • There is a need in the art for intelligent routing as businesses increasingly rely on the Internet for such applications as Web transactions, virtual private networks (VPNs) and Web Services. The notion of intelligent routing based on measurements of real time network conditions is known in the art, e.g., such as the product offerings by RouteScience and other companies. These products, however, only have an ability to control only the first hop of the outbound route, namely, by injecting appropriate directives into the router. Attempts to control the inbound route, e.g., by affecting BGP advertisements, are limited by the low frequency with which these advertisements can be changed, the coarse granularity of the advertisements, the requirement of cooperation from multiple routers on the Internet, and the ubiquity of policy overrides by several ISPs.
  • Distributed computer systems also are well-known in the prior art. One such distributed computer system is a “content delivery network” or “CDN” that is operated and managed by a service provider. The service provider may provide the service on its own behalf, or on behalf of third parties. A “distributed system” of this type typically refers to a collection of autonomous computers linked by a network or networks, together with the software, systems, protocols and techniques designed to facilitate various services, such as content delivery or the support of outsourced site infrastructure. Typically, “content delivery” means the storage, caching, or transmission of content, streaming media and applications on behalf of content providers, including ancillary technologies used therewith including, without limitation, request routing, provisioning, data monitoring and reporting, content targeting, personalization, and business intelligence. The term “outsourced site infrastructure” means the distributed systems and associated technologies that enable an entity to operate and/or manage a third party's Web site infrastructure, in whole or in part, on the third party's behalf.
  • A known distributed computer system is assumed to have a set of machines distributed around the Internet. Typically, most of the machines are servers located near the edge of the Internet, i.e., at or adjacent end user access networks. A Network Operations Command Center (NOCC) may be used to administer and manage operations of the various machines in the system. Third party sites, such as Web site, offload delivery of content (e.g., HTML, embedded page objects, streaming media, software downloads, and the like) to the distributed computer system and, in particular, to “edge” servers. End users that desire such content may be directed to the distributed computer system to obtain that content more reliably and efficiently. Although not shown in detail, the distributed computer system may also include other infrastructure, such as a distributed data collection system that collects usage and other data from the edge servers, aggregates that data across a region or set of regions, and passes that data to other back-end systems to facilitate monitoring, logging, alerts, billing, management and other operational and administrative functions. Distributed network agents monitor the network as well as the server loads and provide network, traffic and load data to a DNS query handling mechanism, which is authoritative for content domains being managed by the CDN. A distributed data transport mechanism may be used to distribute control information (e.g., metadata to manage content, to facilitate load balancing, and the like) to the edge servers. As illustrated in FIG. 3, a given machine 300 comprises commodity hardware (e.g., an Intel Pentium processor) 302 running an operating system kernel (such as Linux or variant) 304 that supports one or more applications 306 a-n. To facilitate content delivery services, for example, given machines typically run a set of applications, such as an HTTP Web proxy 307, a name server 308, a local monitoring process 310, a distributed data collection process 312, and the like.”
  • Content delivery networks such as described above also may include ancillary networks or mechanisms to facilitate transport of certain data or to improve data throughput. Thus, an Internet CDN may provide transport of streaming media using information dispersal techniques whereby a given stream is sent on multiple redundant paths. One such technique is described in U.S. Pat. No. 6,751,673, titled “Streaming media subscription mechanism for a content delivery network,” assigned to Akamai Technologies, Inc. The CDN may also provide transport mechanisms to facilitate communications between a pair of hosts, e.g., two CDN servers, or a CDN edge server and a customer origin server, based on performance data that has been collected over time. A representative HTTP-based technique is described in U.S. Published Patent Application 2002/0163882, titled “Optimal route selection in a content delivery network,” also assigned to Akamai Technologies, Inc.”
  • BRIEF SUMMARY OF THE INVENTION
  • It is an object of the present invention to provide a reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows.
  • It is another object of the invention to provide an overlay on top of the Internet that routes data around problems in the Internet to find a best service route or a path with minimal latency and loss.
  • It is still another object of the invention to leverage an existing content delivery network infrastructure to provide a system that enhances performance for any application that uses the Internet Protocol (IP) as its underlying transport mechanism.
  • Another more general object of the invention is to provide an overlay mechanism that improves the performance and reliability of business applications on the Internet;
  • Still another object of the invention is to provide techniques that enable Internet application users to obtain improved performance during normal network conditions, to obtain or maintain good performance where normal default BGP routing would otherwise force the user over congested or poorly performing paths, or to enable the user to continue communications even during network outages.
  • The present invention provides a scalable, highly available and reliable overlay service that detects poor performing paths and routes around them, as well as finding better performing alternative paths when the direct path is functioning normally, thereby ensuring improved application performance and reliability.
  • The foregoing has outlined some of the more pertinent features of the invention. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed invention in a different manner or by modifying the invention as will be described.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • For a more complete understanding of the present invention and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
  • FIG. 1A illustrates a set of four interconnected networks and how BGP determines a route for packets flowing between Network A and Network D;
  • FIG. 1B illustrates how the destination server is unreachable when a problem exists with how Network A peers with Network D;
  • FIG. 2 illustrates how BGP routing decisions are insensitive to latency, which results in poor performance;
  • FIG. 3 illustrates a typical content delivery network edge server configuration;
  • FIG. 4 illustrates how the overlay mechanism of the present invention routes around data links to improve performance;
  • FIG. 5 illustrates how the overlay mechanism of the present invention finds a path with smallest latency to improve performance;
  • FIG. 6 illustrates a set of components that comprise an overlay mechanism, in accordance with an embodiment of the present invention;
  • FIG. 7 illustrates how data flows through the overlay mechanism of FIG. 6;
  • FIG. 8 illustrates how a gateway region may manage network address translation NAT in one embodiment;
  • FIG. 9 illustrates a sequence number translation function that may be carried out within a given gateway region;
  • FIG. 10 illustrates how different pieces of load information are reported on within the overlay;
  • FIG. 11 illustrates an alternate embodiment of the invention wherein a client behind a corporate firewall IP is mapped directly to a gateway region while other clients are mapped to public regions;
  • FIG. 12 is a process flow diagram that illustrates how a global traffic management (GTM) process may be implemented within the overlay in the alternate embodiment;
  • FIG. 13 illustrates normal overlay routing;
  • FIG. 14 illustrates a fail-safe operation for the overlay shown in FIG. 13;
  • FIG. 15 illustrates how the overlay mechanism is used to implement multi-client remote access to a given application on the target server; and
  • FIG. 16 illustrates how the overlay mechanism is used to implement site-to-site (remote office) to a given application on the target server.
  • DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT
  • The present invention is an “overlay” mechanism that sits on top of the publicly-routable Internet. Preferably, the overlay leverages existing content delivery network (CDN) infrastructure, although this is not a requirement. Generally, the overlay mechanism of the present invention provides performance enhancements for any application that uses IP as a transport protocol by routing around down links (FIG. 4) or finding a path with a smallest latency (FIG. 5). Thus, in FIG. 4, the link between Networks A and D is down, so the overlay routes the packets through Network B. In FIG. 5, the overlay uses the path through Network A to Network C to Network D, instead of slower Network A to Network D path that BGP routing would dictate. In FIGS. 4 and 5, it is assumed that a client is attempting to communicate with a server. The illustration of a single or fixed client is not meant to be taken as limiting, however. This is a generalization. According to the present invention, the overlay mechanism may be used in many different operating environments. One such environment is where there are multiple clients (e.g., roaming clients) who desire to access a single server (or server farm). This is sometimes referred to a “remote access” embodiment. In another scenario, two or more fixed endpoints desire to communicate with each other. This is sometimes referred to a site-to-site or “remote office” embodiment. Generalizing, and as will be seen, the overlay mechanism operates by receiving IP packets at one set of servers, tunnelling these packets through a series of zero or more CDN servers, and delivering them to a fixed, defined IP address.
  • The overlay IP (OIP) routing mechanism of the present invention comprises a representative set of components, as illustrated in FIG. 6:
      • edge server 602—typically, a CDN edge server running an OIP edge server software process as described below. As will be described, this software is responsible for receiving, encapsulating and forwarding IP packets.
      • edge region 600—typically, a CDN edge region configured for the overlay mechanism.
      • intermediate server 606—typically, a server that receives encapsulated packets from an edge region 600 or other intermediate servers and forwards them on to other intermediate servers or to a gateway region.
      • intermediate region 604—a region of intermediate servers.
      • gateway server 610—typically, an ddge server that has been configured to receive encapsulated packets from the overlay, and that applies source network address translation (NAT) to the original packets and forwards them onto the target server.
      • gateway region 608—typically, a type of edge region comprising gateway servers and that is usually deployed on customer premises.
      • Target server 612—a machine whose traffic is to be tunneled through the overlay.
      • target address—the IP address of the target server; this address is sometimes referred to as a direct address when being compared to a CDN virtual IP address.
      • slot—a single “instance” of the overlay; preferably, a slot is a numbered index that corresponds to a single target address.
      • virtual IP address—typically, a CDN address that corresponds to a slot;
  • preferably, there is one virtual IP address per edge region per slot. It is sometimes referred to as a VIP.
      • path 614—an ordered set of CDN regions between an ddge region and a gateway region.
      • path Segment 616—a single hop of a path.
      • tunnel 618—a set of one or more paths from an edge server to a gateway server.
      • session 620—A single end-to-end connection from the client 622 to the target server; preferably, the session is defined by a five tuple (IP payload protocol, source address, destination address, source port, destination port). The source is the client and the destination is the target.
  • In the first embodiment, remote access, there are one or more clients that desire to send packets to a single IP address. FIG. 7 illustrates how the overlay IP mechanism achieves this operation. At step 1, the client 700 makes a DNS request to resolve a hostname. This hostname is aliased (e.g., by a CNAME) to a domain that is being managed by an authoritative DNS 702; typically, the authoritative DNS is managed by the CDN service provider. Preferably, this hostname corresponds to a single gateway region (and target address) 704. This is also referred to as a slot, as described above. At step 2, the DNS query returns a single IP address for the hostname. This address identifies a best performing available edge region 706 and, preferably, that region is dedicated to the hostname. The address is referred to as a virtual IP address, as described above. At step 3, the client 700 begins to send IP packets to the virtual IP address. These packets are received by a server in the edge region 706. The edge region 706 knows the gateway region 704 to which to send the packets based on the destination address in the IP packet header. The packet is then encapsulated. At step 4, and based on routes preferably provided by a CDN mapping system, the edge server in the edge region 706 sends out multiple copies of the encapsulated packets along multiple paths. One technique for performing this multiple path packet transport operation is described in U.S. Patent No. U.S. Pat. Nos. 6,665,726 and 6,751,673, assigned to Akamai Technologies, Inc. As illustrated at step 5, several intermediate servers receive the encapsulated packets and forward them (either directly, or through other Intermediate Regions, not shown) to the gateway region 704, once again, preferably based on routes provided from the CDN mapping system. At step 6, the packets are received by a server in the gateway region 704, where duplicates are removed. Destination NAT translates the virtual IP to the target address and source Network Address Port Translation is applied to the packet before it is sent, so that the return traffic will also be sent over the overlay network. Preferably, information is stored so that return traffic is sent to the edge region 706 from which the client packet originated. At step 7, the gateway region 704 receives an IP packet from the target address and de-NATs the packet. The packet is then encapsulated. At step 8, multiple copies of the packet are sent along multiple paths. At step 9, the intermediate servers send the packets back to the original edge region for this session. At step 10, the packets are received by an edge server and duplicates are removed. The packet is sourced from the virtual IP address and then sent back to the requesting client 700. This completes the end-to-end transmission.
  • The following provides additional details for a representative edge server, intermediate server, and gateway server.
  • I. Edge Servers
  • The edge server runs a process (called oipd, for convenience), that provides the following functions: receives packets on virtual IP addresses; filters packets based on expected protocol/ports, invalid or attack traffic, or specified access control lists (ACLs); encapsulates/decapsultes; forwards duplicate packets; and receives duplicate packets. Each of these functions is now described.
  • Receive Packets
  • IP packets destined for a virtual IP address should not be handled by an edge server's TCP/IP stack; they must always be tunneled or dropped. Thus, the edge server operating system kernel includes a hook to intercept packets (and pass them to oipd) before they are handled locally. In a representative embodiment, the edge server runs commodity hardware and the Linux operating system kernel. The kernel includes modules ip_tables, ip_queue and iptable_filter. Upon machine startup, a configuration script sets rules (in the ip_tables modules) such that all packets destined for any virtual IP address are delivered to user space. If no application is listening using the ip_queue module, the packets are dropped.
  • Packet Filtering
  • Preferably, three (3) types of packet filtering are supported: expected/allowed TCP/UPD; invalid or attack traffic; CDN service provider or customer-specified whitelist/blacklist ACLs. The edge server process oipd preferably filters TCP/UDP packets based on ports.
  • Encapsulation
  • An important function provided by the edge server process is encapsulation. An encapsulation header may contain the following information, or some portion thereof:
      • protocol version—standard version field.
      • TTL—the number of hops the packet can travel through before it is automatically sent to the gateway region (or edge region depending upon direction). It is decremented at each hop.
      • path number—each path a packet is sent along will have an identifier. This number preferably corresponds to an index specified in the map or other data structure.
      • forward state—used by the gateway server.
      • data length—the number of bytes in a payload of a OIP packet.
      • Source service address—the encapsulating machine's service address.
      • destination service address—the service address of the receiving side of the tunnel.
      • message sequence number—used to identify duplicate packets and determine loss across a tunnel.
      • OIP slot number—the slot for the packet.
      • serial number—determined by hashing information of the IP packet.
      • edge region number—the region number of the region sending the packet.
      • SRMM middle region map rule—determines which intermediate regions can be used for this slot.
      • message authentication code—used to determine authenticity of an OIP packet.
  • When a packet is to be encapsulated, the oipd process knows the CDN customer based on the virtual IP address in the destination address field in the IP header. The process then looks up a configuration file that contains the following map:
      • Virtual IP address→SRIP slot number
  • This is sometimes referred to as a “VIP map.” Preferably, this map is determined at install time and the various edge server components have a consistent view of it. The slot number is then used to look up a gateway region number for this customer, preferably using a “Slot Configuration Map” that is generated by a slot configuration file. The edge server process then hashes information into a serial number to break up the load into manageable chunks for load balancing. Preferably, the hash contains at least the source IP address. In the case that the next header is TCP or UDP, the source port and destination port may be hashed as well.
  • The edge server process subscribes to a first low level map (service B, as described below) that maps serial numbers to service addresses. In particular, the serial number is used to index the map to identify the gateway region that contains a single service address. This is the service address for which all (e.g., three (3)) copies of the packet will be sent. The edge server process then checks to see if it has forwarded any packets to this address in the past. If so, it increments a sequence number and uses this number in the “message sequence number” field in the encapsulation header. If the edge server process has not sent anything to this IP address yet, it initializes the state as described below.
  • Forward Packets
  • The “forward packets” function operates as follows. At this point, most of the encapsulation header is filled. The additional information for the header is generated as follows. Preferably, all edge servers subscribe to an assignment process (called SRMM and described below) that maps: MapperX.OIP.sr_assn_D_regionY where X is the correct Mapper prefix and Y is the region number. A destination region number and the SRMM middle region rule are used to index an assignment message. This yields one or more next hop region numbers, and preferably the number of next hop regions is configurable per slot. Each next hop is indexed with a path number. This path number is included in the encapsulation header so downstream intermediate nodes know their next hops. If an assignment message does not have any next regions, a single encapsulated packet is sent to the destination server address (and this should trigger an alert in the NOCC). For each of the next hop regions, a second low level map (service C, as described below) is checked. The serial number used to index this map preferably is the one included in the encapsulation packet header. Preferably, each of these maps contains only a single service address for each serial number. This is the service address where the packet should be forwarded to next. Preferably, the serial number is derived from connection identifying information (i.e., the five tuple). If the next region to be sent to is the gateway region, the first low level map does not need to be checked because this was already done by the sending edge server. Preferably, this initial lookup is done before a packet is sent along multiple paths to avoid having to synchronize intermediate regions.
  • With the header information finished, a MAC is computed for the header and the data. The computation may be based on SHA-1, MD5, or the like. To simplify the forwarding by intermediate nodes, the TTL field may not be included in the hash as this field is mutable.
  • Duplicate Packets
  • The edge server process preferably sends duplicate packets, as has been previously described. As noted above, preferably the edge server process sends multiple copies of each packet in several directions for redundancy. The receiving side needs an efficient way to filter out duplicates. When handling duplicate packets a first goal is to drop as few packets as possible; a next goal is to send as few duplicate packets as possible on to the target server. The edge server process tracks duplicate packets at the tunnel level. For every (edge source address, edge destination address) pair, the edge server process preferably maintains a sliding window of state indicating which packets have been received (and which have not) for every service address it receives a packet from. One processing algorithm is now described. The algorithm has one parameter, which is the size of the window, and two data objects to maintain: the window and the highest sequence number seen so far adjusted for wraparound. Preferably, the window size must be large enough so that it is not unnecessarily reset. So,
      • oip.packet.window.size>=(max_packet_rate*max_packet_age)
        On initialization, preferably, the entire window is initialized to NULL. The highest sequence number is set to the number of the first packet and the entry in the sliding window is set to SEEN. For every new packet, if the sequence number>highest sequence number+oip.packet.window size, the state is initialized and started again. If the packet is within the window but less than the highest so far, the algorithm checks if the entry in the window has already been set to SEEN. If so, the packet is dropped; otherwise, it is marked as SEEN. If the packet is greater than the highest so far, the algorithm sets the highest to the packet, marks that entry as SEEN and all entries between as UNSEEN. Using three values requires two bits for state, although using two values is sufficient for correctness. If a sender restarts, there is a small chance that a randomly chosen sequence number will be in the current window the receiver is maintaining. This could cause packets to be dropped unnecessarily. To prevent this, preferably a sender periodically writes a map of (service address, last sequence number) for every service address to which it has sent data. When an edge server starts up, it reads this file and adds a large number to it to ensure it is not in the window. This value should be safely larger than the window size.
  • As noted above, preferably an edge region comprises a number of edge servers. Accordingly, the system may implement an edge server failover function. This ensures that if a single edge server fails, the number of packets that will be dropped as a result is minimized. Each edge region preferably has buddy groups of a given number of configured machines. For duplicate removal to work on failover, the SEEN packet state is published to all machines in the buddy group periodically. This data is likely to be changing very frequently. So, preferably each machine sends an update to all machines in its buddy group indicating the highest sequence number SEEN. Each machine sends this information over the backend network to all machines in its buddy group, preferably using TCP.
  • II. Intermediate Servers
  • Intermediate servers have a “forward packets” functionality that is similar to that implemented in an edge server. The intermediate servers subscribes to the same MapperX.OIP.sr_assn_D_regionY channel. Each intermediate region is assigned a SRMM middle region rule and should only receive packets for slots that are configured for that rule. A destination region number and the slot's middle region rule are used to index an assignment message to determine the next hop. If a machine receives packets for a slot that has a different middle region rule, it should continue to send it on, but it should trigger an alert. If sending to another intermediate region, the second low level map is used to determine the service address of the next hop server. If the next region is the gateway region, the destination service address in the header preferably is used. Before the packet is sent on, the TTL field must be decremented. If the TTL reaches one, the packet is forwarded to the gateway region service address.
  • III. Gateway Servers
  • As mentioned above, preferably the gateway region is a special edge region that is generally located at the customer's data center. Ideally, this is somewhere close to the target server. The gateway server also runs an instance of the edge server process oipd, which can act simultaneously as an edge and gateway. Each machine in the gateway region preferably has its own CDN-specific IP address used for installs and secure access, and as a service address used for overlay network communication. Preferably, and as described above, a single VIP address is used for fixed client mapping and several NAT addresses are used. The gateway region preferably contains several machines for redundancy. These machines may be load balanced.
  • The gateway server provides the following functions: connection tracking, state synchronization, network address translation, sequence number translation, in-region packet forwarding, returning packets to the edge region. Each of these functions is now described in detail.
  • Connection Tracking
  • To track connections, the edge server process oipd makes use of NAT library such as libalias, which performs masquerading and IP address translation. This library preferably stores connection tracking information in a hash table or other data structure. When a new connection is established, a new entry is added to the table. When an existing entry is referenced, its timestamp is updated. When an existing entry has not be referenced in a given time period (the meaning of which varies based on protocol and connection state), the entry is deleted from the table. When a new connection track is created, in addition to being added to the standard libalias database, preferably it is also added to a list of newly-created but not yet synchronized connection tracks. When an existing connection track is modified, it is added to a list of modified but not yet synchronized entries, unless it is already in that list or in the new entries list. When an existing connection track is deleted, it is removed from the libalias database, and it is added to a list of deleted entries (unless it is currently in the new entries list, which means that information about it was never synchronized and so the deletion information does not need to be synchronized). The oipd process will periodically synchronize updates to the database (the new, modified, and deleted lists). This allows the connection tracking database to be shared across an entire gateway region so that per-connection state information can move between machines in the gateway region.
  • State Synchronization
  • The oipd process running in the gateway server preferably associates several pieces of information with a single connection. This information is synchronized across all machines in a gateway region. To this end, a series of functions is added to libalias to facilitate the gathering of data to be synchronized. There is one function for synchronization of each type of list (new, mod, delete), one for gathering all records to initialize a peer machine, and one for responding to a query for a single entry. These functions are now described.
  • A function GetSyncNewData builds the data packet for synchronization of all new records that have not yet been synchronized. In the case of TCP packets, only those packets that are considered fully connected (i.e. the SYN packet has been seen from both the client and the server) will be synchronized. Preferably, the oipd process ensures that a single connection will always be handled by the same machine until it is marked as fully connected. This ensures that potential race conditions related to synchronization of partially connected tracking entries and their expiration times can be avoided.
  • A function GetSyncModData builds the data packet for synchronization of modified records that have not yet been synchronized. It is desirable that an active entry be synchronized at least often enough to ensure that it does not time out in the database on a remote machine. Preferably, the oipd process ensures that every entry in the list is synchronized at least periodically (e.g., the timeout period for a UDP connection entry). This ensures that connection entries do not incorrectly timeout while at the same time limiting the bandwidth required to keep the gateway region synchronized.
  • A function GetSyncDelData gathers the information for deleted records and a function GetSyncAllData gathers the information for all entries in the database.
  • When synchronization data is received by a remote machine, it is passed in to a function SetSyncAddData if it applies to active connections (i.e. it was gathered by GetSyncAddData, GetSyncModData, or GetSyncAllData). This method creates a new connection entry in the local database or updates an existing entry, if there is one. For TCP connections, the state of the connection is tracked by libalias, preferably using two finite state machines, one for each direction (in/out). The function SetSyncAddData ensures that the finite state machine on the local machine follows valid transitions so that an active connection is not incorrectly marked as not connected or disconnected. Synchronization data relating to records to be deleted is passed to a function SetSyncDelData, which removes entries from the local table as long as the local table's timestamp for the entry is not more recent than that of the deleted entry.
  • Preferably, these libalias synchronization routines are used by a NAT_sync module, which contians a sender thread and a receiver thread. This module is initialized with the frontend and backend addresses of all its synchronization peers (e.g., all other machines in the gateway region). The sender thread connects to one of its peers to retrieve a full snapshot of the connection tracking table as part of its initialization procedure. It then tries each peer in turn until it finds one that is responsive; it then requests a table update from that machine. Preferably, no other overlay activities proceed until this initialization is complete or all peer nodes have been tried. The sender thread preferably uses a real-time clock to drive synchronization. For every clock iteration, the sender thread uses GetSyncAddData, GetSyncDelData, and GetSyncModData to collect data for synchronization. This data is then sent to all live peers in the region. To check for liveness, the sender thread preferably attempts to establish a TCP connect to a peer node using the last known good address (either the frontend or the backend). If that connection fails, then the sender thread attempts to establish a connection over the other address. If that fails, then the peer is assumed to be dead, although it may be tried again on subsequent synchronization attempts. The rate at which the sender's synchronization clock iterates is set in libalias for use by the GetSyncModData algorithm to determine how many modification records to send in each synchronization period. The rate is configurable. As noted above, on each clock iteration, all new connection entries (except half-connected TCP), a reasonable number of modification entries, and all deletion entries will be synchronized. The receiver thread simply waits for remote connections to come in, and then processes them according to message type. Update and deletion messages are passed to the appropriate SetSync method. Initialization requests cause the receiver to run a function GetSyncAllData, and then to send the data to the remote machine.
  • Network Address Translation
  • As described above, the overlay performs source network address translation (NAT) for all packets arriving on a service address (the gateway) that are to be sent to the target server. This ensures that the packets will be sent back through the overlay. Addresses for source NAT preferably are defined on a per-region and per-machine basis. Preferably, given machines are not assigned a static NAT address, as it may be desirable to move around NAT addresses. Also, when a packet arrives at a gateway server, preferably the NAT is applied differently depending on the type of connection involved. These connection types are configurable, e.g., per service or per port: TCP, long-lived; TCP, short-lived; UPD—session; UDP—query/response; and ICMP. Query/response UDP connections typically only involve a single UDP packet from client to server and zero or more response packets from server to client. For UDP query/response and ICMP, preferably a new NAT address, port/ICMP id is chosen for each incoming packet. Preferably, each machine in the gateway region is given its own port/ICMP id range for which only it is allowed to use to create new NAT sessions. Upon receiving a packet, the edge server process (described in more detail below) chooses a free address, port/ICMP id and sends the packet to the target server. The machine that owns the NAT address receives response packets from the server, but typically it will not have NAT session information to translate the addresses back. Thus, the machine preferably checks the destination port/ICMP id and forwards the response to the machine that owns that port/ICMP id. This second machine then translates the packet back to contain the original client address and port/ICMP id and sends it back over the overlay. This operation is illustrated in FIG. 8. For both types of TCP connections, preferably a new NAT address and port are chosen only when a SYN packet is seen. In either case, the NAT source port number is used to direct packets back to the server that created the session. For long-lived connections, it is desirable to reduce the amount of data that needs to be sent between gateway servers; preferably, this is accomplished by synchronizing the NAT session information. In particular, when a data packet arrives, an edge server process (as described below) determines if any connection state data exists. If so, the process applies the source NAT and sends out the packet. If connection state data is not available, the edge server process forwards the packet to an owning machine. For UDP connections that involve multiple packets from the client, a machine will check if it has session information for the client address and port. If so, the machine uses that NAT information. If not, the machine preferably acquires a region-wide lock for this client address, port. This ensures that if packets arrive simultaneously on two gateway servers, only one of them will establish the NAT session.
  • Sequence Number Translation
  • When the oipd process in the gateway server receives a SYN packet from the target server to the client, it performs a sequence number translation to tag the packet indicating its host id in the packet. Whenever this process receives a packet from the overlay, if it does not have the state information for the packet, it looks at the sequence number to determine the gateway server to which it should forward the packet. Preferably , each hosts is assigned a unique identifier. Each gateway region is assigned a maximum size and a sequence number space preferably is broken up into 2*max size pieces. To support SYN cookies, preferably only the highest eight bits are modified and each host is assigned two consecutive blocks of sequence number space. When the first packet arriving at a gateway region is handled by one server and a subsequent packet is handled by a different server before the state is synchronized, the second server preferably one uses the high eight bits in the ACK sequence number to determine the server to which server it should send the packet. This is illustrated in FIG. 9. Before sending a packet to the target server, the oipd process must unapply the sequence number translation so that the target server gets the correct ACK sequence number.
  • In-Region Forwarding
  • Before forwarding a packet to another gateway server, a forward state in a header field is modified to indicate that the packet has already been forwarded. The receiving machine handles this packet as if it was received from the overlay, but if the oipd process needs to forward the packet again for any reason, it will instead drop the packet.
  • Returning Packets to the Edge Region
  • Preferably, a packet is sent from the target server back through the overlay to the same edge region that sent it originally. This ensures that the edge server there will be able to send out an IP packet with the source address of the Virtual IP to which the client originally sent a packet. When a packet is received at the gateway region, the encapsulated IP packet contains the virtual IP that the client used and the header will contain the edge region number. This information is stored with the connection and synchronized in the same way as the NAT session information. When a gateway server receives a packet from the target server, it replaces the source IP address (which contains the target address) with the virtual IP stored. It then forwards the packet on to the correct region.
  • As in the edge server regions, preferably a gateway region implements a buddy group of servers. This group is configured for a region, and each server therein includes a monitoring process to check for liveness and load. As necessary, machine or software failover can then be implemented.
  • Load Reporting
  • The overlay mechanism preferably implements load balancing. To this end, preferably each server executes an in-machine DNS process that performs several load balancing functions: mapping Virtual IP addresses and NAT IP addresses to live edge servers, mapping serial numbers to live edge servers, and mapping serial numbers to live intermediate servers. These functions require three (3) distinct services: service B, service C, and service D. Service B represents the bytes entering the overlay, whereas service C represents the bytes exiting the overlay. FIG. 10 illustrates how different pieces of load information are reported on the different services.
  • Service B maps both virtual IP addresses and NAT addresses onto machines. At install time, a one-to-one mapping from slots to virtual addresses and NAT slots to NAT addresses is established. To ensure that all regions can support the same number of VIP slots, a network wide configuration variable is set, leaving a given set of serial numbers in a service B map for NAT addresses. NAT slots begin with a slot number and work downwards. For service B, the in-machine DNS process needs to know liveness of the edge server, capacity of the machine in bytes, and a byte load per slot. A monitoring process running on the machine is used to determine liveness. The oipd process reports the byte load in and out of the server to the monitoring process. For service C, the in-machine DNS process needs to know the liveness of the edge server, a capacity of the machine (according to a given metric), and a value of that metric per serial number. A representative metric is a “flit.” A “flit” is an arbitrary unit of work generally representing non-bandwidth resource usage on a given server machine. Such utilization typically encompasses CPU utilization, disk utilization, operating system abstraction-limited resources such as threads and semaphores, and the like, and combinations thereof. In a representative embodiment, a flit is a given linear or convex function of several individual machine variables, such as CPU and disk utilizations. For the load balancing described generally below, however, the number of bytes entering the server is a good approximation for the flit value. For service D, the in-machine DNS process needs to know the liveness of the intermediate server, a capacity of the machine (according to the given metric), and a value of that metric per serial number.
  • Region Assignment
  • A region assignment process executes in the overlay mechanism's authoritative DNS. Because every connection mapped to a region has corresponding traffic to that region from client→server and server→client, both services B and C need to be taken into account. To this end, a region monitor process reports a sum of bytes in and out on both services B and C. The region assignment process needs to know an edge region flit capacity metric, and an edge region byte load and flit load (for services B and C). Each connection mapped to a region affects the amount of load the region sends and receives because packets preferably are sent to the edge region from which they originated. To handle this properly, the region assignment process preferably uses the maximum of bytes in, bytes out to determine how loaded a region is.
  • Map Maker
  • The assignment process (SRMM) uses two pieces of information: bytes per destination region (same as slot), edge region pair; and region flit load for intermediate regions. An oipd process on the edge reports byte load that will be injected into the overlay, preferably broken down by destination region. This is the demand that is put on the overlay and is the value sent in a new message from the region monitor to the SRMM process. The intermediate regions report total service D bytes and flits. Because flit load correlates to byte load, the SRMM process preferably use the edge bytes as demand and the flit cap and a byte/flit ratio to infer a reasonable byte cap in the middle regions. This information goes into an existing region assignment message to which SRMM then subscribes.
  • The (SRMM) process generates region level maps that specify the best paths from an edge region to a gateway region, as well as the best paths from the gateway region back to the edge region. As noted above, preferably SRMM receives demand input from edge regions and load and capacity information from the middle regions. In addition, each intermediate region is assigned to one or more middle region map rules and each slot will be assigned to a particular middle region map rule. Preferably, SRMM only maps traffic for a particular slot through intermediate regions that have the same map rule.
  • In operation, SRMM determines multiple paths from every edge region to every gateway region while ensuring that no two paths share the same intermediate region and that all intermediate regions are properly load balanced. By default, it will choose three paths. To handle these requirements, preferably the following simplifying assumptions are made for the SRMM algorithm: the first path chosen is optimal (i.e. it can support multi-hop paths) given information from a ping subsystem providing there is sufficient capacity; and, the next two paths contain only a single intermediate region.
  • The path determination is broken down into two parts corresponding to the assumptions above. The first part involves running a sequence of shortest path algorithms. For each middle map rule, Dijkstra's algorithm is run for each destination, where destination is all edge and gateway regions. The edge byte load is used as demand in these calculations and a running total is tracked for each intermediate region. If the total load is under capacity for each intermediate region, this part of the path determination is done. If it is not, all nodes that are over capacity will have a price adjustment, and the algorithm is repeated. The price adjustment is applied by multiplying a constant factor to the scores of each of the links that use the overflowing node. This process is repeated until stable. Then, the capacities at each node are updated.
  • In essence, a list of (source, destination) pairs of (edge regions gateway regions) is determined. For each one, the algorithm chooses the best intermediate node with sufficient capacity for that pair. The output of the algorithm creates an IP assignment message for service D per region. It is indexed by (destination region, path number).
  • Edge Mapping
  • As noted above, to use the service offering, customers CNAME a hostname to the CDN service provider's DNS, which then becomes authoritative for that domain. If desired, the hostname may in turn be configured to a global traffic management hostname to provide additional advantages.
  • By way of background, the overlay mechanism may use “per-customer” regions. A per-customer region supports traffic for only a single slot. Such an approach is desirable for users behind a corporate firewall. In particular, such users can then be mapped to a CDN region located near the target server that will keep LAN performance from degrading into WAN performance by going to a public CDN edge region. In FIG. 11, the client behind a corporate firewall IP is mapped directly to the gateway region, while other clients are mapped to public regions. Using global traffic management, the users with specific, predefined source IP addresses are mapped to these “per-customer” regions.
  • FIG. 12 is a process flow diagram that illustrates how the global traffic management (GTM) process may be implemented within the overlay to provide this functionality. Where information is not available at a given decision point, preferably the “No” path is taken. Thus, if the GTM does not have any predefined mapping data, the client is mapped to a public region. Likewise, if the GTM does not have any geo information, the client is mapped to a public region. If the GTM does not have liveness feedback information , it assume the region is down and maps the client using the direct IP address. The GTM preferably obtains liveness information from the gateway region by running a ping test to the virtual IP in that region.
  • Fail-Safe Mapping
  • By way of background, a customer's target server address may be considered a fail-safe IP address. If the overlay detects any end-to-end problems (either customer-specific or otherwise), preferably the CDN's authoritative DNS hands out this IP address. This has the effect of removing the overlay from all communications. FIG. 13 illustrates normal overlay routing, and FIG. 14 illustrates the fail-safe operation. The fail-safe operation may be triggered as a result of various occurrences, e.g., any per-customer per-region problems, where multiple regions are down or out of capacity, where the GTM does not have liveness information from testing the virtual IP at a gateway region, where one or more name server processes do not have sufficient information to make assignments, or the like.
  • Management Portal
  • The CDN service provider may provide a customer facing extranet that allows customers to log-in and perform necessary routing configuration as well as view performance and traffic utilization statistics. Routing configuration includes entering BGP information so the edge servers may establish a BGP session with the local router and the application IP information for the applications that will use the overlay network.
  • The overlay mechanism of the present invention provides numerous advantages. Preferably, the mechanism continuously monitors and analyzes performance from each local edge node to the intermediate nodes, from each local edge node to each remote edge node, and between all intermediate nodes. Thus, for example, these performance measurements include latency and packet loss. This data is then combined and analyzed to determine optimal paths between two edge locations. If the overlay network is able to provide a better path than the BGP-determined path, the edge nodes will then intercept packets and route them via intermediate nodes. In addition to routing around poor performing paths, the overlay also uses packet replication to ensure uptime. The performance measurement granularity of the overlay is such that an outage could otherwise disrupt the transmission of packets. To avoid this problem, packets are replicated at the edge and sent via multiple overlay paths. One of the paths may be the directly routed path, but packets are intercepted and encapsulated at the sending edge and de-encapsulated at the remote edge. Preferably, the first packet received at the remote end is used and the others are dropped. In an alternative embodiment, a fail-safe IP address (typically the target server's IP address) is handed out by the CDN authoritative DNS under certain conditions.
  • Remote Access Embodiment
  • In the remote access embodiment illustrated in FIG. 15, multiple clients 1502 a-n are sending packets to a single IP address, the target server 1504. The overlay mechanism 1500 comprises the authoritative DNS 1506, at least one edge region 1508, one or more intermediate regions 1510, and one or more gateway regions 1512. An application of interest is executable in part on the target server. A client side of the application executes on the client machine, which may be a laptop, a mobile computing device, or the like. The particular functionality of the application (or how the application is implemented in a distributed manner across the client and server) is transparent to the overlay. It is merely assumed that communications between client and server occur over an IP transport. In this embodiment, the application is associated with an Internet domain. As has been previously described, that domain is aliased (via a CNAME) to an overlay network domain being managed by the service provider. A DNS query to the application domain causes the authoritative DNS 1506 to return a VIP address. A given client (whose machine typically runs a client side application or instance) then connects to the application on the target server through the overlay mechanism as has been previously described. In particular, data packets destined for the application are encapsulated at the edge, duplicated, forwarded over multiple paths, and then processed at the gateway to remove duplicates. At the gateway, destination NAT translates the virtual IP to the target address and source Network Address Port Translation is applied to the packet before it is sent, so that the return traffic will also be sent over the overlay network. Preferably, information is stored so that return traffic is sent to the edge region from which the client packet originated. When the application responds, the gateway region receives an IP packet from the target address and de-NATs the packet. The packet is then encapsulated. Multiple copies of the packet are then sent along multiple paths. The intermediate servers send the packets back to the original edge region for this session. The packets are received by an edge server and duplicates are removed. The packet is sourced from the virtual IP address and then sent back to the requesting client.
  • Remote Office Embodiment
  • In this embodiment, as illustrated in FIG. 16, an application executes on a target server 1604 and the client machine 1602 (which itself may be an application or application instance) is on a fixed, remote endpoint. In a representative example, the client 1602 and server 1604 are located at a pair of geographically-distributed sites. For ease of discussion, it is assumed that the client machine 1602 is located outside an enterprise firewall but adjacent a router 1608. In this embodiment, the client machine 1602 executes remote office acceleration (ROA) process 1610. The process 1610 advertises itself as the target server IP address; thus, router 1608 sees that process as a router. By advertising the target server IP address, the process 1610 establishes a BGP session with the router 1608 and thereby transparently intercepts data packets intended for the application located at the remote office. Once intercepted, the process 1610 performs the encapsulation and other functions performed by the edge server in the remote access embodiment. The overlay mechanism provides the packet communications between the two sites in the manner previously described.
  • In the remote office embodiment, the overlay provides a method of data encapsulation by degenerate use of the BGP protocol. Generalizing, this technique can be used for any given Internet protocol including EGRP, OSPF, and the like. While the degenerative BGP approach is preferred, packets may also be provided to the overlay using an in-line approach (e.g., a packet grabber).
  • Variants:
  • To save both client bandwidth and reduce service provider bandwidth cost, it may be desirable to implement a dynamic decision as to what degree to replicate packets. Thus, for example, it may be desired to start with a given number (e.g., three (3) copies) and then reduce this number dynamically if the loss/latency is acceptable.
  • Generalizing, the overlay mechanism as described about may be generalized as a routing “cloud” in which the arbitrary data flows are implemented intelligently. Within the cloud, and using the above-described transport techniques, the data may pass through one or more intermediate nodes, be transmitted along redundant paths, employ TCP or UDP as a transport mechanism, obey flow or class specific logic for determining optimality in a multidimensional parameter space, or any combination thereof.
  • Based on the architecture and billing model employed, a local controller or “knob machine” may be employed at the overlay ingress points. The motivation for this element is that bandwidth flowing to the cloud and within the cloud has an inherent cost. In some cases (e.g., all VPN traffic is being handled) it would not make sense for all of this traffic to flow to the cloud because at minimum the cost basis for the solution is twice the alternative. Thus, the purpose of the controller is to provide a simple and effective means of supplying the system with cost/benefit tradeoff business logic. (As such, the box is effectively a knob controlling how aggressively the system is used—hence the name.) Based on the rules supplied, the controller makes a decision on a per packet or per flow basis as to whether or not the traffic should be sent through the cloud or directly to the other end point. By default, preferably all traffic would flow through the controller. However, the controller would be configurable so as to employ appropriate business logic to decide when a flow should be sent to the cloud. This information may also affect the behavior that occurs within the cloud as well.
  • The rules that could be employed by the controller are quite flexible. They include selecting traffic based on: domain (e.g., Intranet traffic is important but a given content provider is not), IP Address (e.g., important to route traffic to the Tokyo office but not necessarily Chicago), performance predictions (e.g., the controller can understand various QoS metrics about the direct and alternate paths using the parent servers and chose to direct the traffic if the predicted improvement is greater than a given threshold and/or if the direct path's quality is below a certain threshold), reliability predictions (e.g., the controller can understand various reliability metrics about the direct and alternate paths and chose to use an alternate path through a parent server for the traffic if the predicted improvement is greater than a given threshold and/or if the direct path's quality is below a certain threshold), or the like. Of course, these rules are more powerful when used together. For example, one could choose to set the performance and reliability metrics differently for different domains.
  • Thus, path optimization is the general idea of getting a data packet from one node to another via a CDN region based on performance data that has been collected over time. This technique, as described above, has been used to improve connectivity back to a customer origin server for HTTP traffic. Moreover, the ability to tune the service feature so that the feature is invoked preferably when it can improve performance enough (e.g., latency reduced by x %). According to the present invention, the content delivery network is viewed as a “cloud” into which data flows may enter, get optimally directed across the “middle” (e.g. by the optimization technique), and appear on the other side as well as possible on their way to their destination. For example, consider the case of VPN traffic coming out of an office on the East Coast (e.g., Cambridge) on its way to an office in California (e.g., in San Mateo). Instead of just using BGP, the present invention may direct this traffic as an “entry point.” This traffic then flows through the network, taking the appropriate number of bounces, and then exits the network at an “exit point” near the appropriate office. In a particular implementation, a controller is placed at the Cambridge office through which all traffic flows. This controller is used to decide, in real-time based on both business rules (e.g., traffic to a first domain is important but traffic to a second domain is not) and quality-of-service (QoS) rules (e.g., use the alternate path only if there is no direct path or if the alternate path is some % better). Of course, these examples do not limit the present invention.
  • Although not meant to be limiting, the controller machine can be implemented in an edge or ISP network. Several potential applications are now described.
  • Web Transactions
  • Entities that provide secure transactions on the Web may use the present invention to improve the reliability and performance of those transactions. One example would be a provider of online credit card transactions. Transactions are appealing both because they are by definition not cacheable and, moreover, because each request is valuable. The present invention can facilitate Web transactions by simply adding HTTPS support to the edge server optimization technique. In this context, CDN regions that support secure content (e.g., via SSL) can be view as “entry points” to the HTTPS routing cloud, with the connections to the origin or through the network being provided accordingly. In essence, the HTTP edge server implementation is extended into the generalized cloud, where every edge region can serve as an entry point and where a certain set of regions (e.g., parent regions) can serve as exit points.
  • Another area of applicability is that fact that both the secure and HTTP clouds support all SOAP-based transactions inasmuch as SOAP is implemented in HTTP. Thus, the present invention can be used to implement SOAP-based Web transactions.
  • VPN QoS
  • Another opportunity for valuable flows lies in the realm of VPNs. VPN traffic has the two attractive properties of being associated with information that the business finds valuable and also by definition flowing between two geographically disparate locations. The simple idea is to see VPN traffic as an encrypted IP data stream with end-to-end semantics behind the firewalls. Thus, it is possible to provide a UDP-based transport mechanism through the cloud. Most likely, the traffic would flow transparently through a knob machine and directed into the cloud appropriately. In this fashion, all traffic on the VPN will benefit from the service based on the knob settings.
  • This technique also is compatible with enterprise content delivery network offerings. With an enterprise CDN and the cloud, the CDN service provider can improve the performance and reliability of every piece of data in the company if the company has a VPN-based architecture. For example, all VoD content and cacheable web content may be cached in the customer's office. However, live streaming cannot be cached. The enterprise CDN can rate limit the traffic to protect finite resources. The cloud provides the redundancy and retransmits to improve the quality of these streams. In addition, other data flows, such as P2P videoconferencing, database synchronization, and the like, can benefit from this functionality.
  • Web Service Networks
  • As web services become more widely accepted, one of the key weaknesses of the systems will be the messaging that occurs between them. While there are several protocols being developed to support this functionality (SOAP, UDDI, etc.) there are two missing ingredients. The first is a set of application layer functionality, such as security, queuing, and non-repudiation/logging. The second is a transport mechanism for these messages.
  • One of ordinary skill will appreciate that the above-described overlay technologies are advantageous as they determine the entire outbound and inbound routes for a given communication using the BGP routing layer as a black box, and hence are not susceptible to BGP-related issues.
  • One of ordinary skill in the art will recognize that the present invention facilitates Layer 3 general purpose routing of IP traffic within a distributed, potentially global computer network for increased performance and reliability, perhaps dependant on application specific metrics. The system will accept arbitrary IP flows and will route them through the content delivery network to attempt achieve a set of metrics, e.g., metrics established by metadata. The metadata could be applied in a number of ways based on such properties as the source, destination, flow, application type, or the like. To achieve these metrics, the system also could decide to route the traffic on a per-hop and per-packet basis along an alternate route and/or multiple routes.
  • Having described our invention, what we now claim follows below.

Claims (3)

1. An overlay network that provides a plurality of client machines remote access to an application executing on a target server, wherein each client machine communicates with the application over the Internet using Internet Protocol (IP) transport, comprising:
a domain name service that is authoritative for a hostname associated with the application;
a first server, a set of second servers, and a third server, wherein each server in the overlay network receives and processes communications over IP, the first server having a virtual IP address determined by resolution of the hostname associated with the application;
wherein, for each IP-based request data packet to be communicated between a client machine and the application executing on the target server, the first server encapsulates the request data packet, duplicates the request data packet as encapsulated, and forwards the request data packet as encapsulated to the third server, the request data packet being forwarded to the third server over each of a set of paths that include at least one second server, the set of paths including at least first and second paths from the first server to the third server that do not share a same second server, the first server subsequently receiving a response to the request data packet at the virtual IP address; and
wherein the third server processes received data to recover the request data packet, applies a network address translation to the request data packet as recovered, and forwards the request data packet to the target server for further processing by the application, wherein the network address translation also applies a source NAT to the packet before it is forwarded to the target server.
2. The overlay network as described in claim 1 wherein the third server receives the response from the target server as a response data packet, encapsulates the response data packet, duplicates the response data packet as encapsulated, and forwards the response data packet as encapsulated over a set of paths.
3. The overlay network method as described in claim 2 wherein the first server processes received data to recover the response data packet, and forwards the response data packet to the client machine.
US14/542,866 2005-12-30 2014-11-17 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows Abandoned US20150074187A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US11/323,342 US7660296B2 (en) 2005-12-30 2005-12-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US12/701,718 US8170017B2 (en) 2005-12-30 2010-02-08 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US13/459,749 US8891522B2 (en) 2005-12-30 2012-04-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US14/542,866 US20150074187A1 (en) 2005-12-30 2014-11-17 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US14/542,866 US20150074187A1 (en) 2005-12-30 2014-11-17 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US13/459,749 Continuation US8891522B2 (en) 2005-12-30 2012-04-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Publications (1)

Publication Number Publication Date
US20150074187A1 true US20150074187A1 (en) 2015-03-12

Family

ID=38224310

Family Applications (4)

Application Number Title Priority Date Filing Date
US11/323,342 Active 2026-07-08 US7660296B2 (en) 2005-12-30 2005-12-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US12/701,718 Active US8170017B2 (en) 2005-12-30 2010-02-08 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US13/459,749 Active US8891522B2 (en) 2005-12-30 2012-04-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US14/542,866 Abandoned US20150074187A1 (en) 2005-12-30 2014-11-17 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Family Applications Before (3)

Application Number Title Priority Date Filing Date
US11/323,342 Active 2026-07-08 US7660296B2 (en) 2005-12-30 2005-12-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US12/701,718 Active US8170017B2 (en) 2005-12-30 2010-02-08 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
US13/459,749 Active US8891522B2 (en) 2005-12-30 2012-04-30 Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Country Status (5)

Country Link
US (4) US7660296B2 (en)
EP (1) EP1972112A4 (en)
JP (1) JP4975760B2 (en)
CN (1) CN101410819B (en)
WO (1) WO2007089352A2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9426690B2 (en) 2010-12-07 2016-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Method for enabling traffic acceleration in a mobile telecommunication network
WO2018148302A1 (en) * 2017-02-07 2018-08-16 Level 3 Communications, Llc System and method for next hop bgp routing in a network
CN108667658A (en) * 2018-04-28 2018-10-16 厦门白山耘科技有限公司 A kind of bandwidth reuse method and device
US10348589B2 (en) * 2015-06-15 2019-07-09 Netflix, Inc. Managing networks and machines that deliver digital content

Families Citing this family (129)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7111163B1 (en) 2000-07-10 2006-09-19 Alterwan, Inc. Wide area network using internet with quality of service
US7660296B2 (en) * 2005-12-30 2010-02-09 Akamai Technologies, Inc. Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows
KR101185570B1 (en) * 2006-03-04 2012-09-24 삼성전자주식회사 Resource reservation method using multiple interfaces in mobile environments
US8181227B2 (en) * 2006-08-29 2012-05-15 Akamai Technologies, Inc. System and method for client-side authenticaton for secure internet communications
US8249081B2 (en) * 2006-09-29 2012-08-21 Array Networks, Inc. Dynamic virtual private network (VPN) resource provisioning using a dynamic host configuration protocol (DHCP) server, a domain name system (DNS) and/or static IP assignment
ZA200903984B (en) * 2006-11-13 2010-08-25 Obvious Ltd B Selective session interception method
US7852861B2 (en) * 2006-12-14 2010-12-14 Array Networks, Inc. Dynamic system and method for virtual private network (VPN) application level content routing using dual-proxy method
US8489670B1 (en) * 2006-12-26 2013-07-16 Akamai Technologies, Inc. Reducing TCP connection establishment time in an overlay network
US7840701B2 (en) * 2007-02-21 2010-11-23 Array Networks, Inc. Dynamic system and method for virtual private network (VPN) packet level routing using dual-NAT method
US20080208961A1 (en) * 2007-02-23 2008-08-28 Hostway Corporation Parallel retrieval system
JP5071165B2 (en) * 2007-03-19 2012-11-14 日本電気株式会社 Route multiplexing communication system, communication node, and communication method
US8695089B2 (en) * 2007-03-30 2014-04-08 International Business Machines Corporation Method and system for resilient packet traceback in wireless mesh and sensor networks
EP2227918B1 (en) * 2007-06-29 2011-11-02 Vodafone Group PLC Method and node to control access to a telecommunications network core
GB2450575B (en) * 2007-06-29 2012-06-20 Vodafone Plc Controlling the use of access points in a telecommunications network
US7961711B2 (en) * 2007-08-06 2011-06-14 Microsoft Corporation Fitness based routing
US8543667B2 (en) 2008-01-14 2013-09-24 Akamai Technologies, Inc. Policy-based content insertion
US20090248793A1 (en) * 2008-03-25 2009-10-01 Contribio Ab Providing Content In a Network
US7865586B2 (en) * 2008-03-31 2011-01-04 Amazon Technologies, Inc. Configuring communications between computing nodes
US8429739B2 (en) 2008-03-31 2013-04-23 Amazon Technologies, Inc. Authorizing communications between computing nodes
US7818430B2 (en) * 2008-10-15 2010-10-19 Patentvc Ltd. Methods and systems for fast segment reconstruction
WO2010045511A2 (en) * 2008-10-15 2010-04-22 Gal Zuckerman Methods and systems for delivering content
US9043919B2 (en) 2008-10-21 2015-05-26 Lookout, Inc. Crawling multiple markets and correlating
US8533844B2 (en) 2008-10-21 2013-09-10 Lookout, Inc. System and method for security data collection and analysis
US8984628B2 (en) 2008-10-21 2015-03-17 Lookout, Inc. System and method for adverse mobile application identification
US9235704B2 (en) 2008-10-21 2016-01-12 Lookout, Inc. System and method for a scanning API
US8108933B2 (en) 2008-10-21 2012-01-31 Lookout, Inc. System and method for attack and malware prevention
US9781148B2 (en) 2008-10-21 2017-10-03 Lookout, Inc. Methods and systems for sharing risk responses between collections of mobile communications devices
US9367680B2 (en) 2008-10-21 2016-06-14 Lookout, Inc. System and method for mobile communication device application advisement
US8347386B2 (en) 2008-10-21 2013-01-01 Lookout, Inc. System and method for server-coupled malware prevention
US8087067B2 (en) 2008-10-21 2011-12-27 Lookout, Inc. Secure mobile platform system
US20110219109A1 (en) * 2008-10-28 2011-09-08 Cotendo, Inc. System and method for sharing transparent proxy between isp and cdn
CN101409684B (en) * 2008-11-25 2011-09-14 清华大学 Method for distributing internet high-performance address based on three-layer soft exchange
JP2010153955A (en) * 2008-12-24 2010-07-08 Hitachi Ltd Switching exchange
US8467768B2 (en) 2009-02-17 2013-06-18 Lookout, Inc. System and method for remotely securing or recovering a mobile device
US8538815B2 (en) 2009-02-17 2013-09-17 Lookout, Inc. System and method for mobile device replacement
US9955352B2 (en) 2009-02-17 2018-04-24 Lookout, Inc. Methods and systems for addressing mobile communications devices that are lost or stolen but not yet reported as such
US8416692B2 (en) * 2009-05-28 2013-04-09 Microsoft Corporation Load balancing across layer-2 domains
US8369332B2 (en) * 2009-08-21 2013-02-05 Alcatel Lucent Server-side load balancing using parent-child link aggregation groups
US9083587B2 (en) * 2009-08-21 2015-07-14 Cisco Technology, Inc. Port chunk allocation in network address translation
US20110110377A1 (en) * 2009-11-06 2011-05-12 Microsoft Corporation Employing Overlays for Securing Connections Across Networks
KR20110071774A (en) * 2009-12-21 2011-06-29 한국전자통신연구원 Smart border router and method for transmitting flow using the same
US8898287B2 (en) * 2010-02-24 2014-11-25 Salesforce.Com, Inc. System, method and computer program product for monitoring data activity utilizing a shared data store
US9137213B2 (en) * 2010-03-26 2015-09-15 Avaya Inc. On-demand feature server activation in the cloud
JP5515985B2 (en) * 2010-04-02 2014-06-11 富士通株式会社 Monitoring device, failure section identification method, and failure section identification program
US20130159547A1 (en) * 2010-09-02 2013-06-20 Nec Corporation Data transfer system
JP5803924B2 (en) * 2010-09-02 2015-11-04 日本電気株式会社 data transfer system
US8560646B1 (en) 2010-09-28 2013-10-15 Amazon Technologies, Inc. Managing communications using alternative packet addressing
EP2638688B1 (en) * 2010-11-08 2018-10-10 Telefonaktiebolaget LM Ericsson (publ) Traffic acceleration in mobile network
US8806032B2 (en) * 2010-12-15 2014-08-12 At&T Intellectual Property I, L.P. Methods and apparatus to migrate border gateway protocol sessions between routers
US8549347B1 (en) 2010-12-20 2013-10-01 Amazon Technologies, Inc. Techniques for network replication
JP5526015B2 (en) * 2010-12-22 2014-06-18 株式会社日立製作所 Gateway system, gateway device, and load balancing method
CN102164078B (en) * 2011-03-25 2014-07-02 北京星网锐捷网络技术有限公司 Policy routing method, device and system
US9172774B2 (en) * 2011-04-13 2015-10-27 Qualcomm Incorporated Technique for managing communications at a router
US8732268B2 (en) * 2011-04-19 2014-05-20 Microsoft Corporation Global traffic management using modified hostname
US8867337B2 (en) 2011-04-26 2014-10-21 International Business Machines Corporation Structure-aware caching
US8738765B2 (en) * 2011-06-14 2014-05-27 Lookout, Inc. Mobile device DNS optimization
US8467395B1 (en) * 2011-07-12 2013-06-18 Qlogic, Corporation Method and system for link aggregation
US8488601B1 (en) 2011-07-12 2013-07-16 Qlogic, Corporation Method and system for link aggregation
US8788881B2 (en) 2011-08-17 2014-07-22 Lookout, Inc. System and method for mobile device push communications
US8625452B2 (en) * 2011-09-15 2014-01-07 International Business Machines Corporation Maintenance of high-speed channels by inserting channel maintenance data in a mobile data network to avoid channel type switching
US9014023B2 (en) 2011-09-15 2015-04-21 International Business Machines Corporation Mobile network services in a mobile data network
CN104081739B (en) 2011-12-23 2018-03-02 阿卡麦科技公司 Compression and the data difference device and system in Intrusion Detection based on host/path of differentiation engine are utilized in overlay network
US9742858B2 (en) 2011-12-23 2017-08-22 Akamai Technologies Inc. Assessment of content delivery services using performance measurements from within an end user client application
CN102571817B (en) * 2012-02-15 2014-12-10 华为技术有限公司 Method and device for accessing application server
US9141957B2 (en) * 2012-04-11 2015-09-22 Netgear, Inc. System and method for filtering advertising in a networking device
US9641405B2 (en) * 2012-05-31 2017-05-02 Ca, Inc. System and method for sequencing per-hop data in performance-monitored network environments
US9407443B2 (en) 2012-06-05 2016-08-02 Lookout, Inc. Component analysis of software applications on computing devices
US9589129B2 (en) 2012-06-05 2017-03-07 Lookout, Inc. Determining source of side-loaded software
US8521153B1 (en) 2012-06-18 2013-08-27 International Business Machines Corporation Using the maintenance channel in a mobile data network to provide subscriber data when a cache miss occurs
US9875267B1 (en) * 2012-09-28 2018-01-23 EMC IP Holding Company LLC State-based message generation
US8875287B2 (en) 2012-10-04 2014-10-28 Akamai Technologies, Inc. Server with mechanism for reducing internal resources associated with a selected client connection
CN102917047B (en) * 2012-10-17 2015-08-05 成都索贝数码科技股份有限公司 System and method for implementing a digital file streams in the Internet environment
US8495221B1 (en) * 2012-10-17 2013-07-23 Limelight Networks, Inc. Targeted and dynamic content-object storage based on inter-network performance metrics
US9386514B2 (en) * 2012-10-25 2016-07-05 Raytheon Company Hybrid communication system for smartphone/tablet network
US8655307B1 (en) 2012-10-26 2014-02-18 Lookout, Inc. System and method for developing, updating, and using user device behavioral context models to modify user, device, and application state, settings and behavior for enhanced user security
US8626876B1 (en) 2012-11-28 2014-01-07 Limelight Networks, Inc. Intermediate content processing for content delivery networks
EP2922251A4 (en) * 2012-12-09 2015-10-14 Huawei Tech Co Ltd Packet forwarding method, device and data center network
US9654579B2 (en) 2012-12-21 2017-05-16 Akamai Technologies, Inc. Scalable content delivery network request handling mechanism
US9509804B2 (en) 2012-12-21 2016-11-29 Akami Technologies, Inc. Scalable content delivery network request handling mechanism to support a request processing layer
US9208215B2 (en) 2012-12-27 2015-12-08 Lookout, Inc. User classification based on data gathered from a computing device
WO2014105906A1 (en) 2012-12-27 2014-07-03 Akamai Technologies, Inc. Stream-based data deduplication using peer node graphs
US9729605B2 (en) 2012-12-27 2017-08-08 Akamai Technologies Inc. Mechanism for distinguishing between content to be served through first or second delivery channels
US9374369B2 (en) 2012-12-28 2016-06-21 Lookout, Inc. Multi-factor authentication and comprehensive login system for client-server networks
US8855599B2 (en) 2012-12-31 2014-10-07 Lookout, Inc. Method and apparatus for auxiliary communications with mobile communications device
US9424409B2 (en) 2013-01-10 2016-08-23 Lookout, Inc. Method and system for protecting privacy and enhancing security on an electronic device
US9338210B2 (en) * 2013-03-15 2016-05-10 Rhapsody International Inc. Distributing content items to users
US9565101B2 (en) * 2013-04-22 2017-02-07 Fujitsu Limited Risk mitigation in data center networks
US9503367B2 (en) 2013-12-10 2016-11-22 Fujitsu Limited Risk mitigation in data center networks using virtual machine sharing
EP2809033B1 (en) * 2013-05-30 2018-03-21 Solarflare Communications Inc Packet capture in a network
US8601565B1 (en) * 2013-06-19 2013-12-03 Edgecast Networks, Inc. White-list firewall based on the document object model
US9391847B2 (en) 2013-08-08 2016-07-12 Level 3 Communications, Llc Content delivery methods and systems
CN104427488A (en) * 2013-08-29 2015-03-18 中兴通讯股份有限公司 Terminal and wireless networking method
US9648125B2 (en) 2013-10-04 2017-05-09 Akamai Technologies, Inc. Systems and methods for caching content with notification-based invalidation
US9641640B2 (en) 2013-10-04 2017-05-02 Akamai Technologies, Inc. Systems and methods for controlling cacheability and privacy of objects
US9813515B2 (en) 2013-10-04 2017-11-07 Akamai Technologies, Inc. Systems and methods for caching content with notification-based invalidation with extension to clients
US9577910B2 (en) 2013-10-09 2017-02-21 Verisign, Inc. Systems and methods for configuring a probe server network using a reliability model
CN103546559B (en) * 2013-10-24 2018-02-02 网宿科技股份有限公司 Data distributing method and device
US9642008B2 (en) 2013-10-25 2017-05-02 Lookout, Inc. System and method for creating and assigning a policy for a mobile communications device based on personal data
US9819721B2 (en) 2013-10-31 2017-11-14 Akamai Technologies, Inc. Dynamically populated manifests and manifest-based prefetching
EP3078173A4 (en) * 2013-12-02 2017-06-21 Akamai Technologies, Inc. Virtual private network (vpn)-as-a-service with delivery optimizations while maintaining end-to-end data security
US10122747B2 (en) 2013-12-06 2018-11-06 Lookout, Inc. Response generation after distributed monitoring and evaluation of multiple devices
US9753796B2 (en) 2013-12-06 2017-09-05 Lookout, Inc. Distributed monitoring, evaluation, and response for multiple devices
US9467478B1 (en) * 2013-12-18 2016-10-11 vIPtela Inc. Overlay management protocol for secure routing based on an overlay network
US9712438B2 (en) 2014-01-08 2017-07-18 Microsoft Technology Licensing, Llc Routing messages between virtual networks
US9887914B2 (en) * 2014-02-04 2018-02-06 Fastly, Inc. Communication path selection for content delivery
US9325586B1 (en) * 2014-03-26 2016-04-26 Marvell Israel (M.I.S.L.) Ltd. Packet duplication measurement in a network device
US9866655B2 (en) 2014-03-31 2018-01-09 Akamai Technologies, Inc. Server initiated multipath content delivery
US9576070B2 (en) 2014-04-23 2017-02-21 Akamai Technologies, Inc. Creation and delivery of pre-rendered web pages for accelerated browsing
US9887874B2 (en) * 2014-05-13 2018-02-06 Cisco Technology, Inc. Soft rerouting in a network using predictive reliability metrics
US9678841B2 (en) * 2014-05-30 2017-06-13 Fastly, Inc. Communication continuation during content node failover
CN104113475B (en) * 2014-07-30 2018-09-11 深圳市共进电子股份有限公司 The network route implementation method and system of home gateway based on Linux
US10135956B2 (en) 2014-11-20 2018-11-20 Akamai Technologies, Inc. Hardware-based packet forwarding for the transport layer
US10154068B2 (en) 2014-12-30 2018-12-11 Akamai Technologies, Inc. Self-adjusting tiered caching system to optimize traffic performance and origin offload
US20160226998A1 (en) 2015-01-30 2016-08-04 Akamai Technologies, Inc. Using resource timing data for server push
US10313463B2 (en) 2015-02-19 2019-06-04 Akamai Technologies, Inc. Systems and methods for avoiding server push of objects already cached at a client
US10097449B2 (en) * 2015-02-20 2018-10-09 Cisco Technology, Inc. Optimized border gateway protocol best path selection for optimal route reflection
US10015073B2 (en) 2015-02-20 2018-07-03 Cisco Technology, Inc. Automatic optimal route reflector root address assignment to route reflector clients and fast failover in a network environment
US9800653B2 (en) * 2015-03-06 2017-10-24 Microsoft Technology Licensing, Llc Measuring responsiveness of a load balancing system
CN104811354B (en) * 2015-05-08 2018-04-06 北京京东尚科信息技术有限公司 A kind of methods, devices and systems for CDN health detections
US9929949B2 (en) 2015-06-29 2018-03-27 Google Llc Systems and methods for inferring network topology and path metrics in wide area networks
CN106453673A (en) * 2016-08-31 2017-02-22 腾讯科技(深圳)有限公司 Data transmission method and related equipment
US9667619B1 (en) 2016-10-14 2017-05-30 Akamai Technologies, Inc. Systems and methods for utilizing client side authentication to select services available at a given port number
US10084855B2 (en) 2017-01-23 2018-09-25 Akamai Technologies, Inc. Pixel-based load balancing
US9986269B1 (en) 2017-03-03 2018-05-29 Akamai Technologies, Inc. Maintaining stream continuity counter in a stateless multiplexing system
WO2018165190A1 (en) 2017-03-07 2018-09-13 Akamai Technologies, Inc. Cooperative multipath
US10218697B2 (en) 2017-06-09 2019-02-26 Lookout, Inc. Use of device risk evaluation to manage access to services
US10439925B2 (en) 2017-12-21 2019-10-08 Akamai Technologies, Inc. Sandbox environment for testing integration between a content provider origin and a content delivery network
US10440142B2 (en) 2018-03-06 2019-10-08 Akamai Technologies, Inc. Automated TTL adjustment using cache performance and purge data
US10452563B1 (en) 2018-05-07 2019-10-22 Akamai Technologies, Inc. Cache eviction scheme for acceptable substitutes in online media

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5734719A (en) * 1993-10-15 1998-03-31 International Business Systems, Incorporated Digital information accessing, delivery and production system
US5774668A (en) * 1995-06-07 1998-06-30 Microsoft Corporation System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing
US6275470B1 (en) * 1999-06-18 2001-08-14 Digital Island, Inc. On-demand overlay routing for computer-based communication networks
US20020055983A1 (en) * 2000-11-03 2002-05-09 The Board Of Regents Of The University Of Nebraska Computer server having non-client-specific persistent connections
US20020069278A1 (en) * 2000-12-05 2002-06-06 Forsloew Jan Network-based mobile workgroup system
US20020107940A1 (en) * 2000-12-12 2002-08-08 Brassil John T. Media program timing and identity delivery method and system
US20030101279A1 (en) * 2001-11-29 2003-05-29 Rajiv Maheshwari Method for transferring messages along optimally redundant network paths in a distributed communication network
US20030135509A1 (en) * 2002-01-11 2003-07-17 Davis Andrew Thomas Edge server java application framework having application server instance resource monitoring and management
US20030188021A1 (en) * 2001-12-19 2003-10-02 International Business Machines Corporation Method and system for processing multiple fragment requests in a single message
US20040093419A1 (en) * 2002-10-23 2004-05-13 Weihl William E. Method and system for secure content delivery
US20040267945A1 (en) * 2001-08-02 2004-12-30 Christophe Reiss Filtering redundant packets in computer network equipments
US7162529B2 (en) * 2002-05-30 2007-01-09 Hitachi, Ltd. System using mobile proxy for intercepting mobile IP message and performing protocol translation to support multiple communication protocols between mobile networks
US7328237B1 (en) * 2002-07-25 2008-02-05 Cisco Technology, Inc. Technique for improving load balancing of traffic in a data network using source-side related information
US8631456B2 (en) * 2002-05-10 2014-01-14 Convergent Media Solutions, LLC Method and apparatus for browsing using alternative linkbases
US8891522B2 (en) * 2005-12-30 2014-11-18 Akamai Technologies, Inc. Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001237876A (en) * 2000-02-21 2001-08-31 Nec Corp Buildup method for ip virtual private network and the ip virtual private network
EP1168718B1 (en) * 2000-06-30 2005-03-23 Alcatel Method and device to communicate with a device not belonging to the same virtual private network
JP4225681B2 (en) * 2000-12-06 2009-02-18 富士通株式会社 Virtual closed network construction method and apparatus, and relay apparatus
US20030032409A1 (en) * 2001-03-16 2003-02-13 Hutcheson Stewart Douglas Method and system for distributing content over a wireless communications system
US20050071439A1 (en) * 2003-09-29 2005-03-31 Peter Bookman Mobility device platform
US20050259689A1 (en) * 2004-04-01 2005-11-24 Azer Bestavros Providing soft bandwidth guarantees using elastic TCP-based tunnels
JP4655619B2 (en) * 2004-12-15 2011-03-23 日本電気株式会社 Radio base station apparatus and rate control method thereof

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5734719A (en) * 1993-10-15 1998-03-31 International Business Systems, Incorporated Digital information accessing, delivery and production system
US5774668A (en) * 1995-06-07 1998-06-30 Microsoft Corporation System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing
US6275470B1 (en) * 1999-06-18 2001-08-14 Digital Island, Inc. On-demand overlay routing for computer-based communication networks
US20020055983A1 (en) * 2000-11-03 2002-05-09 The Board Of Regents Of The University Of Nebraska Computer server having non-client-specific persistent connections
US20020069278A1 (en) * 2000-12-05 2002-06-06 Forsloew Jan Network-based mobile workgroup system
US20020107940A1 (en) * 2000-12-12 2002-08-08 Brassil John T. Media program timing and identity delivery method and system
US20040267945A1 (en) * 2001-08-02 2004-12-30 Christophe Reiss Filtering redundant packets in computer network equipments
US20030101279A1 (en) * 2001-11-29 2003-05-29 Rajiv Maheshwari Method for transferring messages along optimally redundant network paths in a distributed communication network
US20030188021A1 (en) * 2001-12-19 2003-10-02 International Business Machines Corporation Method and system for processing multiple fragment requests in a single message
US20030135509A1 (en) * 2002-01-11 2003-07-17 Davis Andrew Thomas Edge server java application framework having application server instance resource monitoring and management
US8631456B2 (en) * 2002-05-10 2014-01-14 Convergent Media Solutions, LLC Method and apparatus for browsing using alternative linkbases
US7162529B2 (en) * 2002-05-30 2007-01-09 Hitachi, Ltd. System using mobile proxy for intercepting mobile IP message and performing protocol translation to support multiple communication protocols between mobile networks
US7328237B1 (en) * 2002-07-25 2008-02-05 Cisco Technology, Inc. Technique for improving load balancing of traffic in a data network using source-side related information
US20040093419A1 (en) * 2002-10-23 2004-05-13 Weihl William E. Method and system for secure content delivery
US8891522B2 (en) * 2005-12-30 2014-11-18 Akamai Technologies, Inc. Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9426690B2 (en) 2010-12-07 2016-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Method for enabling traffic acceleration in a mobile telecommunication network
US10349305B2 (en) 2010-12-07 2019-07-09 Telefonaktiebolaget Lm Ericsson (Publ) Method for enabling traffic acceleration in a mobile telecommunication network
US10348589B2 (en) * 2015-06-15 2019-07-09 Netflix, Inc. Managing networks and machines that deliver digital content
WO2018148302A1 (en) * 2017-02-07 2018-08-16 Level 3 Communications, Llc System and method for next hop bgp routing in a network
US10277499B2 (en) 2017-02-07 2019-04-30 Level 3 Communications, Llc System and method for next hop BGP routing in a network
CN108667658A (en) * 2018-04-28 2018-10-16 厦门白山耘科技有限公司 A kind of bandwidth reuse method and device

Also Published As

Publication number Publication date
US7660296B2 (en) 2010-02-09
CN101410819A (en) 2009-04-15
WO2007089352A3 (en) 2008-08-14
WO2007089352A2 (en) 2007-08-09
US8891522B2 (en) 2014-11-18
EP1972112A4 (en) 2010-12-01
US8170017B2 (en) 2012-05-01
US20070153782A1 (en) 2007-07-05
CN101410819B (en) 2012-05-30
EP1972112A2 (en) 2008-09-24
US20100138488A1 (en) 2010-06-03
JP2009522877A (en) 2009-06-11
JP4975760B2 (en) 2012-07-11
US20120215938A1 (en) 2012-08-23

Similar Documents

Publication Publication Date Title
Joseph et al. A policy-aware switching layer for data centers
EP2241058B1 (en) Method for configuring acls on network device based on flow information
JP4122232B2 (en) System and method for guaranteeing network service level for intelligent distribution
US10135689B2 (en) Position parameterized recursive network architecture with topological addressing
US8676980B2 (en) Distributed load balancer in a virtual machine environment
US8023421B2 (en) Method and apparatus for the assessment and optimization of network traffic
US8942106B2 (en) Method and apparatus for route optimization enforcement and verification
US9270598B1 (en) Congestion control using congestion prefix information in a named data networking environment
US9432385B2 (en) System and method for denial of service attack mitigation using cloud services
US6269099B1 (en) Protocol and method for peer network device discovery
EP1333642B1 (en) Method and system for integrating performance enhancing functions in a virtual private network (VPN)
Handley Why the Internet only just works
US8423630B2 (en) Responding to quality of service events in a multi-layered communication system
US9647954B2 (en) Method and system for optimizing a network by independently scaling control segments and data flow
CN102316091B (en) Application-layer traffic optimization enhancements for content delivery networks
US9306907B1 (en) Load balancing among a cluster of firewall security devices
US8396954B2 (en) Routing and service performance management in an application acceleration environment
US7336613B2 (en) Method and apparatus for the assessment and optimization of network traffic
US7478427B2 (en) Method and apparatus for providing adaptive VPN to enable different security levels in virtual private networks (VPNs)
EP1350363B1 (en) Method and apparatus for the assessment and optimization of network traffic
US9692713B2 (en) System, apparatus and method for providing a virtual network edge and overlay
US7047315B1 (en) Method providing server affinity and client stickiness in a server load balancing device without TCP termination and without keeping flow states
US6857009B1 (en) System and method for network access without reconfiguration
US9184983B2 (en) Cross-stratum optimization protocol
US7719966B2 (en) Network element architecture for deep packet inspection

Legal Events

Date Code Title Description
AS Assignment

Owner name: AKAMAI TECHNOLOGIES, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FLETCHER, GREGORY;CHAMPAGNE, ANDREW F.;STODOLSKY, DANIEL;AND OTHERS;SIGNING DATES FROM 20060608 TO 20060612;REEL/FRAME:035190/0536

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION