WO2021205212A1

WO2021205212A1 - Traffic controller for cloud native deployment

Info

Publication number: WO2021205212A1
Application number: PCT/IB2020/053369
Authority: WO
Inventors: Lakhdar Bourokba; Jeremy Brown; Marcelo NAHUM; Kishan PATEL; Martin Roy
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2020-04-08
Filing date: 2020-04-08
Publication date: 2021-10-14

Abstract

Embodiments include a traffic controller for a container-based cluster of one or more computing machines. The traffic controller includes a data plane component comprising a proxy configured to route non-HTTP upstream traffic to a plurality of network endpoints within the cluster; and a control plane component coupled to the data plane component and arranged to dynamically configure processing of the non-HTTP upstream traffic by the data plane component and routing of the non-HTTP upstream traffic by the proxy. In some embodiments, the traffic controller can also be configured to process/route non-HTTP downstream traffic, HTTP upstream traffic, and/or HTTP downstream traffic. The control plane can include an API server configured to create one or more traffic objects representative of the data plane component, and a traffic configuration controller configured to determine a configuration for the data plane component based on the traffic objects. Embodiments also include corresponding methods.

Description

TRAFFIC CONTROLLER FOR CLOUD NATIVE DEPLOYMENT

TECHNICAL FIELD

The present application relates generally to the field of communication networks and more specifically to techniques that facilitate cloud-native deployment of applications that use non- HTTP-based signaling (e.g., SIP, etc.) on cloud infrastructure that uses container-based technologies

BACKGROUND

Historically, mobile phones (also referred to herein as “user equipment”, “UEs”, or “wireless devices”) have provided voice services over a circuit-switched (CS) telephone network. On the other hand, in packet-based networks, data packets are routed based on protocols specified by the Internet Engineering Task Force (IETF), such as Internet Protocol (IP), User Datagram Protocol (UDP), etc. IP Multimedia Subsystem (IMS) is an architectural framework for delivering multimedia services to wireless devices based on these Internet-centric protocols. IMS was originally specified by 3rd Generation Partnership Project (3GPP) in Release 5 (ReI-5) as a technology for evolving mobile networks beyond GSM, e.g., for delivering Internet services over GPRS. IMS has evolved in subsequent releases to support other access networks and a wide range of services and applications.

Figure 1 shows a high-level block diagram of a communication network in which IMS is used to deliver applications and services to a variety of end-user devices. In particular, Figure 1 shows three types of access networks from which various types of devices can be connected, namely, a wireless access network that supports access by wireless devices, a fixed access network that supports access by wired multimedia telephony devices (e.g., computers, phones, etc.), and a cable access network that supports access by user premises devices (e.g., set-top boxes). The respective access networks are connected to an intermediate IMS network, which contains various functionality to deliver applications and services to the various devices via the respective access networks. In general, all applications and services are executed by application servers. One application server can be dedicated to a single service or can host several services. It is also possible to combine services from several application servers to create one unified user experience for the end-user.

At a high-level, the functionality of the IMS network can be sub-divided into two types: control and media delivery (below dashed line in Figure 1) and application enablers (above dashed line). The control functionality comprises the Call Session Control Function (CSCF) and the Home Subscriber Server (HSS). The CSCF is used for session control for devices and applications that are using the IMS network. Session control includes the secure routing of the session initiation protocol (SIP) messages, subsequent monitoring of SIP sessions, and communicating with a policy architecture to support media authorization. CSCF functionality can also be divided into Proxy CSCF (P-CSCF), Serving CSCF (S-CSCF), and Interrogating CSCF (I-CSCF).

CSCF also interacts with the HSS, which is the master database containing user and subscriber information to support the network entities handling calls and sessions. For example, HSS provides functions such as identification handling, access authorization, authentication, mobility management (e.g., which session control entity is serving the user), session establishment support, service provisioning support, and service authorization support.

A Media Resource Function (MRF) can provide media services in a user’ s home network and can manage and process media streams such as voice, video, text-to-speech, and real-time transcoding of multimedia data. In general, a WebRTC Gateway allows native- and browser-based devices to access services in the network securely.

An Interconnect Border Control Function (IBCF) provides boundary control between various service provider networks, providing IMS network security in terms of signaling information. When connecting non-SIP or non-IPv6 networks, the IBCF can provide bandwidth allocation and admission control. Likewise, a Breakout Gateway Control Function (BGCF) selects a network for PSTN breakout and interacts (e.g., via SIP signaling) with a media gateway that is responsible for the interworking with the PSTN. Similarly, a Service and Continuity Control Application Server (SRVCC-AS) facilitates centralizing a UE’s IMS services regardless of whether the UE is using circuit-switched or packet-switched services in a mobile network.

Security-related elements and/or functions in an IMS network can include an Authentication and Aggregation Proxy, which is an entry point for the UE into the IMS network and authenticate user requests, thereby separating the authentication procedure from application server specific logic. Likewise, a Non-SIM Device Gateway (NSD Gw) can support IMS access by UEs that do not utilize subscriber identity modules (SIM), while an Entitlement Server can facilitate download of eSIMs to companion devices (e.g., smartwatches and other similar devices) that are paired with a user's primary mobile device (e.g., UE with SIM).

Application enablers can include a Multimedia Telephony Application Server (MMTel AS). MTAS can provide and/or facilitate voice and/or multimedia telephony services via mobile/wireless or fixed access networks. Multimedia telephony applications can also be supported/provided by a Multimedia Telephony Interworking Application Server (MMTel IW). Messages can be sent and delivered from and to all devices by means of an advanced interworking gateway, i.e., IP Short Message Gateway (IP-SM-GW). In general, “cloud computing” refers to the delivery of remote computing services such as application servers, storage, databases, networking, analytics, and intelligence to users over the Internet. “Cloud” infrastructure” generally refers to the hardware and software components such as servers, storage, networking, etc., that are needed to support the computing requirements of a cloud computing model. Cloud infrastructure also typically includes an abstraction layer that virtualizes the hardware resources and logically presents them to users (e.g., as “virtual machines”) through application program interfaces (APIs). Such virtualized resources are typically hosted by a service provider are delivered to users over the public Internet or a private network. Publicly available cloud infrastructure can be referred to as “infrastructure as a service”.

Many web-based services and applications have migrated to cloud infrastructure. These services are typically based on HyperText Transport Protocol (HTTP) signaling. A common platform used to provide cloud-based web-services is Kubernetes, which can coordinate a highly available cluster of connected computers (also referred to as “processing elements” or “hosts”) to work as a single unit. Kubernetes deploys applications packaged in “containers” (e.g., via its “container runtime”) to decouple them from individual computing hosts. These Kubernetes abstractions facilitate deploying applications to a cloud-based computing cluster without tying them specifically to individual computing machines. In this manner, containerized applications are more flexible and available than in deployment modes where applications were installed directly onto specific machines. Such containerized applications are referred as “cloud-native” applications.

In general, a Kubernetes cluster consists of two types of resources: a “master” that coordinates or manages the cluster and “nodes” or “workers” that run applications. Put differently, a node is a virtual machine (VM) or physical computer that serves as a worker machine. The master coordinates all activities in a cluster, such as scheduling applications, maintaining applications' desired state, scaling applications, and rolling out new updates. Each node has a Kubelet, which is an agent for managing the node and communicating with the Kubernetes master, as well as tools for handling container operations. The Kubernetes cluster master starts the application containers and schedules the containers to run on the cluster's nodes. The nodes communicate with the master using the Kubernetes API, which the master exposes. End users can also use the Kubernetes API directly to interact with the cluster.

A “pod” is a basic execution unit of a Kubernetes application, i.e., the smallest and simplest unit that can be created and deployed in the Kubernetes object model. A pod represents processes running on a cluster and encapsulates an application’s container(s), storage resources, a unique network IP address, and options that govern how the container(s) should run. Put differently, a Kubernetes pod represents a single instance of an application, which can consist of one or more containers that are tightly coupled and that share resources. In general, a container is a lightweight, portable executable image that includes software and all of the software’s dependencies. As such, containers are relatively decoupled from underlying physical or virtual computing infrastructure.

Conventionally, IMS services (such as discussed above in relation to Figure 1) have been implemented by dedicated, purpose-specific hardware installed in an operator’s network. However, it is desirable to migrate IMS services to cloud infrastructure for at least the same reasons as for web-based services. Even so, cloud deployment platforms such as Kubernetes are optimized for HTTP-centric, web-based applications, while IMS uses SIP rather than HTTP as a fundamental signaling protocol. As such, this creates various problems, issues, and/or difficulties that must be overcome to successfully migrate IMS services to cloud infrastructure·

SUMMARY

Embodiments of the present disclosure provide specific improvements to cloud computing infrastructure by facilitating solutions to overcome the exemplary problems, issues, and/or difficulties summarized above and described in more detail below.

Exemplary embodiments include traffic controllers for a container-based cluster of one or more computing machines. These traffic controllers can include a data plane component comprising a proxy configured to route non-HTTP upstream traffic to a plurality of network endpoints within the cluster, and a control plane component coupled to the data plane component and arranged to dynamically configure processing of the non-HTTP upstream traffic by the data plane component and routing of the non-HTTP upstream traffic by the proxy. In some embodiments, the traffic controller can also be configured to process/route non-HTTP downstream traffic, HTTP upstream traffic, and/or HTTP downstream traffic. In some embodiments, each network endpoint can be associated with one or more containers (e.g., Kubernetes pod of containers) hosted by the cluster.

Exemplary embodiments also include methods (e.g., procedures) performed by a container-based cluster of one or more computing machines. Such methods can include creating one or more traffic objects representative of a data plane component of the traffic controller, wherein each traffic object is associated with one or more types of network endpoint within the cluster. Each network endpoint can be associated with one or more containers hosted by the cluster. Such methods can also include determining a configuration for the data plane component based on the one or more traffic objects. In some embodiments, the creating and determining operations can be performed by a control plane component of the traffic controller. Other embodiments include non-transitory, computer-readable media storing (or computer program products comprising) computer-executable instructions that, when executed by processing circuitry associated with such traffic controllers, configure the same to perform operations corresponding to any of the exemplary methods described herein.

These and other objects, features, and advantages of embodiments of the present disclosure will become apparent upon reading the following Detailed Description in view of the Drawings briefly described below.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 shows a high-level block diagram of a communication network in which IMS is used to deliver applications and services to a variety of end-user devices.

Figure 2 shows a high-level view of a traffic controller for a container-based cluster of one or more computing machines, according to various exemplary embodiments.

Figures 3-4 show more detailed views of traffic controllers for a container-based cluster of one or more computing machines, according to various exemplary embodiments.

Figures 5-6 show high-level views of traffic controller data planes based on an Envoy proxy, according to various exemplary embodiments of the present disclosure.

Figure 7 illustrates an exemplary architecture in which multiple traffic objects are used for traffic handlers (TFls) associated with respective multiple types of IMS services having endpoints in a cluster, according to various exemplary embodiments of the present disclosure.

Figure 8 illustrates an exemplary architecture in which a single traffic object is used for TFls associated with multiple types of services having endpoints in a cluster, according to various exemplary embodiments of the present disclosure.

Figure 9 illustrates an exemplary method (e.g., procedure) performed by a traffic controller, according to various exemplary embodiments of the present disclosure.

Figure 10 is a block diagram illustrating an exemplary container-based virtualization environment suitable for implementation of various embodiments described herein.

DETAILED DESCRIPTION

Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art.

Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used. All references to a/an/the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods and/or procedures disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein can be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments can apply to any other embodiments, and vice versa. Various objects, features and advantages of the exemplary embodiments will be apparent from the following description.

In the present disclosure, the term “service” is used generally to refer to a set of data, associated with one or more applications, that is to be transferred via a network with certain specific delivery requirements that need to be fulfilled in order to make the applications successful. In the present disclosure, the term “component” is used generally to refer to any component needed for the delivery of the service. Examples of components are cloud infrastructure with related resources such as computation and storage.

One problem in deploying IMS services on cloud infrastructure is providing users a way to access such services. In the context of Kubernetes cloud platforms, there is a need for efficient and smooth traffic ingress and egress management. There are several conventional ways to expose services implemented on a Kubernetes cluster to external traffic. For example, a NodePort service is the most primitive way to get external traffic directly to a service. NodePort, as its name implies, open a specific port on all the worker nodes of the cluster, such that any traffic sent to this port is forwarded to the service. However, NodePort is generally not recommended and/or feasible in commercial deployment due to various reasons including only one service per port, small range of ports supported by Kubernetes, and security risks associated with an open port (e.g., an open port bypasses almost all network security in Kubernetes).

A LoadBalancer service is another standard way to expose a service to outside traffic. For example, a cloud provider will deploy a network LoadBalancer that appears as a single IP address to outside traffic but forwards that traffic to the network endpoints on the various nodes that are running a particular service. However, the LoadBalancer service is only available on specific cloud infrastructure (e.g., Microsoft Azure, Google Compute Engine, Amazon Web Services, etc.) and Kubernetes does not offer native support for services of this type. In addition, each deployed service will have a different IP address, such that there is an underlying infrastructure cost per exposed service, which can get very expensive if there are many services to expose.

As another example, an Ingress is a Kubernetes object that acts as a reverse proxy, providing a single entry point to a Kubernetes cluster (e.g., of VMs) and routing requests to endpoints associated with various services hosted on the cluster. In general, a reverse proxy provides a “public face” (e.g. IP address) for a service to outside traffic and hides the particular architecture used to deploy the service from the outside traffic. A Kubernetes Ingress object can also include load balancer functionality that distributes outside traffic (e.g., requests) for a particular service to various endpoints in the cluster that are associated with the particular service.

Ingress and egress are important components of a cloud-native system because they allow for a clean separation between an application and how it is accessed. In general, Kubernetes Ingress objects expose routes from outside the cluster to services within the cluster. Traffic routing can be controlled by rules defined by the Ingress object associated with the particular type of traffic. However, the native Kubernetes Ingress resource assumes that all ingress traffic is HTTP-based and does not support SIP or any other IMS-related protocols. This inhibits and/or prevents IMS deployment on cloud infrastructure that uses Kubernetes or any other container-based technologies.

Exemplary embodiments of the present disclosure address these and other problems and/or difficulties by providing a traffic controller (also referred to as “application delivery controller” or ADC) for a container-based cluster of one or more computing machines. Embodiments of the traffic controller can be used to support IMS deployment on cloud infrastructure that uses Kubernetes or any other container-based technologies.

The traffic controller can include a data plane component comprising a proxy configured to route non-HTTP upstream (or ingress) traffic to a plurality of network endpoints within the cluster. In various embodiments, the proxy can also be configured to route non-HTTP downstream (or egress) traffic from the plurality of network endpoints, to route HTTP upstream traffic to the plurality of network endpoints, and/or to route HTTP downstream traffic from the plurality of network endpoints.

The traffic controller can also include a control plane component coupled to the data plane component and arranged to dynamically configure processing of the non-HTTP upstream traffic by the data plane component and routing of the non-HTTP upstream traffic by the proxy. In various embodiments, the control plane component can also be arranged to dynamically configure processing of non-HTTP downstream traffic, HTTP upstream traffic, and/or HTTP downstream traffic by the data plane component, and routing of the non-HTTP downstream traffic, the HTTP upstream traffic, and/or the HTTP downstream traffic by the proxy.

For example, the control plane can be packaged into a Kubernetes operator that introduces new custom resources to extend the Kubernetes API. For example, the control plane can be packaged as a “Traffic operator” that includes custom resource description (CRD), a traffic controller (e.g., according to various embodiments) that can include a data plane component with a plurality of configurable traffic filters (e.g., Envoy filters).

Such embodiments can extend and chain together various technologies offered by the cloud-native ecosystems to produce a solution that addresses a gap between conventional HTTP- based traffic controllers and IMS-specific protocols. Although SIP is the most commonly used protocol in IMS, these solutions can also be employed to support other IMS-related protocols such as Diameter, Websocket, Remote Procedure Calls (RPC) over HTTP/2, etc. as well as other protocols that may be used in relation to services and applications hosted by container-based clusters. Examples of such protocols include CoAP and MQTT that can be used in Internet-of- Things (IoT) applications. Furthermore, certain protocols used for IMS (e.g., Diameter) can also be used more generally outside of IMS, and embodiments of the disclosed traffic controller can also be adapted to this more general use.

In general, the protocols discussed above are layer 7 (L7, or application layer) signalling protocols. However, embodiments are also applicable to other IMS protocols related to media delivery, such as RTP, MSRP, and TCAP. These media-related protocols are carried over various L4 transport protocols, such as TCP, UDP, and SCTP. Traffic filters for these media-related protocols can also be included in various embodiments of the traffic controller of the present disclosure.

Figure 2 shows a high-level view of a traffic controller (200) for a container-based cluster of one or more computing machines, according to some embodiments. As shown, traffic controller 200 includes a data plane 210 and a control plane 220, as separated logically by the dashed line. The control plane includes an API gateway (or server) that can be accessed by an external management client (e.g., by an operator). The data plane receives upstream traffic that includes requests for services hosted by the cluster 230, which can be a cluster of virtual machines (“VM cluster”) or a cluster of physical machines (e.g., processing elements). In Figure 2, cluster 230 includes a variety of network endpoints, each associated with one of services A-E. When the data plane receives a request for service A, for example, the load balancer routes the request to one of the endpoints associated with service A (indicated by the dashed ellipse) according to a routing and/or load balancing policy configured by the control plane.

Although Figure 2 shows the traffic controller handling (e.g., processing and routing) upstream traffic to the cluster, it should be understood that in some embodiments, traffic controller 200 can also be configured to handle (e.g., process and route) downstream traffic flowing from the cluster. As discussed in more detail below, the upstream and (optionally) downstream traffic can be non-HTTP traffic and, in some embodiments, HTTP traffic. The control plane is also responsible for discovering and/or determining the availability of services hosted by the cluster, as well as the addresses of the network endpoints associated with the containers in the cluster. For example, each network endpoint can be associated with one or more containers in the cluster. This information can be used to configure the data plane. In addition to routing and/or load balancing policies, the control plane can also configure any security features required in the data plane, such as certificates, encryption keys, etc.

The cluster 230 can also include one or more traffic handlers, each associated with a particular service or function hosted by the cluster. The traffic controller utilizes a back-end protocol between the main traffic controller functionality and the respective traffic handlers (THs).

In some embodiments, the traffic controller can use SIP for the communication with the traffic handlers. The SIP requests in the upstream traffic will be terminated in the traffic controller, which will then use SIP over TLS to send these requests to the correct TH (and receive SIP responses from the TH). In such case, the traffic controller can be considered a “SIP traffic controller” and can act as a SIP stateless proxy. However, certain SIP requests originating in the THs can be sent directly from the THs to the destination network endpoints without going through the same traffic controller functionality as the external upstream traffic.

In other embodiments, the traffic controller can use RPC over HTTP/2 as the transport for SIP protocol data units (PDUs) from the upstream traffic. In this case the SIP transport will be handled by the main traffic controller while the SIP Transactions and Sessions are handled by the THs. Both ingress and egress traffic are managed by the traffic controller in this case.

In some embodiments, the traffic controller can comprise an Envoy SIP filter in the data plane, with the control plane packaged as a Kubernetes operator to allow easy deployment by using a high-level declarative application programming interface (API). In general, Envoy is an application-layer (L7) proxy and communication bus designed for large service-oriented architectures. Envoy also includes a layer-3/layer-4 (L3/L4, e.g., IP and TCP) network proxy and filter chain. Envoy also supports advanced load balancing for upstream traffic into various endpoints available in a cluster.

Envoy can also provide various other advantages in addition to load balancing. For example, Envoy was designed from the ground up for microservices, including features such as hitless reloads (“hot restarts”), observability, resilience, and a small (or lightweight) implementation footprint. Envoy also supports dynamic configuration, including configuration via gRPC APIs. gRPC (also referred to more generally as “RPC”) is an open-source remote procedure call (RPC) system that uses HTTP/2 for transport and Protocol Buffers (“Protohufs”) as an interface description language (IDL). In general, Protohufs facilitate serializing structured data, e.g., for programs to communicate with each other over a wire or for storing data. Conventionally, Envoy primarily supports HTTP/2 traffic as a L7 reverse proxy, e.g., for Web-based applications. However, some exemplary embodiments can include an Envoy proxy that supports multiprotocol traffic for non-Web-based applications. Such multiprotocol traffic can include SIP, gRPC (or RPC), HTTP/1, and/or Diameter (an application-layer protocol for authentication, authorization, and accounting). In such embodiments, a Kubernetes operator can be used to represent the control plane to support dynamic configuration of the Envoy proxy, e.g., to manage service discovery, load balancing, routing rules, security configuration, etc.

As mentioned above, embodiments are also applicable to other IMS protocols related to media delivery, such as Real Time protocol (RTP), Message Session Relay Protocol (MSRP), and Transaction Capabilities Application Part (TCAP). These media-related protocols are carried over various L4 transport protocols, such as TCP, UDP, and stream control transmission protocol (SCTP). Traffic filters for these media-related protocols can also be included in various embodiments of the traffic controller of the present disclosure. For example, the exemplary Envoy application-layer proxy discussed above can be modified to accommodate these media-related protocols and their underlying L3/L4 protocols.

Figure 3 shows a more detailed view of a traffic controller (300) for a container-based cluster of one or more computing machines, according to some embodiments. As shown, traffic controller 300 includes a data plane 310 and a control plane 320. The control plane includes an API server (321) that can be accessed by an external management client (e.g., by an operator). The control plane also includes a traffic configuration controller (322), a traffic commit controller (323), and a discovery controller (324). These will be discussed in more detail below.

The data plane receives upstream traffic that includes requests for services hosted by cluster 330, which can be a VM cluster or a cluster of physical machines (e.g., processing elements). Similar to Figure 2, the cluster includes a variety of network endpoints, each associated with one of services A-E. The data plane includes a proxy/load balancer 311, which also may be referred to herein more simply as “proxy” or “load balancer”. The load balancer can be configured to distribute the upstream traffic among the network endpoints according to a load-balancing policy configured by the control plane 320. The data plane can also include an API 313, which can be specific to the particular type of proxy 311.

The data plane also includes a filter chain 312 (also referred to as a “pipeline” or “pipeline arrangement”), which can include a plurality of traffic filters that are configurable to process upstream traffic toward the network endpoints and to process downstream traffic from the network endpoints. For example, each traffic filter in a pipeline can be configured to perform one or more of the following operations on upstream (and, optionally, downstream) traffic: parsing, compression, decompression, forwarding or routing, encryption, decryption, protocol encapsulation, protocol encoding, and protocol decoding. For example, some requests or types of upstream traffic can be simply forwarded to the appropriate traffic handler, while other requests can be modified, audited, deciphered, or uncompressed before being further processed and/or routed.

In some embodiments, filter chain 312 can be part of proxy/load balancer 311, as illustrated in Figure 3. For example, reverse proxy 311 can be an Envoy proxy and filter chain 312 can include a network-layer (e.g., L3/L4) and/or application-layer (e.g., L7) pluggable filter chain of the Envoy proxy, discussed above. Embodiments of filter chain 312 can include filters for various application-layer (L7) and supporting network-layer (L3/L4) protocols associated with each particular embodiment, including media-related and other protocols discussed briefly above that are not conventionally associated with Envoy proxies.

The control plane of the traffic controller provides a level of abstraction based on “traffic objects” that hold rules, configurations, policies, etc. used to configure the data plane component. These traffic objects can be related to security, load balancing, rate limiting, circuit breaking, blacklisting, white listing, authorization, monitoring, lawful intercept, etc. More generally, they can define how external traffic can be processed by the data plane (e.g., filters 312) and how the processed traffic can be routed to (and within) the endpoints in the cluster. This abstraction is important because it allows separation between the application and how it is accessed.

When a user defines a new Traffic entity (e.g., for a particular service or type of traffic), the API server creates a traffic object representing this entity. Any subsequent change made to this entity automatically updates the traffic object. As an example, the API server 321 can be a Kubernetes API server. More generally, the API server is configured to create various traffic objects representative of the data plane component, with each traffic object being associated with one or more types of network endpoints. Traffic configuration controller 322 can be coupled to the API server and can be configured to determine a configuration for the data plane component (referred to as a “master configuration”) based on the traffic objects created by the API server.

In Kubernetes terminology, a “resource” is an endpoint in the Kubernetes API that stores Kubernetes objects of a certain “kind”. A “custom resource” is essentially a “resource” that can be added to extend the basic Kubernetes API. The Kubernetes API can be extended in this manner via a mechanism called CRD, which can provide fine-grained access and control over what’s inside the cluster. Defining a CRD object creates a new custom resource with a user-specified name and schema. The Kubernetes API serves and handles the storage of this new custom resource. Once a custom resource is installed, users can manage objects of this kind in a similar way as native Kubernetes resources (e.g., “pods”). For example, a user can create a CRD for a traffic object that specifies, at a high level, how the data plane (e.g., Envoy load balancer) should handle a particular type of traffic (e.g., SIP requests). The user can provide an instance of the schema for the CRD (also referred to as a “deployment manifest”) to the API server, which routes the deployment manifest to the traffic configuration controller for parsing and converting to configurations and/or commands that the data plane (e.g., Envoy load balancer) understands.

As shown in Figure 3, there can be a variety of traffic objects, each with different properties. Subsequently, the traffic configuration controller continuously watches for changes to the various Traffic objects. Whenever a change (e.g., creation, deletion, update, etc.) is detected, the traffic configuration controller determines a master configuration for the data plane based on all of the traffic objects. This can include, for each particular traffic object, validating a configuration associated with the particular traffic object against a schema associated with the particular traffic object and verifying that there are no conflicts within the configuration (e.g., rules, policies, endpoints, certificates, keys, etc.) associated with the particular traffic object. This determination can also include verifying that there are no conflicts between configurations associated with different ones of the Ingress objects.

The traffic configuration controller can also run watch loops for the various objects to detect changes and update the master configuration accordingly. The traffic configuration controller can determine a current state for the data plane component and then determine a difference between the current state and the traffic objects (e.g., provided by API server 321) that represent a desired state of the data plane component. The traffic configuration controller can then determine the master configuration for the data plane component based on the difference.

In some embodiments, the traffic objects can be arranged as a plurality of traffic objects, each associated with a different type of network endpoint (e.g., for a different service). In other embodiments, a single traffic object can be associated with a plurality of different types of network endpoints (e.g., for different services). For example, in a cloud-native IMS implementation, the different types of network endpoints can be associated with services related to any of the following: P-CSCF, I-CSCF, S-CSCF, MMTel AS, MMTel IW, SRVCC-AS, IP-SM-GW, BGCF, IBCF, MRF, Authentication and Aggregation Proxy, WebRTC Gateway, NSD Gw, and Entitlement Server.

The master configuration for the data plane component can include rules, policies, etc. related to any of the following: network interface, transport layer, layer-7 (L7) protocol, listening port, connection pooling, logging, tracing, performance counters, transport layer security (TLS), certificate management, circuit breaking, blacklisting, white listing, authorization, health check, retries, timeout, outlier detection, and rate limiting. Furthermore, the master configuration can include various rules, policies, etc. that are specifically related to routing of the traffic (e.g., upstream, downstream, non-HTTP, and/or HTTP) by the proxy. These can include and/or be related to any of the following: internal layer-7 (L7) protocol encapsulation, non-HTTP header manipulation, custom consistent hash key for non- HTTP traffic, charging data, data analytics, ingress load balancing, service endpoints, distribution policy, traffic mirroring, and number of replicas of data plane instances.

Once determined by traffic configuration controller 322, the master configuration is pushed to the data plane by traffic commit controller 323 using an API specific to the data plane (e.g., Envoy API, NGINX API, etc.). In some embodiments, the data plane (including the proxy) can have “live/hot restart” capabilities such that it is able to reload itself with the new configuration without dropping any existing connections.

In some embodiments, more abstraction may be used to better represent routing rales. For example, a “rule object” can be created and associated with one or more traffic objects. Each rule object can define combinations of rales and policies for routing traffic. This abstraction can facilitate complex routing scenarios by chaining simple traffic rules together. For example, complex traffic routes can be represented by a tree of simple rules. Different traffic objects could potentially share commonly used sub-trees, branches, leaves (e.g., common security policies, default error handling, etc.).

As mentioned above, a Kuhernetes operator includes a CRD and associated controller. In some embodiments, an traffic operator can be defined to include the following aspects of the traffic controller: traffic definition (e.g., CRD, schema, API extension), traffic configuration controller (business logic used to create a master configuration from an traffic object), traffic commit controller (using API specific to data plane), and data plane filters. A traffic operator can improve lifecycle management from an application perspective. Whenever a new application protocol (or protocol combination like SIP-RPC) is introduced, a corresponding new traffic operator can be defined based on CRD, controller, and filters.

Another important task to be handled by the control plane is service discovery. In the arrangement shown in Figure 3, this task is performed by discovery controller 324, which determines backend endpoints are available in the cluster and the address of the available endpoints. Kuhernetes exposes the topology of the cluster to the services using Kuhernetes client APIs. This is illustrated in Figure 3 by the service registry containing the real-time list of network endpoints. This can be used by the traffic controller (e.g., the discovery controller component) to discover the traffic handler pods. The traffic configuration controller can discover updates to network endpoints determined by the discovery controller based on the running watch loops and can update the master configuration accordingly (discussed above). Figure 4 shows a detailed view of a traffic controller (400) for a container-based cluster of one or more computing machines, according to some embodiments. As shown, traffic controller 400 includes a data plane component (or “data plane”) 410 and a control plane component (or “control plane”) 420. The control plane includes an API server (421) that can be accessed by an external management client (e.g., by an operator). The control plane also includes a traffic configuration controller (422), a traffic commit controller (423), and a discovery controller (424). The data plane 410 receives upstream traffic that includes requests for services hosted by cluster 430. The upstream traffic is processed by filter chain 412 and routed by proxy/load balancer 411 the corresponding network endpoints in the cluster. Optionally, downstream traffic from the cluster can be processed and routed in a similar and/or complementary manner. The traffic commit controller can configure the data plane (including traffic filters in filter chain 412) via data plane API 413. Each of these components can comprise similar functionality as discussed above for corresponding components in Figure 3.

Figure 4 also illustrates the roles of kubectl command tool 450 and ADC traffic schema 440 in the external configuration of the traffic controller. For example, this configuration can be done through a Kubernetes API 425 of the control plane.

Figure 5 shows a high-level view of a traffic controller data plane based on an Envoy proxy, according to various exemplary embodiments of the present disclosure. As shown in Figure 5, beginning from the workers, each request in the upstream traffic traverses a chain of filters including listener filters, connection filter chains, and finally on to the routers and backend services. In contrast, each response in the downstream traffic traverses the filter chain in the opposite direction, e.g., from backend services through the connection filter chain, listener filters, and finally to the downstream clients. These filters are powerful and configurable. For example, they can be configured to modify data at L4, insert headers, check and guide traffic, call out to an auth service, or transcode between protocols. Custom filters can be added via the data plane API (e.g., API 413 in Figure 4), and the traffic controller can include a plug-in architecture for observability outputs. In addition, the traffic controller can have multiple extension points including L3/L4/L7 filters, access loggers, tracers, health checkers, transport sockets, retry policies, resource monitors, statistics, etc.

In particular, Figure 5 shows an example where the filter chain implements a SIP filter for IMS services. For example, this arrangement can correspond to embodiments in which the traffic controller receives upstream (and, optionally, sends downstream) traffic having SIP as the external protocol, and also uses SIP as an internal protocol for communication with the THs. For example, the SIP requests in the upstream traffic will be terminated in the traffic controller, which will then use SIP over TLS to send these requests to the correct Til (and receive SIP responses from the TΉ). This is illustrated in Figure 5 as a chain of SIP client filters SIP codecs and SIP TH filters. For example, the SIP codec can include one or more first decoder filters configured to decode requests in the upstream traffic, and one or more first encoder filters configured to encode responses in the downstream traffic.

Figure 6 shows a high-level view of another traffic controller data plane based on an Envoy proxy, according to some embodiments. In particular, Figure 6 shows an example where the filter chain implements a SIP-to-RPC filter for IMS services. For example, this arrangement can correspond to embodiments in which the traffic controller uses RPC over HTTP/2 as an internal protocol for transport of SIP protocol data units (PDUs) in the upstream (and, optionally, downstream) traffic. This is illustrated in Figure 6 as the chain of client SIP filter, SIP codec, Protobuf codec, and RPC TH filter. Since SIP is the same external protocol used in Figures 5-6, the SIP codec can have substantially similar functionality as the SIP codec shown in Figure 5. Likewise, the Protobuf codec can include one or more second encoder filters configured to encode the decoded SIP requests in the upstream traffic according to RPC, and one or more second decoder filters configured to decode RPC responses in the downstream traffic for input to the first (SIP) encoder filters.

The SIP and RPC protocols are shown in Figure 6 only for illustration purposes. An arrangement similar to Figure 6 can be employed for various external and internal types of protocols, including any of SIP, RPC over HTTP/2, HTTP/2, HTTP/1, Websocket, and Diameter.

Various embodiments can support different types of deployments. For example, there can be many traffic objects associated with a single data plane, or multiple data planes with each associated with a different traffic object. In either case, a single traffic configuration controller is employed in the control plane. In general, this traffic configuration controller is stateless, although embodiments employing stateful controllers are not precluded.

In an IMS cloud-native architecture, a traffic controller distributes incoming traffic to the different THs based on protocol and distribution policies. Each TH type may have its own distribution policies and back-end protocol, such as SIP, RPC, HTTP/] , and Diameter. Depending on the implementation it is possible that an ADC instance per traffic handler type will be required or a single ADC with multiple ingress routes, one for each traffic handler.

Figure 7 illustrates an exemplary architecture in which multiple traffic objects are used for traffic handlers (THs) associated with respective multiple types of IMS services having endpoints in a cluster. In this example, the different types of services correspond to two different P-CSCFsl (P-CSCF1 and P-CSCF2), one IM-AS, and one S-CSCF. These services are associated with respective traffic objects that are commonly configured (e.g., by the traffic configuration controller). Figure 8 illustrates an exemplary architecture in which a single traffic object is used for traffic handlers (THs) associated with multiple types of services having endpoints in a cluster. In particular, THs for P-CSCF1, P-CSCF2, IM-AS1, and S-CSCF1 are associated with a single IMS traffic object, which is configured collectively based on information relating to the different THs. This option may require the introduction of a new CRD (IMS-route for instance) to extend the functionality of the traffic controller API to allow for richer routing configuration.

For THs, an expectation is that sessions are externalized into the data layer, such that very little information about traffic session state is kept on a specific pod in the Kubernetes cluster. Therefore, any requirements of session stickiness to a certain pod is expected to have a short lifespan. However, there will be corner cases where requests don't end-up in the pod they are supposed to. This is because any method used to determine the destination of a request will suffer from inconsistencies among different traffic controller instances handling the requests, due to fact that the data used by the distribution method is eventually consistent among the traffic controller instances. It is also assumed that, from performance perspective, some pods are better suited to serve a specific request, e.g., if they have cached information about a specific session. A consistent hash algorithm, based on protocol headers (e.g., SIP call-id, from-tag, etc.) can be used to ensure this type of distribution with limited impact due to cluster changes (e.g., pod re-start, scaling, upgrade, etc.).

Load distribution algorithms used by a traffic controller should also consider the load of each traffic handler instance. The traffic handler is responsible for calculating and reporting its load. The load calculation is traffic-handler-specific but should consider any share resource such as CPU, memory, threads, buffers, bound queues, etc. Another aspect of the distribution is canary testing. When a new version of a traffic handler is deployed ("the canary"), a portion of the traffic could be redirected to it for testing purposes.

The embodiments described above can be further illustrated by the exemplary method (e.g., procedure) shown in Figure 9. For example, the method shown in Figure 9 can be performed by a traffic controller for a container-based cluster of computing machines, such as described herein with reference to other figures. Although the method is illustrated in Figure 9 by specific blocks in a particular order, the operations corresponding to the blocks can be performed in different orders than shown and can be combined and/or divided into blocks and/or operations having different functionality than shown. Optional blocks and/or operations are indicated by dashed lines.

The method can include the operations of block 910, in which the traffic controller can create one or more traffic objects representative of a data plane component of the traffic controller. Each traffic object can be associated with one or more types of network endpoints within the cluster. In some embodiments, each network endpoint can be associated with one or more containers hosted by the cluster. The method can also include the operations of block 930, in which the traffic controller can determine a configuration for the data plane component based on the one or more traffic objects.

In some embodiments, the method can also include the operations of block 920, wherein the traffic controller can determine availability and addresses of a plurality of network endpoints within the cluster. In such embodiments, determining the configuration for the data plane component (e.g., in block 930) can be based on traffic objects associated with network endpoints determined to be available (e.g., in block 920). In some embodiments, the operations of blocks 910 and 930 (and 920, when performed) can be performed by a control plane component of the traffic controller, such as illustrated in other figures herein.

In some embodiments, determining the configuration for the data plane component in block 930 can include the operations of sub-blocks 931-933. In sub-block 931, the traffic controller can, for each particular traffic object, validate a configuration associated with the particular traffic object against a schema associated with the particular traffic object. In sub-block 932, the traffic controller can, for each particular traffic object, verify that there are no conflicts within the configuration associated with the particular traffic object. In sub-block 933, the traffic controller can verify that there are no conflicts between configurations associated with different ones of the traffic objects.

In some embodiments, determining the configuration for the data plane component in block 930 can include the operations of sub-blocks 934-936. In sub-block 934, the traffic controller can determine a current state for the data plane component. In sub-block 935, the traffic controller can determine a difference between the current state and the traffic objects representative of a desired state of the data plane component. In sub-block 936, the control plane component can determine the configuration for the data plane component based on the difference.

The method can also include the operations of block 940, in which the traffic controller can, based on the configuration, process non-HTTP upstream traffic and routing the non-HTTP upstream traffic to a plurality of network endpoints within the cluster. In some embodiments, the routing operations in block 940 can include the operations of sub-block 941, where the traffic controller can distribute the processed non-HTTP upstream traffic among the network endpoints according to a load-balancing policy configured by the control plane component.

In some embodiments, the method can also include the operations of block 950, where the traffic controller can perform various other operations based on the configuration. These operations can include any of the following:

• routing non-HTTP downstream traffic from the plurality of network endpoints; • processing the non-HTTP downstream traffic;

• processing HTTP upstream traffic;

• routing the HTTP upstream traffic to one or more network endpoints within the cluster;

• routing HTTP downstream traffic from the one or more network endpoints; and

• processing the HTTP downstream traffic.

In various embodiments, any of the non-HTTP upstream, non-HTTP downstream, HTTP upstream, and non-HTTP upstream traffic can be processed (e.g., in block 940 and, optionally, block 950) by a plurality of configurable traffic filters that are configured based on the configuration determined for the data plane component in block 930.

In some embodiments, the operations of block 940 (and 950, when performed) can be performed by the data plane component, such as illustrated in other figures herein. In some embodiments, the processing operations in block 940 (and 950, when performed) can be performed by a plurality of configurable traffic filters arranged to process upstream traffic toward the network endpoints and to process downstream traffic from the network endpoints. In such embodiments, the routing operations in block 940 can include the operations of sub-block 941, where the traffic controller (e.g., the data plane component) can distribute the processed upstream traffic among the network endpoints according to a load-balancing policy configured by the control plane component.

Although Figure 9 describes a method (e.g. procedure), the operations corresponding to the method (including any blocks and sub-blocks) can also be embodied in a non-transitory, computer-readable medium storing computer-executable instructions. The operations corresponding to the method (including any blocks and sub-blocks) can also be embodied in a computer program product storing computer-executable instructions. In either case, when such instructions are executed by processing circuitry associated with a traffic controller, they can configure the traffic controller to perform operations corresponding to the method of Figure 9.

Figure 10 is a schematic block diagram illustrating a virtualization environment 1000 of functions implemented by some embodiments.

In some embodiments, some or all of the functions described herein can be implemented as virtual components executed by one or more virtual machines implemented in virtual environment 1000 hosted by one or more of hardware nodes 1030. Such hardware nodes can be computing machines arranged in a cluster (e.g., such as in a data center or customer premise equipment (CPE)) where many hardware nodes work together and are managed via management and orchestration (MANO) 10100, which, among others, oversees lifecycle management of applications 1040. In some embodiments, however, such virtual components can be executed by one or more physical computing machines, e.g., without (or with less) virtualization of the underlying resources of hardware nodes 1030.

Hardware nodes 1030 can include processing circuitry 1060 and memory 1090. Memory 1090 contains instructions 1095 executable by processing circuitry 1060 whereby application 1040 can be operative for various features, functions, procedures, etc. of the embodiments disclosed herein. Processing circuitry 1060 can include general-purpose or special-purpose hardware devices such as one or more processors (e.g., custom and/or commercial off-the-shelf), dedicated Application Specific Integrated Circuits (ASICs), or any other type of processing circuitry including digital or analog hardware components or special purpose processors. Each hardware node can comprise memory 1090-1 which can be non-persistent memory for temporarily storing instructions 1095 or software executed by processing circuitry 1060. For example, instructions 1095 can include program instructions (also referred to as a computer program product) that, when executed by processing circuitry 1060, can configure hardware node 1020 to perform operations corresponding to the methods/ procedures described herein.

Each hardware node can comprise one or more network interface controllers (NICs)/ network interface cards 1070, which include physical network interface 1080. Each hardware node can also include non-transitory, persistent, machine-readable storage media 1090-2 having stored therein software 1095 and/or instructions executable by processing circuitry 1060. Software 1095 can include any type of software including software for instantiating one or more optional virtualization layers/ hypervisors 1050, and software to execute applications 1040.

In some embodiments, virtualization layer 1050 can be used to provide VMs that are abstracted from the underlying hardware nodes 1030. In such embodiments, processing circuitry 1060 executes software 1095 to instantiate virtualization layer 1050, which can sometimes be referred to as a virtual machine monitor (VMM). For example, virtualization layer 1050 can present a virtual operating platform that appears like networking hardware to containers and/or pods hosted by environment 1000. Moreover, each VM (e.g., as facilitated by virtualization layer 1050) can manifest itself as a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each VM can have dedicated hardware nodes 1030 or can share resources of one or more hardware nodes 1030 with other VMs.

Various applications 1040 (which can alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, containers, pods, etc.) can be hosted and/or run by environment 1000. Such applications can implement various features, functions, procedures, etc. of various embodiments disclosed herein.

In some embodiments, each application 1040 can be arranged in a pod, which can include one or more containers 1041, such as 1041a-b shown for a particular application 1040 in Figure 10. Container 1041a-b can encapsulate respective services 1042a-b of the particular application 1040. For example, a “pod” (e.g., a Kubernetes pod) can be a basic execution unit of an application, i.e., the smallest and simplest unit that can be created and deployed in environment 1000. Each pod can include a plurality of resources shared by containers within the pod (e.g., resources 1043 shared by containers 1041a-b). For example, a pod can represent processes running on a cluster and can encapsulates an application’s containers (including services therein), storage resources, a unique network IP address, and options that govern how the container(s) should run. In general, containers can be relatively decoupled from underlying physical or virtual computing infrastructure. In addition to the applications 1040, a traffic controller 1045 can also be run in the virtualization environment 1000 shown in Figure 10. Traffic controller 1045 can include, but is not limited to, various features, functions, structures, configurations, etc. of various traffic controller embodiments shown in various other figures and discussed in more detail above.

Further, to the extent that the prior art knowledge has not been explicitly incorporated by reference herein above, it is explicitly incorporated herein in its entirety. All publications referenced are incorporated herein by reference in their entireties.

The foregoing merely illustrates the principles of the disclosure. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, arrangements, and procedures that, although not explicitly shown or described herein, embody the principles of the disclosure and can be thus within the spirit and scope of the disclosure. Various exemplary embodiments can be used together with one another, as well as interchangeably therewith, as should be understood by those having ordinary skill in the art.

Claims

1. A traffic controller (200, 300, 400, 1245) for a container-based cluster (230, 330, 430) of one or more computing machines (1230), the traffic controller comprising: a data plane component (210, 310, 410, 510, 610) comprising a proxy (311, 411) configured to route non-HTTP upstream traffic to a plurality of network endpoints within the cluster; and a control plane component (220, 320, 420) coupled to the data plane component and arranged to dynamically configure processing of the non-HTTP upstream traffic by the data plane component and routing of the non-HTTP upstream traffic by the proxy.

2. The traffic controller of claim 1, wherein: the proxy is further configured to route non-HTTP downstream traffic from the plurality of network endpoints; and the control plane component is further arranged to dynamically configure processing of the non-HTTP downstream traffic by the data plane component and routing of the non-HTTP downstream traffic by the proxy.

3. The traffic controller of any of claims 1-2, wherein: the proxy is further configured to route HTTP upstream traffic to the plurality of network endpoints; and the control plane component is further arranged to dynamically configure processing of the HTTP upstream traffic by the data plane component and routing of the HTTP upstream traffic by the proxy.

4. The traffic controller of claim 3, wherein: the proxy is further configured to route HTTP downstream traffic from the plurality of network endpoints; and the control plane component is further arranged to dynamically configure processing of the HTTP downstream traffic by the data plane component and routing of the HTTP downstream traffic by the proxy.

5. The traffic controller of any of claims 1-4, wherein the control plane component comprises: an application programming interface, API, server (321, 421) configured to create one or more traffic objects representative of the data plane component, each traffic object being associated with one or more types of network endpoints; and a traffic configuration controller (322, 422) coupled to the API server and configured to determine a configuration for the data plane component based on the one or more traffic objects.

6. The traffic controller of claim 5, wherein the traffic configuration controller is configured to determine the configuration for the data plane component based on: determining a current state for the data plane component; determining a difference between the current state and the traffic objects representative of a desired state of the data plane component; and determining the configuration for the data plane component based on the difference.

7. The traffic controller of any of claims 5-6, wherein the control plane component further comprises a traffic commit controller (323, 423) configured to send the determined configuration to the data plane component.

8. The traffic controller of any of claims 5-7, wherein: the control plane component further comprises a discovery controller (324, 424) configured to determine availability and addresses of the plurality of network endpoints within the cluster; and the traffic configuration controller is configured to determine the configuration for the data plane component based on traffic objects associated with network endpoints determined to be available.

9. The traffic controller of any of claims 5-8, wherein each network endpoint is associated with one or more containers (1241) hosted by the cluster.

10. The traffic controller of any of claims 5-9, wherein the traffic configuration controller is configured to determine the configuration for the data plane component based on: for each particular traffic object, validating a configuration associated with the particular traffic object against a schema associated with the particular traffic object; for each particular traffic object, verifying that there are no conflicts within the configuration associated with the particular traffic object; and verifying that there are no conflicts between configurations associated with different ones of the traffic objects.

11. The traffic controller of any of claims 5-10, wherein the one or more traffic objects are arranged according to one of the following: a plurality of traffic objects, each associated with a different type of network endpoint; or a single traffic object associated with a plurality of different types of network endpoints.

12. The traffic controller of any of claim 5-11, wherein: the traffic configuration controller is further configured to determine the configuration for the data plane component based on one or more rule objects defining combinations of rules and policies for routing traffic; and each rule object is associated with one or more of the traffic objects.

13. The traffic controller of claim 12, wherein at least one of the rule objects includes a plurality of individual rules arranged in a tree structure.

14. The traffic controller of any of claims 5-13, wherein the types of network endpoints associated with the respective traffic objects can include any of the following:

Proxy Call Session Control Function (P-CSCF), Interrogating Call Session Control Function (I-CSCF), Serving Call Session Control Function (S-CSCF), Multimedia Telephony Application Server (MMTel AS), Multimedia Telephony Interworking Application Server (MMTel IW), Service and Continuity Control Application Server (SRVCC-AS), IP Short Message Gateway (IP-SM-GW), Breakout Gateway Control Function (BGCF), Interconnection Border Control Function (IBCF), Media Resource Function (MRF), Authentication and Aggregation Proxy, WebRTC Gateway,

Non-SIM Device Gateway (NSD Gw), and Entitlement Server.

15. The traffic controller of any of claims 5-14, wherein the configuration for the data plane component, as determined by the traffic configuration controller, includes one or more of the following: network interface, transport layer, layer-7 (L7) protocol, listening port, connection pooling, logging, tracing, performance counters, transport layer security (TLS), certificate management, circuit breaking, blacklisting, white listing, authorization, health check, retries, timeout, outlier detection, and rate limiting.

16. The traffic controller of any of claims 5-15, wherein the configuration for routing of the traffic by the proxy, as determined by the configuration controller, includes rules or policies for one or more of the following: internal layer-7 (L7) protocol encapsulation, non-FITTP header manipulation, custom consistent hash key for non-FITTP traffic, charging data, data analytics, ingress load balancing, service endpoints, distribution policy, traffic mirroring, and number of replicas of data plane instances.

17. The traffic controller of any of claims 1-16, wherein: the data plane component further comprises a plurality of configurable traffic filters (312, 412, 512, 612) to process upstream traffic toward the network endpoints and to process downstream traffic from the network endpoints; and the proxy comprises a load balancer configured to distribute the processed upstream traffic among the network endpoints according to a load-balancing policy configured by the control plane component.

18. The traffic controller of claim 17, wherein the traffic filters are configurable via a configuration provided by a traffic commit controller (323, 423) of the control plane component.

19. The traffic controller of any of claims 17-18, wherein: the plurality of traffic filters are configured in a pipeline arrangement; and each traffic filter in the pipeline arrangement is configured to perform one or more of the following operations on the traffic: parsing, compression, decompression, forwarding or routing, encryption, decryption, protocol encapsulation, protocol encoding, and protocol decoding.

20. The traffic controller of any of claims 17-19, wherein the plurality of traffic filters include: one or more first decoder filters configured to decode requests in the upstream traffic; and one or more first encoder filters configured to encode responses in the downstream traffic.

21. The traffic controller of claim 20, wherein: the first decoder filters are configured to decode requests of an external protocol type; the first encoder filters are configured to encode responses according to the external protocol type; and the plurality of traffic filters further comprises: one or more second encoder filters configured to encode the decoded requests according to an internal protocol type, and one or more second decoder filters configured to decode responses of the internal protocol type for input to the first encoder filters.

22. The traffic controller of claim 21, wherein at least one of the external protocol type and the internal protocol type is from the following list: Session Initiation Protocol (SIP), Remote Procedure Calls (RPC) over HTTP/2, HTTP/1, HTTP/2, Websocket, and Diameter.

23. The traffic controller of any of claims 1-22, wherein the proxy is an Envoy proxy including network-layer and application-layer filters for non-HTTP traffic.

24. A method performed by traffic controller (200, 300, 400, 1245) for a container-based cluster (230, 330, 430) of one or more computing machines (1230), the method comprising: creating (1110) one or more traffic objects representative of a data plane component of the traffic controller, wherein each traffic object is associated with one or more types of network endpoint within the cluster; determining (1130) a configuration for the data plane component based on the one or more traffic objects; and based on the configuration, processing (1140) non-HTTP upstream traffic and routing (1140) the non-HTTP upstream traffic to a plurality of network endpoints within the cluster.

25. The method of claim 24, wherein routing (1140) the non-HTTP traffic comprises distributing (1141) the processed non-HTTP upstream traffic among the network endpoints according to a load-balancing policy configured by the control plane component.

26. The method of any of claims 24-25, further comprising performing one or more of the following processing or routing operations (1150) based on the configuration: routing non-HTTP downstream traffic from the plurality of network endpoints; processing the non-HTTP downstream traffic; processing HTTP upstream traffic; routing the HTTP upstream traffic to one or more network endpoints within the cluster; routing HTTP downstream traffic from the one or more network endpoints; and processing the HTTP downstream traffic.

27. The method of claim 26, wherein any of the non-HTTP upstream, non-HTTP downstream, HTTP upstream, and non-HTTP upstream traffic is processed by a plurality of configurable traffic filters that are configured based on the configuration.

28. The method of any of claims 24-27, wherein: the method further comprises determining (1120) availability and addresses of the plurality of network endpoints within the cluster; and determining (1130) the configuration for the data plane component is based on traffic objects associated with network endpoints determined to be available.

29. The method of any of claims 24-28, wherein: the creating (1110) and determining (1120, 1130) operations are performed by a control plane component of the traffic controller; and the processing (1140, 1150) and routing (1140, 1150) operations are performed by the data plane component.

30. The method of any of claims 24-29, wherein each network endpoint is associated with one or more containers hosted by the cluster.

31. The method of any of claims 24-30, wherein determining (1131) the configuration for the data plane component comprises: for each particular traffic object, validating (1131) a configuration associated with the particular ingress object against a schema associated with the particular traffic object; for each particular traffic object, verifying (1132) that there are no conflicts within the configuration associated with the particular traffic object; and verifying (1133) that there are no conflicts between configurations associated with different ones of the traffic objects.

32. The method of any of claims 24-31, wherein determining (1130) the configuration for the data plane component comprises: determining (1134) a current state for the data plane component; determining (1135) a difference between the current state and the traffic objects representative of a desired state of the data plane component; and determining (1136) the configuration for the data plane component based on the difference.

33. A non-transitory, computer-readable medium (1290) storing computer-executable instructions (1295) that, when executed by processing circuitry (1260) associated with a traffic controller (200, 300, 400, 1245) for a container-based cluster (230, 330, 430) of one or more computing machines, configure the traffic controller to perform operations corresponding to any of the methods of claims 24-32.

34. A computer program product storing computer-executable instructions (1295) that, when executed by processing circuitry (1260) associated with a traffic controller (200, 300, 400, 1245) for a container-based cluster (230, 330, 430) of one or more computing machines, configure the traffic controller to perform operations corresponding to any of the methods of claims 24-32.