CN118118205A

CN118118205A - Self-learning firewall policy enforcement party

Info

Publication number: CN118118205A
Application number: CN202311554097.6A
Authority: CN
Inventors: R·科穆拉; R·古普塔; G·B·M·松卡达; T·班卡; T·斯里达尔; R·雅瓦特卡
Original assignee: Juniper Networks Inc
Current assignee: Juniper Networks Inc
Priority date: 2022-11-30
Filing date: 2023-11-21
Publication date: 2024-05-31

Abstract

Embodiments of the present disclosure relate to a self-learning firewall policy enforcement party. An example network system includes a processing circuit and one or more memories coupled to the processing circuit. The one or more memories are configured to store instructions that, when executed by the processing circuitry, cause the network system to obtain first business session metric data and execute a machine learning model to determine a business prediction based on the first business session metric data. The instructions cause the network system to obtain second traffic session metric data and determine anomalies in traffic based on a comparison of the traffic prediction and the second traffic session metric data. The instructions cause the network system to generate an indication of the anomaly based on the determination of the anomaly.

Description

Self-learning firewall policy enforcement party

Cross Reference to Related Applications

The present application claims priority from U.S. application Ser. No. 18/472,092 filed on month 21 of 2023, which claims priority from Indian provisional patent application Ser. No. 202241069004 filed on month 11 of 2022, which is incorporated herein in its entirety.

Technical Field

The present disclosure relates to computer networks.

Background

In a typical cloud data center environment, there are many interconnected servers that provide computing and/or storage capabilities to run various applications. For example, a data center may include facilities to host applications and services for subscribers, such as customers of a data center provider. For example, the data center may host all infrastructure equipment, such as network and storage systems, redundant power supplies, and environmental controllers. In a typical data center, clusters of storage servers and application servers (compute nodes) are interconnected via a high-speed switch fabric provided through one or more layers of physical network switches and routers. More complex data centers provide subscriber support equipment located in various physical hosting facilities to infrastructure throughout the world.

The connection between the server and the switch fabric occurs at a hardware module called a Network interface card (Network INTERFACE CARD, NIC). Conventional NICs include Application-specific integrated circuits (ASICs) to perform packet forwarding, which include some basic Layer 2/Layer 3 (L2/L3) functions. Among the many conventional NICs, packet processing, management, and other high-level functions, referred to as "data paths," are performed by the host CPU, i.e., the CPU of the server that includes the NIC. Thus, the CPU resources in a server are shared by applications running on that server and also by the data path processing. For example, in a 4-core x86 server, one core of multiple cores may be reserved for the data path, leaving three cores (or 75% of the CPU) for the application and the host operating system.

Some NIC suppliers have begun to include additional processing units in the NIC itself to offload at least some of the data path processing from the host CPU to the NIC. For example, the processing unit in the NIC may be a multi-core ARM processor with some hardware acceleration provided by the data processing unit (Data Processing Unit, DPU), field programmable gate array (Field Programmable GATE ARRAY, FPGA), and/or ASIC. Multiple NICs that include such enhanced data path processing capabilities are commonly referred to as smart NICs.

Disclosure of Invention

The advent of cloud-native applications is bringing additional complexity to the communication modes within the network due to the highly distributed nature and dynamic deployment conditions of cloud-native applications. The network is now expected to be application aware, which requires end nodes, e.g. servers, to participate in various network tasks that are not traditionally performed by switches and/or routers. This has accelerated the advent of Data Processing Units (DPUs) or intelligent Network interface cards (Smart networks INTERFACE CARD, SMARTNIC) for performing different Network services in an efficient manner. The present disclosure proposes a closed loop framework for implementing application aware network services using a smart NIC (also referred to herein as a DPU). In some examples, a machine learning model executing on a device and one or more smart NICs may create a self-correcting network for management and observation of micro-service based applications. With or through the use of machine learning techniques, the smart NIC may perform continuous monitoring of application performance metrics and may be able to take corrective action to remedy a security or performance issue in real-time (or near real-time).

Modern microservice-based applications can assume that the network is a black box. These services may interact with each other using frameworks such as remote procedure calls (e.g., gRPC) and Representational state transfer (Representational STATE TRANSFER, REST). These assumptions and interactions create an environment in which the network is unaware of the applications that the network is transmitting, and vice versa, which may result in underutilization of the capabilities of the network and micro-service based applications. An administrator may monitor applications and networks that occur within networks having different tool sets, which may increase the cost of such monitoring, and create unnecessary delays during such monitoring in response to discovered aberrations, anomalies, or other problems.

Network monitoring tools are typically not fully integrated with the rest of the infrastructure. In almost all monitoring architectures, switches and routers derive telemetry data using, for example, simple network management protocol (Simple Network Management Protocol, SNMP) or similar mechanisms so that monitoring software running on the same network or on a server in the cloud can analyze the telemetry data. These tools may identify problems, but since the tools receive limited telemetry data (e.g., network layer telemetry data), the tools have limited impact on packet flow. Similarly, application monitoring requires complex software or sidecars running in a service grid using many system resources. Thus, it may be highly desirable to integrate network and application layer monitoring and packet forwarding pipelines.

Switches based on packet processors (Programming Protocol-INDEPENDENT PACKET Processors, P4) that are independent of programming protocols, such as programmable pipelines, offer some promise, but are not welcome, for implementing network and application monitoring in combination with packet forwarding pipelines. Network processors have the potential to address such monitoring problems to some extent, but network processors are often presented at the edge of the network, which may not be optimal. The advent of DPUs and/or smart NICs provides opportunities to address such monitoring issues by allowing the monitoring software to be part of the packet forwarding flow without affecting latency and throughput. This advent enables the use of smart NICs to design a closed-loop framework for implementing application-aware services, such as application-aware threat detection and SLA enforcement, on a large scale in terms of bandwidth and latency. For example, smart NICs may be used to monitor, detect, and eliminate threats in real-time and/or near real-time. Previous work focused on using smart NICs to accelerate workload performance. The closed loop framework disclosed herein makes possible use cases that were not previously possible. For example, existing learning-based solutions monitor only some portions of network traffic, such as data coming in and out of a data center. Because of this limitation, existing learning-based solutions may only implement network policies at the network edge. With the popularity of public clouds, many enterprises share the same computing and network resources, requiring greater demands for policy enforcement in data centers. The closed loop system using smart NICs described in this disclosure may examine each packet generated in the data center to form a true application aware policy enforcement system.

In this disclosure, the term smart NIC is intended to cover a class of devices known as DPUs or infrastructure processing units (Infrastructure Processing Units, IPUs). These smart NICs are typically attached to the server in the form of PCIe cards. Each card may be overlaid with a dedicated silicon component (typically a single chip ASIC), which includes a set of processor cores and a dedicated silicon complex that may be used to perform packet processing and security functions. One advantage of utilizing a smart NIC is that the smart NIC includes a hybrid architecture, providing programmability, for example, through processing circuitry such as an ARM processor core, and dedicated acceleration and offloading functions through an ASIC. Due to the ability to introduce services on the ARM core, the server may be completely isolated from the network and security services running on the smart NIC.

In general, techniques are described for closed loop application aware network services that include an edge service platform that leverages the processing units of a smart NIC to increase the processing and network functionality of a network that includes servers of the smart NIC. Features provided by the edge services platform may include, for example: arranging an intelligent NIC; deployment of API driven services on smart NICs; addition, deletion and replacement of intelligent NICs; monitoring of services (e.g., security services) and other resources on the smart NIC; and management of connections between various services running on the smart NIC. More specifically, the present disclosure describes techniques for dynamically generating and implementing firewall policies, dynamically revising service level agreement (SERVICE LEVEL AGREEMENT, SLA) issues, dynamically processing packets of ranked based flows, dynamically creating flow table entries, dynamically optimizing firewall policy searches, and dynamically accessing and/or mitigating egress traffic security issues, each using an edge service platform and one or more intelligent NICs.

In one example, the present disclosure describes a network system including processing circuitry; and one or more memories coupled to the processing circuitry and configured to store instructions that, when executed by the processing circuitry, cause the network system to: acquiring first service session measurement data; executing a machine learning model to determine a business prediction based on the first business session metric data; acquiring second service session measurement data; determining an anomaly in the traffic based on a comparison of the traffic prediction and the second traffic session metric data; and generating an indication of the anomaly based on the determination of the anomaly.

In another example, the present disclosure describes a network system including processing circuitry; and one or more memories coupled to the processing circuitry and configured to store instructions that, when executed by the processing circuitry, cause the network system to: monitoring service grid traffic exiting each network interface card, NIC, of a plurality of NICs, each NIC of the NICs including NIC processing circuitry; monitoring service grid traffic entering each NIC; determining traffic session metric data based on the mesh traffic leaving each NIC and the mesh traffic entering each NIC; receiving an indication of an anomaly in the business from the machine learning model based on the business session metric data; determining a firewall policy based on the indication of the anomaly; and sending an indication of the firewall policy to at least one NIC of the one or more NICs.

In another example, the present disclosure describes a network system including a network interface card NIC including processing circuitry; and one or more memories coupled to the processing circuitry and configured to store instructions that, when executed by the processing circuitry, cause the network system to: transmitting telemetry data to the controller, the telemetry data comprising traffic session metric data; receiving an indication of a firewall policy from the controller based on the determination of the anomaly in the traffic, the anomaly in the traffic being determined based on traffic session metric data; and realizing the firewall policy.

The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

Drawings

FIG. 1 is a block diagram illustrating an example network system having a data center in which examples of the techniques described herein may be implemented.

FIG. 2 is a block diagram illustrating an example computing device that uses a network interface card with a separate processing unit to perform services managed by an edge services platform in accordance with the techniques described herein.

Fig. 3 is a conceptual diagram illustrating a data center having servers that each include a network interface card with a separate processing unit controlled by an edge services platform in accordance with the techniques of this disclosure.

FIG. 4 is a block diagram illustrating an example computing device that uses a network interface card with separate processing units to perform services managed by an edge services platform in accordance with the techniques described herein.

Fig. 5 is a block diagram illustrating an example system in accordance with the techniques of this disclosure.

Fig. 6 is a conceptual diagram illustrating an example closed-loop architecture for application-aware services in an intelligent network interface controller (SmartNIC) according to one or more aspects of the present disclosure.

Fig. 7 is a block diagram illustrating an example telemetry collector architecture, according to one or more aspects of the present disclosure.

Fig. 8 is a block diagram illustrating an example action unit architecture in accordance with one or more aspects of the present disclosure.

FIG. 9 is a block diagram illustrating an example Performance Analysis and Troubleshooting Engine (PATE) service unit architecture, according to one or more aspects of the present disclosure.

FIG. 10 is a block diagram illustrating an example anomaly detection pipeline in accordance with one or more aspects of the present disclosure.

Fig. 11A and 11B are conceptual diagrams illustrating service discovery control plane functions of an example attacker attempting to block a service grid, in accordance with one or more aspects of the present disclosure.

Fig. 12 is a graphical diagram illustrating example anomaly mitigation times as the number of attacks is increased, in accordance with one or more aspects of the present disclosure.

Fig. 13 is a table illustrating example anomaly detection for per-minute service discovery DNS count metrics in accordance with one or more aspects of the present disclosure.

FIG. 14 is a flow diagram illustrating example anomaly detection in accordance with one or more aspects of the present disclosure.

Fig. 15 is a conceptual diagram illustrating an example application communication topology before and after an attack in accordance with one or more aspects of the present disclosure.

Fig. 16 is a conceptual diagram illustrating an example automated Service Level Agreement (SLA) enforcement technique, according to one or more aspects of the present disclosure.

Fig. 17 is a flow diagram illustrating an example offloading technique in accordance with one or more aspects of the present disclosure.

Fig. 18 is a conceptual diagram illustrating an example self-learning stream priority technique in accordance with one or more aspects of the present disclosure.

Fig. 19 is a flow diagram illustrating an example flow ordering technique in accordance with one or more aspects of the present disclosure.

Fig. 20 is a flow diagram illustrating an example packet processing technique of a firewall service in accordance with one or more aspects of the present disclosure.

Fig. 21 is a flow diagram illustrating an example policy table filling technique in accordance with one or more aspects of the present disclosure.

Fig. 22 is a block diagram illustrating an example firewall policy optimizer in accordance with one or more aspects of the disclosure.

Fig. 23 is a flow diagram illustrating an example technique for reducing the number of firewall policies distributed to nodes of a distributed firewall in accordance with one or more aspects of the disclosure.

Fig. 24 is a conceptual diagram illustrating an example attack based on egress traffic in accordance with one or more aspects of the present disclosure.

Fig. 25 is a conceptual diagram illustrating an example self-learning egress traffic controller according to one or more aspects of the present disclosure.

Fig. 26 is a conceptual diagram illustrating an example application knowledge graph prior to an exit attack in accordance with one or more aspects of the present disclosure.

FIG. 27 is a conceptual diagram illustrating an example updated knowledge graph, according to one or more aspects of the present disclosure.

Fig. 28 is a timing diagram illustrating an example technique for configuring firewall policies to mitigate exit attacks in accordance with one or more aspects of the disclosure.

Fig. 29 is a flow diagram illustrating an example technique for configuring firewall policies to mitigate exit attacks in accordance with one or more aspects of the disclosure.

FIG. 30 is a flow diagram illustrating an example technique for responding to a malicious egress connection in accordance with one or more aspects of the present disclosure.

Like reference numerals refer to like elements throughout the description and drawings.

Detailed Description

Fig. 1 is a block diagram illustrating an example network system 8 having a data center 10 in which examples of the techniques described herein may be implemented. Typically, the data center 10 provides an operating environment for applications and services for a customer site 11, the customer site 11 having one or more customer networks coupled to the data center 10 by a service provider network 7. For example, the data center 10 may host infrastructure equipment such as network and storage systems, redundant power supplies, and environmental controls. The service provider network 7 is coupled to the public network 4. Public network 4 may represent one or more networks managed by other providers and may thus form part of a large-scale public network infrastructure, such as the internet. For example, public network 4 may represent a local area network (Local Area Network, LAN), a wide area network (Wide Area Network, WAN), the internet, a Virtual LAN (VLAN), a corporate LAN, a layer 3 Virtual private network (Virtual Private Network, VPN), an internet protocol (Internet Protocol, IP) intranet executed by a service provider executing service provider network 7, a corporate IP network, or some combination thereof.

Although the customer sites 11 and public network 4 are primarily shown and described as edge networks of the service provider network 7, in some examples one or more of the customer sites 11 and public network 4 are tenant networks within the data center 10 or another data center. For example, the data center 10 may host a plurality of tenants (customers), each associated with one or more Virtual Private Networks (VPNs). Each of the VPNs may implement one of the customer sites 11.

The service provider network 7 provides packet-based connectivity to additional customer sites 11, data centers 10 and public networks 4. The service provider network 7 may represent a network that is executed (and possibly owned) by a service provider to interconnect multiple networks. The service provider network 7 may implement multiprotocol label switching (Multi-Protocol Label Switching, MPLS) forwarding and may be referred to as an MPLS network or MPLS backbone in such instances. In some examples, service provider network 7 represents a plurality of interconnected autonomous systems, such as the internet, that provide services from one or more service providers.

In some examples, data center 10 may represent one of many geographically distributed network data centers. As shown in the example of fig. 1, the data center 10 may be a facility that provides network services to customers. The clients of the service provider may be collective entities such as corporations and governments or individuals. For example, a network data center may host web services for several companies and end users. Other model services may include data storage, virtual private networks, business engineering, file services, data mining, scientific or super computing, and the like. Although shown as a separate edge network of the service provider network 7, elements of the data center 10, such as one or more physical network functions (Physical Network Function, PNF) or virtualized network functions (Virtualized Network Function, VNF), may be included within the service provider network 7 core.

In this example, the data center 10 includes storage and/or computing servers interconnected via a switching fabric 14, the switching fabric 14 being provided by one or more layers Of physical network switches and routers, wherein the servers 12A-12X (referred to herein as "servers 12") are depicted as being coupled to Top-Of-Rack (TOR) switches 16A-16N. The present disclosure may refer to TOR switches 16A through 16N collectively as "TOR switches 16". TOR switches 16 may be network devices that provide layer 2 (MAC) and/or layer 3 (e.g., IP) routing and/or switching functions.

The server 12 may also be referred to herein as a "host" or "host device. The data center 10 may include a number of additional servers that are coupled to the other TOR switches 16 of the data center 10. In the example of fig. 1, servers 12A and 12X are directly coupled to TOR switch 16 in the example shown, and servers 12B, 12C, and 12D are not directly coupled to TOR switch 16 in the example shown. Servers 12B, 12C, and 12D may reach TOR switch 16 and IP fabric 20 via servers 12A or 12X, as will be described in further detail below.

The switch fabric 14 in the illustrated example includes interconnected TOR switches 16 (or other "leaf" switches), which TOR switches 16 are coupled to the distribution layers of rack switches 18A-18M (collectively, "rack switches 18"). The chassis switch may also be referred to as a "rotating" or "core" switch. Although not shown in the example of fig. 1, the data center 10 may also include one or more non-edge switches, routers, hubs, gateways, security devices (e.g., firewalls, intrusion detection, and/or intrusion prevention devices), servers, computer terminals, notebook computers, printers, databases, wireless mobile devices (e.g., cell phones or personal digital assistants), wireless access points, bridges, cable modems, application accelerators, and/or other network devices.

In some examples, TOR switches 16 and chassis switches 18 provide redundant (e.g., multi-homing) connections to IP fabric 20 and service provider network 7 to servers 12. The chassis switches 18 aggregate traffic flows and provide connections between TOR switches 16. The TOR switch 16 and the chassis switch 18 may each include one or more processors and memory and be capable of executing one or more software processes. The chassis switch 18 is coupled to an IP fabric 20, which IP fabric 20 may perform layer 3 routing to route network traffic between the data center 10 and the customer sites 11 via the service provider network 7. The switching architecture of the data center 10 shown in fig. 1 is merely an example. For example, other switching fabrics may have more or fewer switching layers. TOR switch 16 and chassis switch 18 may each include a physical network interface.

In this disclosure, the terms "packet flow", "traffic flow" or simply "flow" refer to a set of packets originating from a particular source device or endpoint and sent to a particular destination device or endpoint, respectively. For example, a single packet stream may be composed of 5 tuples: < source network address, destination network address, source port, destination port, protocol > identity. This 5-tuple typically identifies the packet stream to which the received packet corresponds. An n-tuple refers to any n items derived from the 5-tuple. For example, a 2-tuple for a packet may refer to a combination of < source network address, destination network address > or < source network address, source port > for the packet. The term "source port" refers to a transport layer (e.g., transmission control protocol (Transmission Control Protocol, TCP) or user datagram protocol (User Datagram Protocol, UDP)) port. "port" may refer to the physical network interface of the NIC.

Each server 12 may be a compute node, an application server, a storage server, or other type of server. For example, each of the servers 12 may represent a computing device configured to perform according to the techniques described herein, such as an x86 processor-based server. The server 12 may provide a network function virtualization infrastructure (Network Function Virtualization Infrastructure, NFVI) for a network function virtualization architecture (Network Function Virtualization, NFV).

Server 12 may host endpoints for one or more virtual networks executing on the physical networks represented in fig. 1 by IP fabric 20 and switch fabric 14. Endpoints may include one or more virtual network endpoints, such as virtual machines, containerized applications (e.g., orchestration platforms, including Kubernetes, docker swarm, mesos/Marathon, openShift, openStack, VMWare, and Amazon ECS), or applications executing natively on an operating system or bare machine. Containerization is a virtualization scheme based on operating system level virtualization. Like virtual machines, each container is virtualized and can be maintained isolated from each other and from the host. However, unlike virtual machines, each container may omit a personal operating system and only provide application suites and application specific libraries. Typically, the containers are executed by the host as isolated user space instances, and the operating system and general purpose libraries may be shared with other containers executing on the host. The containers may be managed as logically related groups of elements (sometimes referred to as "pods" in some orchestration platforms such as Kubernetes). Although described primarily in terms of a data center-based switching network, other physical networks, such as the service provider network 7, may support one or more virtual networks.

Each of the servers 12 includes at least one network interface card NIC of NICs 13A to 13X (collectively, "NIC 13"). For example, the server 12A includes the NIC 13A. Each NIC of the NICs 13 includes at least one port. Each NIC 13 may send and receive packets over one or more communication links coupled to ports of the NIC.

In some examples, each NIC in each NIC 13 provides one or more virtual hardware components for virtualized Input/Output (I/O). The virtual hardware component used to virtualize the I/O may be the virtualization of the physical NIC 13 ("physical function"). For example, in Single Root I/O virtualization (SR-IOV) described in the peripheral component interface special interest group SR-IOV specification, peripheral component interface (PERIPHERAL COMPONENT INTERFACE, PCI) Express (PCI Express, PCIe) physical functions of a network interface card (or "network adapter") are virtualized to present one or more virtual network interface cards as "virtual functions" for use by individual endpoints executing on server 12. In this way, virtual network endpoints may share the same PCIe physical hardware resources, and virtual functions are examples of virtual hardware components. As another example, one or more servers 12 may implement an available para-virtualization framework Virtio, e.g., for a Linux operating system, that provides emulated NIC functionality as a type of virtual hardware component. As another example, one or more servers 12 may implement Open vswitches to perform distributed Virtual multi-layer exchanges between one or more Virtual NICs (vnics) for hosted Virtual machines, where such vnics may also represent the types of Virtual hardware components. In some examples, the virtual hardware component is a virtual I/O (e.g., NIC) component. In some examples, the virtual hardware component is an SR-IOV virtual function, and may provide SR-IOV direct process user space access based on a Data Plane Development Kit (DPDK).

In some examples, including the example of fig. 1, one or more of the NICs 13 include multiple ports. NIC 13 may be interconnected via ports and communication links of NIC 13 to form NIC structure 23 having a NIC structure topology. NIC structure 23 is a collection of NICs 13 connected to at least one other NIC of NICs 13, the NICs 13, and a communication link coupling the NICs 13 to each other.

The NICs 13A to 13X include respective processing circuits, such as processing units 25A to 25X (collectively, "processing units 25"). The processing unit 25 may offload (offload) aspects of the data path from the CPU of the server 12. The one or more processing units 25 may be a multi-core ARM processor with hardware acceleration provided by a data processing unit (Data Processing Unit, DPU), a field programmable gate array (Field Programmable GATE ARRAY, FPGA), and/or an Application-specific integrated Circuit (ASIC). Because NIC 13 includes processing unit 25, NIC 13 may be referred to as a "smart NIC" or "GeniusNIC".

According to various aspects of the disclosed technology, the edge services platform uses the processing unit 25 of the NIC 13 to enhance the processing and networking functions of the switching fabric 14 and/or server 12 that includes the NIC 13. In the example of fig. 1, network system 8 includes an edge service controller 28. The present disclosure may also refer to edge service controllers, such as edge service controller 28, as edge service platform controllers.

The edge service control 28 can manage the operation of the edge service platform within the NIC 13 through the orchestration service portion executed by the processing unit 25; orchestrating API-driven service deployment on NIC 13; arranging addition, deletion and replacement of the NIC 13 within the edge service platform; monitoring services and other resources on NIC 13; and/or manage connections between various services 133 running on the NIC 13. Edge service controller 28 may include one or more computing devices, such as server devices, personal computers, intermediate network devices, and the like.

Edge service controller 28 may communicate information describing the availability on NIC 13, the topology of NIC structure 23, or other information about the edge service platform to orchestration system (not shown) or controller 24. An example orchestration system includes the vCenter of OpenStack, VMWARE or Microsoft corporation of Redmond, washington. Example controllers include Contrail controllers of Juniper NETWORKS or Tungsten Fabric. The controller 24 may be a network fabric manager. Additional information about the controller 24 operating with the data center 10 or other devices of other software-defined networks may be found in international application number PCT/US2013/044378, filed on 5/6/2013 and entitled "physical path determination for virtual network packet flows"; and U.S. patent No. 9,571,394, filed on month 3, 26 of 2014, and entitled "tunnel packet aggregation for virtual networks," each of which is incorporated by reference as if fully set forth herein.

In some examples, edge service controller 28 programs processing unit 25 of NIC 13 to route data packets along a data path through NIC structure 23, e.g., based on an application (service) associated with the data packets. Routing data packets along the data path through NIC structure 23 may avoid overloading the personal NIC in NIC structure 23 when multiple services on the host pair communicate with each other. For example, edge service controller 28 may manage data packets routed in NIC structure 23. As shown in fig. 1, NIC fabric 23 includes a plurality of NICs 13 coupled by communication links in a NIC fabric topology. In this example, edge service controller 28 may receive a resource availability value from NIC 13. Edge service controller 28 may determine a data path for data packets of a flow transmitted from a source NIC to a destination NIC using a protocol via a set of NICs including at least one NIC. NIC 13 includes a source NIC, a destination NIC, and a set of NICs. As part of determining the data path, edge service controller 28 may select a set of NICs based on the resource availability value. Edge service controller 28 may transmit data path data to the originating NIC and each NIC in the set of NICs such that the originating NIC and each NIC in the set of NICs identify data packets of the flow using identifiers of protocols and transmit data packets of the flow from the originating NIC to the destination NIC via the data path. The edge service controller 28 can establish multiple data paths in this manner. Unlike in conventional data center structures, the servers 12 may therefore directly switch packets, rather than via separate switching devices (e.g., the rack switch 18). The above may be considered as a form of service load balancing.

The example system of fig. 1 may form a closed loop framework for implementing application-aware network services using a smart NIC, such as NIC 13, as will be described further below.

Fig. 2 is a block diagram illustrating an example computing device 200 that uses a NIC230 with a separate processing unit 25 to perform services managed by an edge services platform in accordance with the techniques described herein. For example, NIC230 may implement a portion of a distributed firewall service, a portion of a service executing on a server, such as one of servers 12 in fig. 1, or the like. Computing device 200 of fig. 2 may represent a real or virtual server, and may represent an example instance of any of servers 12 of fig. 1. In the example of fig. 2, computing device 200 includes a bus 242 that couples the hardware components of the hardware environment of computing device 200. Specifically, in the example of FIG. 2, bus 242 couples NIC230, storage media 246, and microprocessor 210, which support single-route input/output virtualization (SR-IOV). In some examples, a front side bus couples microprocessor 210 and a memory device. In some examples, bus 242 couples memory device 244, microprocessor 210, and NIC230. Bus 242 may represent a PCIe bus. In some examples, a direct memory access (Direct Memory Access, DMA) controller may control DMA transfers between components coupled to bus 242. In some examples, components coupled to bus 242 control DMA transfers between components coupled to bus 242. In some examples, components coupled to bus 242 control DMA transfers between components coupled to bus 242.

Microprocessor 210 may include one or more processors, each including a separate execution unit ("processing core") to execute instructions conforming to an instruction set architecture. The execution units may be implemented as separate integrated circuits (INTEGRATED CIRCUIT, ICs) or may be combined within one or more multi-core processors (or "many-core" processors), each implemented using a single IC (i.e., a chip multiprocessor).

Disk 246 is representative of computer readable storage media including volatile and/or nonvolatile, removable and non-removable storage media implemented in any method or technology for storage of information such as processor readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, random access Memory (Random Access Memory, RAM), read-Only Memory (ROM), EEPROM, flash Memory, CD-ROM, digital versatile disks (DIGITAL VERSATILE DISC, DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic stories, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by microprocessor 210.

Storage 244 includes one or more computer-readable storage media that may include Random Access Memory (RAM), such as various forms of dynamic RAM (DYNAMIC RAM, DRAM), such as DDR2/DDR3 SDRAM, or static RAM (STATIC RAM, SRAM), flash memory, or any other form of fixed or removable storage media, that can be used to carry or store desired program code and program data in the form of instructions or data structures, and that can be accessed by a computer. Memory device 244 provides a physical address space that is comprised of addressable memory locations.

A Network Interface Card (NIC) 230 includes one or more interfaces 232 configured to exchange data packets using links of an underlying physical network. Interface 232 may include a port interface card having one or more network ports. NIC230 also includes an in-card memory 227 to, for example, store packet data. Direct memory access transmissions between NIC230 and other devices coupled to bus 242 may be read from and written to memory 227.

The memory device 244, NIC230, disk 246, and microprocessor 210 provide an operating environment for a software stack executing the hypervisor 214 and one or more virtual machines 228 managed by the hypervisor 214. Typically, virtual machines provide virtualized/guest operating systems to execute applications in an isolated virtual environment. Because the virtual machines are virtualized from the physical hardware of the host server, the executing applications are isolated from the hardware of the host and other virtual machines. The computing device 200 executes the hypervisor 214 to manage the virtual machines 228. Example hypervisors include Kernel-based virtual machines (Kernel-based Virtual Machine, KVM) of the Linux Kernel, xen, windows Hyper-V provided by ESXi, MICROSOFT provided by vmwire, and other open source and proprietary hypervisors. The hypervisor 214 may represent a Virtual machine manager (Virtual MACHINE MANAGER, VMM). Virtual machine 228 can host one or more applications, such as virtual network function instances. In some examples, virtual machine 228 may host one or more VNF instances, wherein each of the VNF instances is configured to apply network functions to the packets.

Another alternative to virtual machines is to virtualize containers, such as those provided by open source DOCKER container applications. Like virtual machines, each container is virtualized and can remain isolated from hosts and other containers. However, unlike virtual machines, each container may ignore a single operating system, providing only application suites and application-specific libraries. The containers are executed by the host as isolated user space instances and may share an operating system and a common library with other containers executing on the host. Thus, the container may require less processing power, storage, and network resources than the virtual machine. As used herein, a container may also be referred to as a virtualization engine, virtual private server, silo, or prison (jail). In some examples, the techniques described herein relate to a container and a virtual machine or other virtualized component.

Although the virtual network endpoint in fig. 2 is shown and described in terms of virtual machines, other operating environments, such as containers (e.g., DOCKER containers), may implement the virtual network endpoint. An operating system kernel (not shown in FIG. 2) may execute in kernel space and may include, for example, linux, berkeley software distribution (Berkeley Software Distribution, BSD), another Unix variant kernel, or a Windows server operating system kernel available from MICROSOFT.

The hypervisor 214 includes a physical driver 225 to use the physical functions provided by the NIC 230. In some cases, NIC230 may also implement SR-IOV to enable sharing of physical network functions (I/O) among virtual machines 224. Each port of NIC230 may be associated with a different physical function. The shared virtual devices, also referred to as virtual functions, provide dedicated resources such that each virtual machine 228 (and corresponding guest operating system) can access the dedicated resources of the NIC230, and thus are dedicated NICs for each virtual machine 228. The virtual functions may be lightweight PCIe functions that share physical resources with physical functions and with other virtual functions. According to the SR-IOV standard, NIC230 may have thousands of virtual functions available, but for I/O intensive applications, the number of virtual functions configured is typically much smaller.

The virtual machine 228 includes a separate virtual NIC229 that is submitted directly into the guest operating system of the virtual machine 228, thereby providing direct communication between the NIC230 and the virtual machine 228 via bus 242, using virtual functions assigned to the virtual machine. This may reduce the overload that hypervisor 214 involves in software-based VIRTIO and/or vSwitch implementations, where the memory address space of hypervisor 214 within memory device 244 stores packet data, and consume cycles of microprocessor 210 because packet data is copied from NIC230 to the memory address space of hypervisor 214 and from the memory address space of hypervisor 214 to the memory address space of virtual machine 228.

NIC230 may also include a hardware-based ethernet bridge 234. Ethernet bridge 234 may be an example of an embedded switch. Ethernet bridge 234 may perform layer 2 forwarding between virtual functions and physical functions of NIC 230. Thus, in some cases, ethernet bridge 234 provides hardware acceleration of packet forwarding between virtual machines 228, as well as hardware acceleration of packet forwarding between hypervisor 214 and any virtual machines 228, via bus 242. The hypervisor 214 may access physical functions via a physical driver 225. Ethernet bridge 234 may be physically separate from processing unit 25.

The computing device 200 may be coupled to a physical network switch fabric that includes an overlay network that extends the switch fabric from a physical switch to software or "virtual" routers coupled to physical servers of the switch fabric, including virtual router 220. The virtual router may be a process, thread, or component thereof executed by a physical server, such as server 12 in fig. 1, that dynamically creates and manages one or more virtual networks that may be used for communication between virtual network endpoints. In one example, the virtual router implements each virtual network using an overlay network that provides the ability to decouple the virtual address of the endpoint from the physical address (e.g., IP address) on the server executing at the endpoint. Each virtual network may use its own addressing and security scheme and may be considered orthogonal to the physical network and its addressing scheme. Various techniques may be used to transport packets within and across virtual networks through a physical network. At least part of the functions of the virtual router may be performed as one of the services 233 or the fabric service 235. In the example of fig. 2, virtual router 220 executes within management function 214 using physical functions for I/O, but virtual router 220 may execute within a hypervisor, a host operating system, a host application, one of virtual machines 228, and/or processing unit 25 of NIC 230.

In general, each virtual machine 228 may be assigned a virtual address for use within a corresponding virtual network, where each virtual network may be associated with a different virtual subnet provided by virtual router 220. For example, virtual machine 228 may be assigned its own virtual layer three (L3) IP address for sending and receiving communications, but may not be aware of the IP address of computing device 200 on which the virtual machine is executing. In this case, a "virtual address" is an address for an application that is different from the logical address of the underlying physical computer system (e.g., computing device 200).

In one implementation, computing device 200 includes a Virtual Network (VN) agent (not shown) that controls the coverage of the Virtual Network of computing device 200 and orchestrates the routing of data packets within computing device 200. Typically, a VN agent communicates with virtual network controllers of multiple virtual networks, generating commands that control the routing of packets. VN agents may operate as proxy servers for controlling flat messages between virtual machine 228 and virtual network controllers, such as controller 24 (fig. 1). For example, the virtual machine may request to send a message using its virtual address via the VN agent, and the VN agent may send the message in turn and a request to respond to the received message that issued the virtual address of the virtual machine of the first message. In some cases, virtual machine 228 may invoke a procedure or function call proposed by the application programming interface of the VN agent, and the VN agent may also handle encapsulation of the message, including addressing.

In one example, network packets generated or consumed by an instance of an application executing through virtual machine 228 within a virtual network domain, e.g., a three layer (L3) IP packet or a two layer (L2) ethernet packet, may be encapsulated in another packet (e.g., another IP or ethernet packet) transmitted by a physical network. Packets transmitted in a virtual network may be referred to herein as "inner packets" and physical network packets may be referred to herein as "outer packets" or "tunnel packets. Encapsulation and/or decapsulation of virtual network packets within physical network packets may be performed by virtual router 220. This function is referred to herein as tunneling and may be used to create one or more overlay networks. In addition to ipineip, other example tunneling protocols that may be used include IP throughout the generic routing encapsulation (Generic Route Encapsulation, GRE), virtual extensible local area network (Virtual Extensible Local Area Network, VXLAN), multiprotocol label switching (Multiprotocol Label Switching, MPLS) throughout GRE (MPLSoGRE), MPLS (MPLSoUDP) throughout the user datagram protocol (User Datagram Protocol, UDP), and the like.

As described above, the virtual network controller may provide a logically centralized controller to facilitate the operation of one or more virtual networks. For example, the virtual network controller may maintain a routing information base, e.g., one or more routing tables storing routing information for the physical network and one or more overlay networks. Virtual router 220 of hypervisor 214 implements network forwarding tables (Network Forwarding Table, NFT) 222A through 222N for N virtual networks for which virtual router 220 operates as a tunnel endpoint. Typically, each NFT 222 stores information forwarded for the corresponding virtual network and identifies where the data packet is forwarded and whether the data packet is encapsulated in a tunneling protocol, e.g., using a tunneling header that may include one or more headers that are different layers of the virtual network protocol stack. Each NFT 222 may be an NFT of a different routing instance (not shown) implemented by virtual router 220.

In accordance with the techniques of this disclosure, edge service controller 28 (fig. 1) uses processing unit 25 of NIC 230 to enhance processing and network functions of computing device 200. The processing unit 25 includes a processing circuit 231 to execute services orchestrated by the edge service controller 28. The processing circuitry 231 may represent any combination of processing cores, ASICs, FPGAs, or other integrated circuits and programmable hardware. In an example, the processing circuitry may include a System-on-Chip (SoC) with, for example, one or more cores, a network interface for high-speed packet processing, one or more acceleration engines for specific functions (e.g., security/cryptography, machine learning, storage), programmable logic, integrated circuits, and the like. Such a SoC may be referred to as a data processing unit (Data Processing Unit, DPU). The DPU may be an example of the processing unit 25.

In the example NIC230, the processing unit 25 executes an operating system kernel 237 and a user space 241 for services. The kernel 237 may be a Linux kernel, a Unix or BSD kernel, a real-time OS kernel, or other kernel for managing hardware resources of the processing unit 25 and managing the user space 241.

Services 233 may include networking, security (e.g., distributed firewall services), storage, data processing, co-processing, machine learning, or other services. The services 233, edge Services Platform (ESP) agent 236, and fabric service 235 include executable instructions. Processing unit 25 may execute instructions of service 233, ESP agent 236, and structure 235 as processes and/or within virtual execution elements, such as containers or virtual machines. As described elsewhere in this disclosure, the service 233 may enhance the processing capabilities of the host processor (e.g., microprocessor 210), for example, by enabling the computing device 200 to offload packet processing, security, or other operations that would otherwise be performed by the host processor. The service 233 may also provide security at the edge. The network services of service 233 may include security services (e.g., distributed firewalls), policy enforcement, proxy, load balancing, or other L4-L7 services.

Processing unit 25 executes ESP agent 236 to exchange data for edge service platform with edge service controller 28 (fig. 1). Although shown in the example of fig. 2 as being in user space 241, in other examples ESP agent 236 is a kernel module of kernel 237. For example, ESP agent 236 may collect and send telemetry data to edge service controller 28. Telemetry data may be generated by service 233 and may describe traffic in the re-network, availability of computing device 200 or network resources, resource availability of resources (e.g., memory or core utilization) of processing unit 25, or other information. As another example, ESP agent 236 may receive from edge service controller 28 a service code to execute any service 233, to configure the service configuration of any service 233, packets for injection into the network, or other data.

The edge service controller 28 manages the operation of the processing unit 25 through, for example, orchestration and configuration of the services 233, deployment of the services 233, which are performed by the processing unit 25; adding, deleting, and replacing NICs within the edge service platform, monitoring services 233, and other resources on the NIC230, and managing connections between the various services 233 running on the NIC 230. Example resources on NIC230 include memory 227 and processing circuitry 231.

Processing circuitry 231 executes fabric service 235 to perform packet switching between NIC230 and one or more other NICs directly connected to ports of NIC230, i.e., not via an external switch, such as TOR switch 16. Edge service controller 28 may provide topology information describing the topology of NIC structure 23 to fabric service 235 via ESP agent 236. Edge service controller 28 may provide flow information and/or forwarding information to fabric service 235 via ESP agent 236. The flow information describes and may be used to identify the packet flow. This forwarding information may be used to map packets received by NIC230 to the output ports of NIC 230. In some cases, the network service 235 may calculate forwarding information and/or flow information independently.

The fabric service 235 may determine the processing and forwarding of packets received at the NIC230 and bridge to the processing unit 25 through the ethernet bridge 234. Packets received by NIC230 may be sent from the NIC of another computing device to NIC230 or may originate from user space 245 of computing device 200. Like other services 233 of NIC230, fabric service 235 may process received packets. Based on information received from edge service controller 28 or information generated by fabric service 235, such as forwarding information and/or flow information, fabric service 235 may map received packets to an output port that is directly coupled to another NIC in the NIC fabric via a communication link.

Fig. 3 is a conceptual diagram illustrating a data center having servers that each include a network interface card with a separate processing unit controlled by an edge services platform in accordance with the techniques of this disclosure. Racks of computing nodes 307A-307N (collectively, "racks of computing nodes 307") may correspond to servers 12 of fig. 1, and switches 308A-308N (collectively, "switches 308") may correspond to switches of switching fabric 14 of fig. 1. Agent 302 or orchestrator 304 represents software executed by the processing unit (as a data processing unit or DPU, as shown in fig. 3) and receives configuration information for the processing unit and sends telemetry and other information for the NIC comprising the processing unit to orchestrator 304. The web service 312, the L4-L7 service 314, the telemetry service 316, and the Linux and software development kit (Software Development Kit, SDK) service 318 may represent examples of the service 233. Orchestrator 304 may represent an example of edge services controller 28 of fig. 1.

The network automation platform 306 connects and manages the network devices and orchestrators 304 through which the network automation platform 306 can utilize an edge services platform. For example, the network automation platform 306 may deploy network device configurations, manage the network, extract telemetry information, and analyze and provide an indication of network status.

FIG. 4 is a block diagram illustrating an example computing device that uses a network interface card with separate processing units to perform services managed by an edge services platform in accordance with the techniques described herein. Although a virtual machine is shown in this example, other instances of computing device 400 may also or alternatively run containers, local processes, or other endpoints for packet flows. Different types of vSwitch may be used, such as Open vSwitch or virtual router (e.g., contrail). Other types of interfaces between the endpoint and the NIC are also contemplated, such as tap interfaces, veth pair interfaces, and the like.

Fig. 5 is a block diagram illustrating an example system 500 in accordance with the techniques of this disclosure. The system 500 includes a plurality of servers 512A through 512H (collectively, "servers 512") communicatively coupled via a NIC fabric 523 and a switching fabric 514. The system 500 includes an edge service controller 528. Each of the plurality of servers 512A-512H may include a corresponding one of NICs 513A-513H (collectively, "NICs 513"). NIC structure 523 includes NIC 513.NIC fabric 523 may include a number of potential data paths between pairs of NICs 513 of switches that do not traverse switch fabric 514. Each of these "data paths" is a path from the source NIC to the destination NIC through NIC structure 523, and this term is different from the data path processing. Edge service controller 528 may be communicatively coupled to each NIC 513 in NIC fabric 523. NIC fabric 523 is communicatively coupled to switch fabric 514. Switch fabric 514 may include one or more switches.

Each of the servers 512 may have a configuration similar to that of the computing device 200. Each of the NICs 513 may have a configuration similar to that of the NIC 230. Edge service controller 528 may be similar to edge service controller 28. Although eight servers 512 and eight NICs 513 are shown in the example system 500 of fig. 5, alternative examples of systems may include fewer or greater numbers of servers 512 and NICs 513. Although each server is shown as including a single NIC, alternative examples of systems may include servers having more than one NIC.

Server 512 may execute one or more applications. As described with respect to fig. 1, in an example, the one or more applications may be server applications hosted by server 512 and may represent endpoints. In one example, the one or more applications may be NIC applications executed by a processing unit of NIC 513. Implementation of a data path between two different NICs at different servers may involve two phases. The first phase may be an orchestration phase and the second phase may be a forwarding phase. The edge service controller 528 may define or orchestrate one or more data paths between two different NICs at two different servers during the orchestration phase. The edge service controller 528 may provide the NIC with data path data associated with the orchestrated data path in the data path. The NICs in the orchestrated data paths may forward data packets according to the orchestrated data paths during the forwarding phase. The data path data may be an example of forwarding information described in accordance with fig. 1.

Implementation of the orchestration phase and forwarding phase will be described in terms of applications A1, A2 running on server 512E and applications A3, A4 running on server 512D. The applications A1, A2, A3, and A4 may be server applications (e.g., applications executed by a host processor) or may be NIC applications (e.g., applications executed by a processing unit on a NIC). In this example, application A1 and application A3 may be services in a service chain, and application A2 and application A4 may be services in a service chain.

Application A1 may be configured to generate application data for transmission in a data packet, and server 512E may be configured to send the data packet to application A3 according to a first protocol for transmission. The application A1 may be referred to as a first source application A1 and the application A3 may be referred to as a first destination application. Application A2 may be configured to send data packets for transmission to application A4 according to a second protocol in accordance with generating application data for transmission in the data packets. Application A2 may be referred to as a second source application A2 and application A4 may be referred to as a second destination application. The second protocol may be different from the first protocol.

Examples of the first and second protocols include, but are not limited to, transport layer protocols or tunneling protocols (which may make full use of transport layer protocols). For example, the first protocol may be a VXLAN protocol. For example, the second protocol may be a multi-protocol label switching/user datagram protocol (MPLSoUDP) protocol. Although the present example is described in terms of VXLAN and MPLSoUDP protocols, other protocols may be used. The server 512E that includes source applications A1 and A2 may be referred to as a source server 512E. NIC 513E at origin server 512E may be referred to as origin NIC 513E. The server 512D includes destination applications A3 and A4 and may be referred to as destination server 512D. NIC 513D at destination server 512D may be referred to as destination NIC 513D.

NIC513 and edge service controller 528 in NIC fabric 523 may implement NIC-based data packet forwarding. In such an environment, processing unit 25 in NIC513 may be shared by services running on association server 512 and NIC fabric 523. If all traffic between the two sets of servers 512 always takes the same data path, traffic between the servers can overload NIC513 and affect the services running on servers 512. For example, if traffic from application A1 to application A3 and traffic from application A2 to application A4 are forwarded from source NIC 513E to destination NIC 513D on the same data path, this may result in a relatively high resource utilization of any NIC513 on that data path, thereby affecting performance.

Edge service controller 528 can solve this problem by implementing "service aware" or "application-based" routing of data packets. Edge service controller 528 may orchestrate application-based data paths and one or more NICs in NIC51 forward data packets according to the orchestrated application-based data paths for the application pairs executing on server 512 or NIC 513.

When an application (or service) is deployed at one of the servers 512 or one of the NICs 513, the web service controller 528 may be provided with data regarding the deployed application during configuration of the deployed application. Examples of such data may include protocols associated with deployed applications and other applications that may communicate with the deployed applications. In addition, when an application is deployed to a host (e.g., one of servers 512), edge service controller 528 may configure the preferred transmission of the application in NIC structure 523. For example, if the first service (S1) and the third service (S3) communicate with each other using VXLAN and the second service (S2) and the fourth service (S4) use MPLSoUDP for communication, the edge service controller 528 may configure the NIC structure 523 to ensure that the transmission requirements of each application are met. For example, the edge service controller 528 may specify, e.g., in a flow table, that packets sent between services are to be externally header encapsulated. The service may run on top of the host OS or by the processing unit of NIC513, or both. In some examples, edge service controller 528 may deploy applications or devices to server 512 using techniques described elsewhere in this disclosure, e.g., a local service level agreement (SERVICE LEVEL AGREEMENT, SLA) and external SLAs based on NIC 513.

In examples where NIC 513E is a source NIC and NIC 513D is a destination NIC, NIC fabric 523 may include a plurality of different data paths between source NIC 513E and destination NIC 513D. Application of the packets by service 233 may utilize computing and bandwidth resources at each network card in NIC structure 523. In many cases, application of the packet by service 233 may utilize a percentage of the total available computing resources at some NICs 513, and the remaining percentage of computing resources may be used to implement data packet forwarding functionality (e.g., fabric service 235). Each NIC in NIC 513 in NIC fabric 523 may provide an edge service controller 528 with a resource availability value that indicates available computing resources at that network card 513. Example types of resource availability values may include values indicating CPU utilization, network utilization, and the like. The edge service controller 528 may identify the NIC 513 in the NIC structure 523 that is suitable for implementing the data packet forwarding function based on the resource availability value. For example, edge service controller 528 may compare the resource availability value received from each of NICs 513 to a resource availability threshold or to each other's NIC 513 resource availability to identify NICs 513 in NIC fabric 523 that are suitable for implementing data packet forwarding functions. Suitable NICs 513 may include NICs 513 with sufficient computing resources in processing unit 25 to apply the structural services to an expected amount of traffic for the application communication pair, a threshold amount of computing resources, or other criteria. The edge service controller 528 may use the identified NICs to orchestrate the data paths between the NICs in the NIC structure 523. When edge service controller 528 orchestrates a data path between pairs of NICs in NIC structure 523, edge service controller 528 may provide data path data to the NICs logically located in the data path to cause the NICs to forward data packets according to the orchestrated data path.

Fig. 6 is a conceptual diagram illustrating an example closed loop architecture for application aware services in a smart NIC according to one or more aspects of the present disclosure. The closed loop architecture of FIG. 6 includes a controller 602 and a performance analysis and troubleshooting engine (PATE analysis unit) 604. The controller 602 may be an example or portion of the edge service controller 28 of FIG. 1 or the controller 24 of FIG. 1. The PATE analysis unit 604 may be an example or part of the edge service controller 28, or may be executed on a server or smart NIC separate from the controller 602.

For example, a Kubernetes cluster (e.g., smart NIC cluster 606) may be created using smart NICs 606A-606N, which may be an example of NIC 13 of fig. 1, as a regular working node. The controller 602 may be configured to monitor applications executing on the smart NIC (e.g., smart NIC cluster 606) and the health of the smart NIC infrastructure, which may form a K8 cluster. In addition to monitoring, the controller 602 may be configured to provide and manage different application-aware services provided on the smart NIC-based working nodes. For example, two services in the controller 602, namely the telemetry collector 610 and the action unit 612, may be responsible for monitoring and managing applications, respectively.

Fig. 7 is a block diagram illustrating an example telemetry collector architecture 610 in greater detail, in accordance with one or more aspects of the present disclosure. As shown in fig. 7, telemetry collector 610 collects logs and/or metrics from various software and/or hardware components present in smart NIC cluster 606, including smart NIC 606A managed by controller 602. Telemetry controller 610 may use a metric collector 614, such as a promethaus metric collector. Telemetry collector 610 may also use a log collector 618, such as an elastiscearch log collector, which may be an open source log collector. Agents (not shown in fig. 7) present in the DPU, the service grid, the applications and the Operating System (OS) may export metrics and/or logs to the metrics collector 614 and the log collector 618, which may store the metrics and/or logs in at least one repository, such as a time series Database (TIME SERIES Database, TSDB) (e.g., metrics DB615 and/or log DB 617). The collected telemetry information may ultimately be used by the PATE analysis unit 604 for further monitoring and troubleshooting purposes, as will be explained later in this disclosure.

Fig. 8 is a block diagram illustrating an example action unit 612 architecture in greater detail, in accordance with one or more aspects of the present disclosure. As shown in fig. 8, the action unit 612 may be configured to determine a set of actions to take based on feedback received or obtained from the PATE analysis unit 604. Examples of actions include, but are not limited to, dynamic security policies for applications executing on the smart NIC cluster 606, e.g., a distributed firewall such as a containerized firewall, actions related to SLA requirements of applications that ensure the offloading of workloads by the smart NICs (e.g., the smart NICs of the smart NIC cluster 606), and the like. For different use cases, such as security or SLA, the PATE analysis unit 604 may generate different types of operational feedback. For each type of feedback, there may be corresponding plug-ins configured to generate a set of actions, including one or more operations to be performed on the managed entity. As shown in fig. 8, the security action unit 620 may be configured to generate firewall policies for a distributed firewall hosted on a smart NIC (e.g., the smart NIC of the smart NIC cluster 606). Similarly, SLA action unit 622 may be configured to dynamically offload workload onto smart NICs (e.g., smart NICs of smart NIC cluster 606) for improved performance. The network consistency unit 624 may be configured to provide consistency across the configuration of managed entities, such as smart NICs, switches, hosts, and the like.

FIG. 9 is a block diagram illustrating an example PATE analysis unit 604 architecture in greater detail, in accordance with one or more aspects of the present disclosure. The PATE analysis unit 604 may be configured to collect and analyze application and underlying infrastructure performance telemetry for end-to-end observability and troubleshooting. According to use cases, the PATE analysis unit 604 may generate actionable feedback that the action unit 612 of the controller 602 may use to take corrective steps to fix the problem.

The PATE analysis unit 604 may include a cross-layer telemetry extraction pipeline 630, an anomaly detection service 632, a cause and effect analysis service 634, and a topology analysis service 636. The PATE analysis unit 604 may receive telemetry from multiple layers in the heap, such as applications, networks, computing, firewalls, smart NICs, etc. The cross-layer telemetry extraction pipeline 630 may be configured to normalize the data models from the multiple layers and persist in TSDB (not shown in fig. 9) for further analysis, such as by the anomaly detection service 632, the cause and effect analysis service 634, and the topology analysis service 636, as further described below.

FIG. 10 is a block diagram illustrating an example anomaly detection pipeline in accordance with one or more aspects of the present disclosure. Anomaly detection service 632 can include one or more machine learning models 645 stored in model server 646. The anomaly detection service 632 may be configured to detect any anomalies in the time-series telemetry data received from the multiple layers in the stack. For example, the anomaly detection service 632 can use an unsupervised learning-based random cut forest (Random Cut Forest, RCF) method to detect any deviation from expected behavior, and/or other unsupervised learning models to detect anomalies in the received time series telemetry data.

Telemetry data from multiple sources across application, computing, and network layers may be extracted by REST API 640 (e.g., kafka service) and then persisted to TSDB 642, which may be Thanos TSDB, for example. The training pipeline 644 may be configured as a baseline performance model of different key performance indicators (Key Performance Indicator, KPIs) learned across layers in the network. Model server 646 may be used to host a dynamically learned performance model (e.g., a trained machine learning model of machine learning model 645). During the inference period, inference pipeline 648 may subscribe to KPI telemetry from API 640 and perform real-time inference for anomaly detection. Whenever an anomaly is detected, according to the KPI, an appropriate anomaly detection message indicating the anomaly may be published on the API 640, which may then be used by a different subscription service, such as the cause and effect analysis service 634 (FIG. 9).

Referring back to FIG. 9, the cause and effect analysis service 634 may be configured to analyze cross-layer telemetry from the application layer to the network layer (or vice versa) to infer where the possible root cause came from for troubleshooting purposes. For example, where each application includes a plurality of micro-services in communication with each other, the PATE analysis unit 604 may perform continuous monitoring of each instance of the micro-services and each instance of the micro-services that interact with other services. The cause and effect analysis service 634 can be configured to determine a likely root cause of the application performance degradation. In micro-service based applications, upstream micro-service performance may be affected by downstream application performance. For example, the cause and effect analysis service 634 can independently analyze telemetry data from each segment of the call sequence and help infer whether any downstream micro-services are possible root causes of end-to-end performance degradation of the application.

The topology analysis service 636 can be configured to monitor any topology changes and generate feedback message(s) for the monitored consumer at any tier. An unlimited example of topology changes is deviations in the normal sequence of service call orders in micro-service based applications. The topology analysis service 636 can analyze such deviations for each layer to determine topology changes. For example, the topology analysis service 636 can determine: i. ) Newly adding a node set; ii.) the removed set of nodes; iii.) a new edge set; and/or iv.) deleted edge sets. Such analysis is useful for security use cases, such as detecting improper access to services by an intrusion service. The topology analysis service 636 can generate anomaly signals including any analysis results and provide the anomaly signals as feedback to the controller 602 to take remedial action, such as via the action unit 612 (fig. 8).

The self-correcting framework of the present disclosure, such as that of fig. 6, can dynamically detect different types of attacks and mitigate those attacks, e.g., using a micro-service application based on a service grid (e.g., istio). This disclosure now discusses experiments using Distributed Denial Of Service (DDOS) aggressors, external aggressors, and internal aggressor use cases.

In accordance with the techniques of this disclosure, a system, such as that of fig. 6, may be configured to execute a machine learning model to determine business predictions based on business session metric data collected over a first period of time. The system may be configured to determine anomalies in the traffic based on a comparison of traffic predictions and traffic session metric data collected at a second time period or a second time (e.g., current traffic session metric data). For example, the anomaly may represent a Domain name service (Domain NAME SERVICE, DNS) attack, a TCP flooding attack, or the like. Based on the determination of the anomaly, the system may be configured to generate an indication of the anomaly. The system may be configured to use the indication of the anomaly to determine or generate a firewall policy configured to mitigate the anomaly.

Fig. 11A and 11B are conceptual diagrams illustrating service discovery control plane functions of an example attacker attempting to block a service grid, in accordance with one or more aspects of the present disclosure. For example, an attacker may attempt to block the service discovery control plane function of the service grid (fig. 11A). When an attack is successful, it can crash the service grid due to failure of service discovery requests from other unaffected services. Thus, a self-learning firewall policy enforcement procedure may be required. The service grid experiment platform builds a service discovery technology based on Domain Name Service (DNS) on the Kubernetes platform. As shown in fig. 11B, a distributed firewall (e.g., DF 650) may run on each smart NIC (e.g., POD 1, POD 2, POD 3) attached to each node of the Kubernetes platform. A distributed firewall may monitor service grid traffic to and from each node of the Kubernetes cluster. The distributed firewall may derive traffic session metrics (not shown in fig. 11A-11B for simplicity) to TSDB (e.g., metrics DB 615 or the like). The PATE analysis unit 604 (also not shown in FIGS. 11A-11B for simplicity) may analyze session metric data derived from TSDB to identify a large number of DNS service requests by a compromised (populated) pod 654 running on a Kubernetes node. The PATE analysis unit 604 may perform such analysis in near real-time and, thus, may detect anomalies in near real-time. In some examples, the distributed firewall may derive the traffic session metric data directly to the PATE analysis unit 604, e.g., when real-time learning is desired or needed, in addition to or instead of deriving the traffic session metrics to the time series database.

When the number of service discovery requests sent by a service grid pod (e.g., the damaged pod 654) within a predetermined period of time is higher than the regular service request count, the PATE analysis unit 604 may identify the sending of a higher number of service discovery requests within the predetermined period of time as anomalous and notify the controller 602 of the anomaly. The regular service request count may be the number of service requests normally made by the service, such as during a previous period equal to the predetermined period. The regular service request count may be learned by a machine learning model 645 (fig. 10) of the model server 646 of the PATE analysis unit 604. For example, when a service is running for a period of time, the service may generate a pattern of the number of service discovery requests in the timeline. This pattern may take the form of a series of service discovery request counts, which may be learned by a machine learning model. Similar techniques as those discussed with respect to fig. 11A-11B may be similarly applied to identify a TCP flooding attack as anomalous.

In some examples, action unit 612 (fig. 8) may dynamically create and/or select a distributed firewall policy to mitigate anomalies. As shown in fig. 11B, any traffic from the pod 654 that is compromised may be blocked by the distributed firewall.

Fig. 12 is a graphical diagram illustrating example anomaly mitigation times as the number of attacks is increased, in accordance with one or more aspects of the present disclosure. As shown, under some known preconditions, the PATE analysis unit 604 may detect and mitigate anomalies almost immediately, for example, when the machine learning model of the machine learning model 645 run by the PATE analysis unit 604 is a model that has been trained, and the mitigation process includes predetermined firewall rules for a distributed firewall.

Fig. 13 is a table illustrating example anomaly detection for per-minute service discovery DNS count metrics in accordance with one or more aspects of the present disclosure. The table of fig. 13 shows the performance of the anomaly detection service 632 of the above-described path analysis unit 604 (two of fig. 9). In this use case, the PATE analysis unit 604 randomly cuts the forest using an unsupervised learning model to perform anomaly detection. As shown in fig. 13, the model accuracy is about 98% overall for injected anomalies, with f-1 scores as high as 97% due to the DDOS attack shown in fig. 11B.

FIG. 14 is a flow diagram illustrating example anomaly detection in accordance with one or more aspects of the present disclosure. The technique of fig. 14 is discussed in terms of fig. 6 as being performed by different portions of the system. Each portion of the system may perform those techniques that belong to the respective portion, or in some examples, one or more techniques that belong to any portion of the system may be performed by another portion of the system or a combination of portions of the system.

The smart NIC 606A may send telemetry data to the controller 602, including traffic session metric data (660). For example, the smart NIC 606A may monitor telemetry associated with the smart NIC 606A and send telemetry data (e.g., metrics and/or logs) including traffic session metrics, such as a count of the number of DNS requests issued by a virtual network endpoint of a host device of the smart NIC 606A, to the telemetry collector 610 of the controller 602.

Telemetry collector 610 may monitor service grid traffic leaving each of a plurality of NICs, each of which includes NIC processing circuitry (662). For example, telemetry collector 610 may monitor the service grid traffic represented in the telemetry data, leaving smart NIC 606A and other smart NICs of smart NIC cluster 606. Telemetry collector 610 may monitor service grid traffic entering each NIC (664). For example, telemetry collector 610 may monitor service grid traffic represented in telemetry data that enters smart NIC 606A and other smart NICs of smart NIC cluster 606.

Telemetry collector 610 may determine traffic session metric data based on the mesh traffic exiting each NIC and the mesh traffic entering each NIC (666). For example, telemetry collector 610 may determine the number of DNS requests during various time periods based on grid traffic leaving smart NIC 606A, and leaving smart NIC 606A

The PATE analysis unit 604 may obtain first business session metric data (668). For example, the PATE analysis unit 604 may receive or retrieve the first business session metric data from the telemetry collector 610.

The PATE analysis unit 604 may execute a machine learning model to determine business predictions based on the first business session metric data (670). For example, the PATE analysis unit 604 may execute the machine learning model(s) 645 (FIG. 10) to analyze the first traffic session metric data to determine a predicted number of predictions for the DNS request. For example, the machine learning model(s) 645 may be trained to predict traffic flows, such as the number of DNS requests by virtual network endpoints of host devices expected to be in a given expected time period, and the PATE analysis unit 604 may compare the number of DNS requests by virtual network endpoints of host devices in a predetermined time period reflected in the traffic session metric data to the expected number of DNS requests.

The PATE analysis unit 604 may obtain second business session metric data (672). For example, the PATE analysis unit 604 may receive or retrieve the second business session metric data from the telemetry collector 610. The second traffic session metric data may be related to traffic subsequent to the traffic associated with the first traffic session metric data.

The PATE analysis unit 604 may determine anomalies in the traffic based on a comparison of the traffic prediction and the second traffic session metric data (673). For example, the PATE analysis unit 604 may determine that the number of DNS requests of the virtual network endpoint of the host device for traffic session metric data within the predetermined period of time is greater than the number of intended DNS requests for the predetermined period of time based on the analysis of the traffic session metric data.

The PATE analysis unit 604 may generate an indication of the anomaly based on the determination of the anomaly (674). For example, the PATE analysis unit 604 may generate an indication of the anomaly to provide to the controller 602 and/or the smart NIC 606A.

The controller 602 may receive or otherwise obtain an indication of an anomaly in the business from the machine learning model and based on the business session metric data (676). For example, the controller 602 may receive an indication of the anomaly generated by the PATE analysis unit 604 in step 674.

The controller 602 may determine a firewall policy based on the indication of the anomaly (678). For example, the controller 602 may select a firewall policy from a plurality of existing firewall policies or generate a new firewall policy to address the anomaly.

The controller 602 may send an indication of the firewall policy to at least one NIC of the one or more NICs (680). For example, the controller 602 may send an indication of the firewall policy to the smart NIC 606A.

The smart NIC 606A may receive an indication of a firewall policy from the controller 602 based on a determination of an anomaly in the traffic, the anomaly in the traffic being determined based on traffic session metric data (682). For example, the smart NIC 606A may receive an indication of a firewall policy from the controller 602. The smart NIC 606A may implement a firewall policy (684). For example, if the indication of the firewall policy identifies an existing firewall policy stored in memory of the smart NIC 606A, the smart NIC 606A may load and execute the existing firewall policy. If the indication of the firewall policy includes a new firewall policy, the smart NIC 606A may store the new firewall policy in memory and/or execute the new firewall policy. In this way, the technique of fig. 14 may block current DNS attacks and/or block duplication of future DNS attacks in real-time and/or near real-time.

In some examples, the PATE analysis unit 604 may send an indication of the anomaly to a controller of the distributed firewall. For example, the controller 602 may implement a controller of a distributed firewall. In some examples, the anomaly indicates a DNS attack or a TCP flood attack. In some examples, the PATE analysis unit 604 may obtain business session metric data from a time series database (e.g., the metric database 615). In some examples, the second traffic session metric data includes a number of domain name service requests by the virtual network endpoint of the host device within the time period, and the traffic forecast includes an expected number of domain name service requests expected during the time period. In some examples, as part of determining the anomaly, the PATE analysis unit 604 may determine that the number of domain name service requests by the virtual network endpoint of the host device of the smart NIC 606A is greater than the number of intended domain name service requests within the time period. In some examples, the number of expected domain name service requests is based on domain name service requests during operation during a previous period of time (e.g., during normal operation). For example, the number of intended domain name service requests may be learned by a machine learning model, such as an unsupervised random cut forest machine learning model. In some examples, the first traffic session metric data and the second traffic session metric data are indicative of a service grid traffic. In some examples, the first traffic session metric data and the second traffic session metric data are associated with one or more NICs. In some examples, one or more NICs may implement a distributed firewall.

In some examples, the controller 602 may generate or select a firewall policy based on the indication of the anomaly. In some examples, the indication of the firewall policy includes at least one of a firewall policy or an identification of a firewall policy.

In some examples, the controller 602 may store the business session metric data in a time series database (e.g., metric database 615). In some examples, the traffic session metric data includes a number of domain name service requests by the virtual network endpoint of the host device within the time period, and the anomaly indicates that the number of domain name service requests by the virtual network endpoint of the host device within the time period is greater than a number of expected domain name service requests determined by the machine learning model. In some examples, the number of expected domain name service requests is based on domain name service requests made during normal operation and is determined by a machine learning model. In some examples, the controller 602 may select a firewall policy or generate a firewall policy.

In some examples, the traffic session metric data includes a number of domain name service requests by virtual network endpoints of the host device over a period of time, and the anomaly is determined by a machine learning model. In some examples, implementing the firewall policy causes the smart NIC 606A to stop sending domain name service requests from the virtual endpoint of the host device. In some examples, the smart NIC 606A implements an instance of a distributed firewall.

Grid export malware attacks are now discussed. In this second use case, the attacker gains access to the service grid application from outside, for example through a vulnerability or DevOps process in the application, and then the attacker can further continue to attack the external web server, application and/or database system. An attacker may attempt to steal data and transmit the data to an attacker's external server. In some cases, the attacker may attempt to download malicious code from their server to the service grid. All of these conditions can lead to variations in the conventional communication mode of the application. In this use case, a self-correcting framework, such as the framework of FIG. 6, may detect such an egress traffic attack and mitigate such an attack, for example, by isolating a compromised application pod using a distributed firewall. The PATE analysis unit 604 may use the topology analysis service 636 to detect and/or analyze any unnecessary changes in the application communication mode. By using telemetry data of the service grid, the topology analysis service 636 can dynamically build a knowledge graph that captures the baseline application communication patterns and continuously monitor the evolution of the knowledge graph to discover any suspected topology changes.

Experimental results show that the current self-correcting closed loop framework using topology analysis service 636 is able to detect and mitigate anomalies within 4 minutes. However, experiments indicate that real-time or near real-time event-based designs will further reduce the detection and troubleshooting time for the use case.

Grid DDoS attacks are now discussed. In this third example, when a service grid application pod is compromised, an attacker can use the compromised pod to generate malicious traffic to render some or all of the services of the grid unavailable. While service grid implementations provide some techniques to deter these kinds of attacks, these techniques are mostly limited to layer 7 and to low bandwidth traffic.

Similar to the method for the second use case, the PATE analysis unit 604 uses the topology analysis service 636 to detect DDoS attacks. In an experimental setup, the self-correcting framework is able to detect anomalies in a sequence of several minutes.

Other use cases of a closed loop framework for implementing application aware services using smart NICs are now described. Fig. 15 is a conceptual diagram illustrating an example application communication topology before and after an attack in accordance with one or more aspects of the present disclosure. For example, the communication topology 685 describes communication paths between hosts, pods, and deployments in a network. Communication topology 687 describes the communication paths between hosts, pods, and deployments after an attack. Attacks may result in new service calls between deployments and/or new deployments.

Smart NICs may be popular due to the ability of smart NICs to perform packet processing in hardware (e.g., DPUs). This capability allows applications to offload packet processing of some applications to the DPU, which may improve throughput and packet delay, thus speeding up the workflow of the applications. However, hardware resources in the DPU are limited. For example, on most DPUs, the number of stream entities may be limited to 4K (e.g., 4,096). Other DPU resources, such as encryption units, regular expression processing, etc., have their own limitations. To cut costs, most cloud providers run hundreds of client applications on the same server. In order to improve overall system performance, care should be taken not to press through the DPU. Current solutions either allocate resources manually or on a first-come-first-served basis, which may not be optimal.

The present disclosure includes techniques for allocating DPU resources using closed loop monitoring and machine learning techniques to allocate DPU resources when needed. For example, if an application fails to meet its associated SLA, the system may accelerate the application.

Fig. 16 is a conceptual diagram illustrating an example automated SLA enforcement technique, according to one or more aspects of the present disclosure. The PATE analysis unit 604 may be configured to accelerate the application, such as initiating offloading of portions of the application or service from processing circuitry of the server to processing circuitry (e.g., a DPU) of the smart NIC based on the application not meeting the requirements of the relevant SLA. For example, web application 690 may include or utilize encryption, such as IP security (IPSec 694). In one example, web application 690 may have an SLA that requires 10,000 active connections. PATE analysis unit 604 may continuously monitor telemetry data of web application 690 and may enable the password to offload IPSec 694 to DPU 692, for example, if the web application is losing a connection, resulting in a total number of active connections below 10,000, thereby speeding up IPSec 694 to address the failure of web application 690 to meet the requirements of the associated SLA. Similar techniques may be used to dynamically increase network throughput, packet delay, and/or other SLA requirements.

In some examples, the PATE analysis unit 604 may continue to monitor the number of active connections and if a level of improvement in the loss of an active session is observed, it may initiate moving IPSec694 back to the processing circuitry of the server. For example, PATE analysis unit 604 may initiate offloading IPSec694 from DPU 692 back to the processing circuitry of the server based on the number of dropped lines or the ratio of dropped lines of an active connection or session that meets a threshold. In some examples, the PATE analysis unit 604 may use one or more machine learning models 645 to determine the threshold.

Fig. 17 is a flow diagram illustrating an example offloading technique in accordance with one or more aspects of the present disclosure. The PATE analysis unit 604 may obtain telemetry data (700). For example, the PATE analysis unit 604 may retrieve or receive telemetry data associated with the web application 690.

The PATE analysis unit 604 may determine that an application running on a server processing circuit that does not include processing circuitry residing on the network interface card NIC (702) does not meet at least one Service Level Agreement (SLA) requirement based on the telemetry data. For example, the PATE analysis unit 604 may determine that the web application 690 running on the server 696 is meeting the SLA requirements.

The PATE analysis unit 604 may determine to offload at least one component of the application from the server processing circuitry to processing circuitry residing on the NIC (704) based on the application not meeting at least one SLA requirement. For example, PATE analysis unit 604 may determine to offload IPSec 694 from server 696 processing circuitry to processing circuitry of DPU 692.

In some examples, the at least one SLA requirement includes at least one of: the number of active connections, network throughput, or packet delay. In some examples, at least one component includes a cryptographic function. In some examples, the PATE analysis unit 604 may send a first notification to at least one of the servers (e.g., the server 696) or the controller (e.g., the controller 602) indicating a determination to offload at least one component of the application from the server processing circuitry to processing circuitry residing on the NIC. In some examples, in response to the first notification, the server 696 can offload at least one component of the application from the server processing circuitry to processing circuitry residing on the NIC.

In some examples, the PATE analysis unit 604 may determine that the application meets at least one SLA requirement based on a determination to offload at least one component of the application from the server processing circuitry to processing circuitry residing on the NIC and based on analysis of telemetry data. The PATE analysis unit 604 may determine that the attribute of the application for at least one SLA requirement meets a threshold. The PATE analysis unit 604 may determine to move at least one component of the application from processing circuitry residing on the NIC to the server processing circuitry based on the attributes of the application meeting the threshold. In some examples, the PATE analysis unit 604 may execute a machine learning model (e.g., machine learning model(s) 645) to determine the threshold. In some examples, the attributes of the application include a number of dropped lines within a predetermined period of time or a rate of dropped lines of an active connection or session during a predetermined period of time.

In some examples, the PATE analysis unit 604 may send a second notification to at least one server or controller indicating a determination to move at least one component of the application from processing circuitry residing on the NIC to server processing circuitry. In some examples, in response to the second notification, the server 696 can move at least one component of the application from processing circuitry residing on the NIC to the server processing circuitry.

Fig. 18 is a conceptual diagram illustrating an example self-learning stream priority technique in accordance with one or more aspects of the present disclosure. One of the popular services deployed by users in a smart NIC environment is a distributed firewall. Managing east-west traffic in a data center, where traffic rates can reach 100xTbps, can be difficult for a network administrator. Running a firewall on a smart NIC may solve this problem by policing traffic at the host rather than in the network. Typically, firewalls work by applying all configured policies to each incoming and outgoing packet to determine whether the packet should be allowed or denied. Because applying each policy for each data packet can be very time consuming, to expedite processing, a firewall may create a flow entry for each new flow and use the information cached in the flow table to process future data packets for that flow. For example, the firewall may apply a plurality of policies for each first packet of the flow. The firewall may skip multiple policy lookups for subsequent packets of the flow, but these subsequent packets may still go through the flow processing stage (e.g., look up the subsequent packets in the flow table). The entire sequence of such searches may also be time consuming and may occupy a significant amount of CPU resources, which may limit the overall throughput of the firewall.

Thus, to increase firewall throughput, a distributed firewall may analyze flows where flow processing is based on criticality or importance of the flow. For example, one or more machine learning models 645 of the PATE processing unit 604 may learn the flow of each application and rank each application according to the importance of the application, e.g., according to weight. For example, the PATE analysis unit 604 may predict the weights of the streams based on the criteria set forth below. Accordingly, the techniques described herein may improve firewall throughput. For example, a firewall may perform flow processing based on the criticality or importance of the flow, rather than processing the flow of each subsequent data packet of the flow. Using dynamic telemetry data, one or more machine learning models 645 may learn the flows for each application and order the flows in order based on the importance of the respective flows. Example criteria that may be used to determine flow ordering based on importance include, but are not limited to, flow hit rate, hit rate for the relevant flow, flow near miss rate, flow metric criticality, and the like.

The Flow Hit Rate (FHR) may be the number of successful firewall Flow evaluations of the total number of evaluations. For example, FHR may be calculated as: FHR = number of successful flow evaluations/total number of evaluations.

The relevant stream hit rate (Related Flow Hit Rate, RFHR) may be an average of the hit rates of the streams associated with the stream. For example, two streams may be considered related streams if they include commonalities in any of the following attributes. These attributes may include source address, destination address, source port, destination port, protocol, etc. For example, RFHR may be calculated as: RFHR = (total hit rate of all relevant flows)/(total number of evaluations of all relevant flows). In some examples, the stream itself may be excluded when computing RFHR.

The Flow Close miss rate (Flow Close MISSED RATE, FCMR) may be the ratio of the number of Close miss rate evaluations of a Flow to the number of total Flow evaluations. Proximity evaluation misses may be defined as flow evaluation failures due to a user-configured threshold number of attributes not matching. For example, if the stream has 5 attributes for evaluation, and the user configures the proximity miss threshold to be 2. Then, if the stream evaluation fails due to 2 (or less) attribute mismatches, the stream evaluation is considered to be a near miss rate evaluation. For example FCMR can be calculated as

FCMR = (number of near miss rate evaluations)/(total number of evaluations).

Flow criticality may be represented by a flag. For example, a flag may be appended to a stream when any attribute involved in the stream is considered part of a critical event and the stream is marked as critical by the user at run-time. For example, when a user observes a suspicious security threat log event, the user may mark all relevant streams as critical streams. When a stream is marked as critical, the weight of the stream may be automatically set to a maximum value. In such an example, the stream evaluator will preferentially fluidize for evaluation.

Using the above referenced analysis, the PATE analysis unit 206 may determine or predict the weight of each stream. The flow processor of the distributed firewall may evaluate the flow based on the weight of the flow. These analyses are performed periodically in the evaluation history and the stream weights are predicted or updated. For example, if the stream hit rate is R1, the relevant stream hit rate is R2 and the near miss rate is R3. The PATE analysis unit 604 may predict or determine the flow weights using a linear regression formula and the personal ratios calculated above:

W1＝a+b(R1)

W2＝a+b(R2)

W3＝a+b(R3)

Wherein a= (Σwi) (Σrix 2) - (Σri) (Σ WiRi)/n (Σrix 2) - (Σri) 2

b＝n(∑WiRi)-(∑Ri)(∑Wi)/n(∑Ri*2)-(∑Ri*)2

The PATE analysis unit 604 may then determine the average of the predicted weights (W1, W2, and W3) as the weights of the stream:

The PATE analysis unit 604 may send the flow ordering to the distributed firewall. The flow processor of the distributed firewall may evaluate the flow based on these dynamically predicted weights.

As shown in fig. 18, the example closed loop system may determine the ranking and provide the ranking back to the firewall module (e.g., DPUs 670A-670B). The distributed firewall may process each flow in turn proportional to the flow class. For example, a firewall may process higher-level flows more frequently than lower-level flows. This may reduce the number of flows that the firewall needs to process for each data packet, which may increase firewall throughput. Based on the above calculated analytical metrics, a weight factor may be calculated for each flow to indicate the priority and importance of the flow. A higher weight factor may indicate a higher priority and higher priority flows should be processed first and/or more frequently than other flows with lower weights. This technique of dynamic priority prediction for flows also enables distributed firewalls to process flows in any order, not just in descending order. This means that the stream processor is able to select the streams in the reverse order of weights. This may allow most failed flows to be evaluated by priority.

Fig. 19 is a flow diagram illustrating an example flow ordering technique in accordance with one or more aspects of the present disclosure. The PATE analysis unit 604 may obtain telemetry data that includes stream processing data associated with the plurality of streams (720). For example, the PATE analysis unit 604 may retrieve or receive telemetry data from the controller 602.

The PATE analysis unit 604 may order the plurality of streams indicated by the telemetry data according to the importance 722. For example, the PATE analysis unit 604 may assign a respective rank to each of the plurality of streams. The respective rank may represent the determined level of relative importance of the respective stream.

The PATE analysis unit 604 may send information indicative of the respective ranks at least one of the plurality of streams to at least one of the controller or the network interface card NIC, the NIC including NIC processing circuitry (724). For example, the PATE analysis unit 604 may send the controller 602 or the smart NIC 606A a respective ranking of one or more of the plurality of streams.

In some examples, as part of ranking the plurality of streams indicated by the telemetry data according to importance, the PATE analysis unit 604 may determine an importance criterion for each respective stream of the plurality of streams, the importance criterion including at least one of: a flow hit rate, wherein the flow hit rate comprises a number of successful firewall flow evaluations for a respective flow in a total number of evaluations for the respective flow during a time period; a related flow hit rate, wherein the related flow hit rate comprises an indication of a number of successful firewall flow evaluations for one or more related flows of the respective flow in a total number of evaluations for the one or more related flows during the time period; a flow approach miss rate, wherein the flow approach miss rate includes an indication of a number of failed firewall flow evaluations for a respective flow that meets an attribute threshold in a total number of evaluations for the respective flow during the time period; or flow criticality.

In some examples, as part of ordering the plurality of streams indicated by the telemetry data based on importance, the PATE analysis unit 604 may determine the respective importance of each respective stream based on an importance criterion associated with the respective stream. In some examples, each of the one or more related streams includes commonalities in at least one attribute with the respective stream. In some examples, the at least one attribute includes at least one of: source address, destination address, source port number, destination port number, or protocol.

In some examples, as part of determining that a flow is approaching a miss rate, the PATE analysis unit 604 may determine a first number of attributes of the respective flow that failed during a first firewall flow evaluation of the respective flow; determining that the first number of attributes meets an attribute threshold; and classifying a first firewall flow assessment of the respective flow as a proximity miss based on a first number of attributes meeting an attribute threshold; determining a second number of attributes of the respective flow that failed during a second firewall flow assessment of the respective flow; determining that the second number of attributes does not satisfy the attribute threshold; based on a second number of attributes that do not meet the attribute threshold, a second firewall flow assessment for the respective flow is classified as a near miss. In some examples, the attribute threshold is user definable.

In some examples, the stream criticality is indicated by a user selectable flag that indicates a highest importance or a non-highest importance. In some examples, the indication of the highest importance causes the network system (e.g., the PATE analysis unit 604) to rank the respective flows as the highest importance, and wherein the indication of the non-highest importance causes the network system to utilize other importance criteria to determine the importance of the respective flows.

In some examples, the PATE analysis unit 604 may execute a machine learning model to rank the plurality of streams indicated by the telemetry data based on importance. In some examples, the smart NIC 606A may process one or more of the plurality of flows according to the ranking.

Fig. 20 is a flow diagram illustrating an example packet processing technique of a firewall service in accordance with one or more aspects of the present disclosure. Fig. 19 shows a packet processing pipeline for firewall services. When a packet arrives at a firewall (e.g., a distributed firewall such as may be implemented on DPUs 710A-710B), the firewall may construct a flow ID (e.g., SIP, DIP, SPORT, DPORT, PROTO) from the packet header of packet 730. The firewall may search this flow ID in the flow table to determine if this is an existing flow or a new flow 732. If the flow is an existing flow (the "yes" path from block 732), the packet may stay in the data path and the firewall may perform the actions presented in flow table 734. For example, the firewall may apply policies based on the flow table. If the flow is a new flow (the "no" path from block 732), the firewall may send packets to the control plane, which may analyze the configured policies (which may be thousands) to identify actions that the firewall should take for all future packets in this flow. Such control plane lookup may be time consuming and may block or delay forwarding of all packets in the flow until the lookup is complete. For example, if the flow is a new flow (the "no" path from block 732), the firewall may determine whether the new flow is currently being processed by the firewall's control plane 736. If the new flow is not currently being processed by the control plane (the "no" path from block 736), the control plane may look up the configured policies, analyze the configured policies, determine the actions that the firewall should take in this flow for all future packets, and create an entry in the flow table indicating which action should be taken on the packets of the new flow (738). . If the new flow is currently being processed by the control platform (the "yes" path from block 736), the control plane may determine if the buffer is full (740). The buffer may be used to store packets of a new flow waiting to be processed to avoid dropping such packets. If the buffer is not full ("no" path from block 740), the control plane may store the packet in the buffer until an entry for the flow is completed in the flow table (742). However, if the buffer is full ("yes" path from block 740), the packet will be discarded 744. In conventional firewalls, packet buffers are limited because the system must support thousands of active flows, which may result in a large number (in some cases, most) of packets being dropped when a flow table entry in the flow table is created.

In a typical enterprise data center, most applications follow a predictable pattern. For example, an administrator typically schedules backups in the middle of the night when most resources are idle. Using continuous monitoring and machine learning by the PATE analysis unit 604, the PATE analysis unit 604 may learn the behavior (e.g., patterns) of the application. Accordingly, the firewall can create a flow entry on the backup service in advance, which can increase the speed of backup.

Fig. 21 is a flow diagram illustrating an example policy table filling technique in accordance with one or more aspects of the present disclosure. The PATE analysis unit 604 may obtain telemetry data that includes an indication 750 of creation of an instance of the stream. For example, the PATE analysis unit 604 may retrieve or receive telemetry data from the controller 602.

The PATE analysis unit 604 may determine the pattern 752 of stream instance creation based on an indication of creation of the instance of the stream. For example, an instance of a stream has a schema for its creation, e.g., created periodically at a particular time and/or day of the week.

The PATE analysis unit 604 may generate an action entry in a policy table for the particular instance of the flow before receiving the first packet of the particular instance of the flow based on the pattern of creation of the instance of the flow (754). For example, the PATE analysis unit 604 may create a particular instance of the predicted stream based on a pattern of creation of multiple instances of the stream and generate an action entry based on the prediction.

In some examples, creation of an instance of a stream occurs before creation of a particular instance of the stream. In some examples, the PATE analysis unit 604 may execute a machine learning model (e.g., machine learning model(s) 645) as part of determining the pattern of creation of the instance of the stream. In some examples, as part of generating the action entries, the PATE analysis unit 604 may execute a machine learning model (e.g., machine learning model(s) 645). In some examples, the machine learning model is an unsupervised machine learning model. In some examples, the machine learning model is trained using a plurality of created respective indications of instances of the plurality of streams.

In some examples, the smart NIC 606A may receive a first packet of a particular instance of a flow. The smart NIC 606A may determine an action based on an action entry in the policy table and perform the action on the first packet of the particular instance of the flow. In some examples, the action entry relates to a backup service.

Fig. 22 is a block diagram illustrating an example firewall policy optimizer in accordance with one or more aspects of the disclosure. Firewall software is generally divided into three main components, a management plane 760, a control plane 762 (described above), and a data plane 764. In some examples, management plane 760 may be located in a distributed firewall controller, such as in controller 602. In a distributed firewall, whenever a new instance of the firewall is created, the management plane 760 may send all user-configured policies to this new instance of the firewall. In some complex deployments, these strategies can be in the thousands of ranges. As described above, control plane 762 on local host 766 may apply all of these policies to each first data packet of a flow to create a forwarding table.

For each first packet of the new flow, all policies are applied to the first packet, and the result of the application of the policies is used to update the flow table. This operation can be expensive and consumes many CPU cycles, which limits the scalability of the firewall, as firewalls can typically handle thousands of flows per second. Although management plane 760 typically sends all configured policies to control plane 762, control plane 762 may only need policies used by applications (current or future) running on that particular host, such as host 766. Thus, the techniques of this disclosure include using dynamic telemetry data, a closed loop PATE machine learning unit (e.g., one or more machine learning models 645) of the PATE analysis unit 604 may learn the firewall policies used by each application, and which applications are running or expected to run on a particular host. Such information may be obtained by management plane 760. For example, the PATE analysis unit 604 may send such information to the management plane 760. Management plane 760 may use this information to prune the policy set that management plane 760 may send to a given host based on the current and future needs of the host's determination and/or prediction.

For example, if the PATE analysis unit 604 determines that a particular policy is predicted to be used for a given host, the management plane 760 may include that policy in the firewall policy set sent to the control plane of that host. If the PATE analysis unit 604 determines that a particular policy is predicted not to be used for a given host, the management plane 760 may remove that policy from the firewall policy set sent to the control plane of that host. In this way, only policy predictions used by the host may be sent to the host. Because the host may receive only a subset of policies, the control plane of the host may apply only the reduced set of policies to the first packet of any flow, thereby reducing the CPU cycles required for the firewall instance.

For example, when a packet is received by the control plane 762, the control plane 762 may determine whether the packet is the first packet of a flow or a packet of an existing flow. If the packet is the first packet of a flow, the control plane 762 may perform a flow table lookup and apply all policies applicable to the flow. If the packet is a packet of an existing flow or already has policies applied, the control plane 762 may perform a flow table lookup to determine how to process the packet and then provide the packet to the forwarding unit.

Fig. 23 is a flow diagram illustrating an example technique for reducing the number of firewall policies distributed to nodes of a distributed firewall in accordance with one or more aspects of the disclosure. The PATE analysis unit 604 may obtain telemetry data associated with a plurality of applications running on a plurality of hosts (780). For example, the PATE analysis unit 604 may retrieve or receive telemetry data from the controller 602. As used herein, a plurality of applications running on a plurality of hosts means that an application of the plurality of applications may run on one or more hosts of the plurality of hosts without necessarily requiring that each application run on each host or that all applications run on each host.

The PATE analysis unit 604 may determine an application subset of the plurality of applications running on a first host of the plurality of hosts based on the telemetry data (782). For example, the PATE analysis unit 604 may determine that only portions of the plurality of applications are actually operating normally on a particular host (e.g., host 766 of FIG. 22).

The PATE analysis unit 604 may determine a firewall policy subset of the plurality of firewall policies, each firewall policy in the firewall policy subset being applied to at least one corresponding application of the application subset (784). For example, the PATE analysis unit 604 may determine which firewall policies are actually applied to each of the applications in the subset of applications and determine the subset of firewall policies such that each firewall policy in the subset of firewall policies is applied to at least one application in the subset of applications.

The PATE analysis unit 604 may generate an indication of the subset of firewall policies (786). The PATE analysis unit 604 may send the indication to the management plane of the distributed firewall (788). For example, the PATE analysis unit 604 may send an indication to a firewall controller (e.g., controller 602) identifying a subset of firewall policies.

In some examples, the PATE analysis unit 604 may execute a machine learning model to determine at least one of: which applications run on the host, or which firewall policies apply to the determined applications. In some examples, the machine learning model is an unsupervised machine learning model.

In some examples, the host includes a network interface card, NIC, that includes NIC processing circuitry. In some examples, an instance of the control plane and the data plane of the distributed firewall run on the host.

In some examples, the management plane may receive the indication. In some examples, the management plane may prune firewall policy sets corresponding to the plurality of firewall policies based on the indication to generate a pruned firewall policy set corresponding to a subset of the firewall policies. In some examples, the management plane may send the pruned firewall policy set to a control plane executing an instance of the distributed firewall on the host.

In some examples, the smart NIC 606A may apply only the pruned set of firewall policies to the first packet of the new flow. In some examples, as part of only applying the pruned firewall policy set to the first packet of the new flow, the smart NIC 606A may apply each policy of the pruned firewall policy set to the first packet of the new flow, while avoiding applying policies of the firewall policy set that are not part of the pruned firewall policy set.

As discussed herein, implementing different network services on a smart NIC using a closed loop framework may effectively identify and respond to network attacks. The PATE analysis unit 604 may detect anomalies and topology changes, which may be used by the controller action unit 612 to dynamically control operations of the distributed firewall, which may be implemented on the smart NIC to repair security threats. In addition, additional use cases are presented in which the framework can be used for SLA improvement and security firewall scalability and optimization.

Fig. 24 is a conceptual diagram illustrating an example egress traffic-based attack in accordance with one or more aspects of the present disclosure. Network security typically emphasizes traffic entering a cluster ("ingress traffic") when micro-service based applications are deployed in Kubernetes or service grid based environments. Sometimes, when a zero trust policy is favored, network security may be required to monitor and protect cluster internal traffic. But when an application is compromised, e.g., in a bug (bug), malicious supply chain, or DevOps pipeline, the attacker can use the compromised application service or attack any external servers, or download malware from the attacker's server(s). Thus, in order to deter attacks based on the egress traffic, it is necessary to monitor and protect the egress traffic.

For example, vulnerabilities in security code that may be executed within the clustered network 792 of the data center may exist. An attacker may take advantage of this vulnerability to gain access and compromise the audit service 790 within the clustered network 792. For example, the attacker may then log in to the attacker's host server 794 from the review service 790 via the exit traffic to download malware to the clustered network 792.

As a basic step in protecting egress network traffic, kubernetes provides cluster network policies to help cluster administrators configure static network policies (e.g., policies remain unchanged in place until altered or deleted) to control access to external servers by application services. For example, a plug-in for Kubernetes may provide support for clustered network policies, as well as clustered networks. However, this network policy approach has certain limitations. For example, such an approach may be IP address based and may not support domain names in the clustered network policies, may not be able to handle high bandwidth traffic, may consume significant computing resources of clustered nodes for filtering traffic, and/or may negatively impact traffic bandwidth when expanding application services or network policies.

When applications are deployed in a service grid, egress traffic may be monitored and controlled using a reverse proxy (e.g., an "egress gateway"). The egress gateway may run at the edge of the cluster. Application services that may wish to communicate with an external server may be configured to reach the external server only through the egress gateway. However, this egress gateway approach may also have limitations. For example, such an egress gateway approach may be bypassed because the application service may bypass the egress gateway by establishing a direct external network connection, may add additional network hops in the external service communication, may concentrate external traffic and cause single point failures, may work only with TLS and HTTP traffic, may cause scalability and/or availability issues for egress gateway instance management, may consume computing resources of the cluster node, and/or may not handle high bandwidth traffic well.

Thus, in accordance with the techniques of this disclosure, a self-learning egress traffic controller is disclosed. For example, a distributed firewall technique based on a self-learning knowledge graph may detect malicious egress connections and mitigate attacks caused by the malicious egress connections. These techniques may use a distributed firewall to monitor external traffic and mitigate any possible egress traffic-based attacks at the cluster node level. In some examples, a distributed firewall may run on one or more intelligent NICs attached to each node of the cluster. For example, each node in the cluster may be attached to a smart NIC, and traffic for each node in the cluster may pass through the smart NIC.

The smart NIC attached to each cluster node may be managed by a component, such as a smart NIC controller. The smart NIC controller may orchestrate distributed firewalls running over the smart NICs. For example, the smart NIC controller may learn the application topology using past network communication metric data for applications running on the cluster nodes. The learned application topology may help the smart NIC controller identify malicious or compromised services that may make egress connections to launch an egress traffic-based attack, such as a malware attack.

Fig. 25 is a conceptual diagram illustrating an example self-learning egress traffic controller according to one or more aspects of the present disclosure. As shown in fig. 25, these techniques utilize a telemetry data deriver, such as a Node network metric deriver (Node Network Metrics Exporter, NNME) 802 on a cluster Node, such as cluster Node-1 804, and their corresponding smart NICs, such as smart NIC-1 806. When a service establishes a network connection, a clustered network provider, such as clustered network provider 808, may perform network address translation (Network Address Translation, NAT) and also route the connection according to the destination address. Typically, in a Kubernetes environment, a clustered network provider may perform Source NAT (SNAT) when the destination address does not belong to a clustered network address range. In this case, the distributed firewall running on the smart NIC may not be able to obtain the actual source IP address of the service that originated the connection. To obtain the actual source address, the clustered network provider may derive network connection metrics including connection attributes such as source IP address and port number, destination address and port, and so on. However, when SNAT is disabled at the clustered network provider, the derivation of the metrics is also disabled at the clustered network provider.

Accordingly, techniques of the present disclosure may include the following metric deriver. One solution involves one or more cluster node network metric exporters (e.g., node Network Metric Exporter (NNME) 802) and one or more firewall metric exporters (e.g., firewall metric exporter (FIREWALL METRIC Exporter, FME) 810) to provide input data for training a machine learning model (e.g., machine learning model 645).

In some examples, a cluster node network metric deriver (e.g., node network metric deriver 802) may run on each cluster node to derive a connection metric provided by a cluster network provider (e.g., cluster network provider 808). The node network metric deriver 802 derives the metrics with connection details mainly as labels for each connection originating from the workload (which may be a container or pod) running on the cluster node. Example labels include source and destination IP addresses, source and destination port numbers, source workload names (which may include pod or container names), connection protocols and directions, cluster node names and Identifications (IDs), and the like.

A firewall metric deriver (e.g., firewall metric deriver 810) may run on the smart NIC (e.g., smart NIC (SN) 806) of each cluster node and derive the connection metrics provided by the distributed firewall (which may include firewall 812). The firewall metric deriver 810 primarily derives metrics with connection attributes as metric tags. Example labels include source and destination IP addresses, source and destination port numbers, directions, etc.

A metric collector, such as telemetry metric collector 814, may collect metrics derived by node network metric deriver 802 and firewall metric deriver 810. Telemetry metric collector 814 may run within smart NIC controller 816 or somewhere within or outside data center network 800. The smart NIC controller 816 may be an example of the controller 602 of fig. 6. In some examples, telemetry metric collector 814 may include an event queue, for example, when real-time metric data is needed. Telemetry metric collector 814 may feed back the collected metric data to a Topology analysis service, such as Topology analysis and learning engine (Topology ANALYSIS AND LEARNING ENGINE, TALE) unit 820. The TALE unit 820 may be an example of the topology analysis service 605 of the PATE analysis unit 604 of fig. 6.

TALE unit 820 may analyze the network connection metric data and construct a knowledge graph representing the communication topology of all services of the application. TALE unit 820 may include a machine learning model (similar to machine learning model 645 of fig. 10) that may be trained using historical communication data of an application. TALE unit 820 may analyze the historical metric data acquired by TALE unit 820 and construct a knowledge graph based on the historical metric data. In some examples, TALE unit 820 may be configured to construct a knowledge graph using dynamic communication data of an application.

In some examples, when a network connection is established by an application service and metrics associated with the network connection are derived, TALE unit 820 may analyze the metric tag data and begin building a knowledge graph. TALE unit 820 may map source workload names to knowledge graph nodes and may map any network connections from the workload to graph edges. For example, { Source workload name } to knowledge graph node, and { Source IP Address, destination IP Address } to knowledge graph edge service.

Fig. 26 is a conceptual diagram illustrating an example application knowledge graph prior to an exit attack in accordance with one or more aspects of the present disclosure. FIG. 26 depicts a knowledge graph 830 constructed by TALE unit 820 using historical data for a simple e-commerce application. Knowledge graph 830 shows that only billing service 832 establishes (e.g., is allowed to establish) an egress connection, such as an egress connection to payment gateway 834. TALE unit 820 may continuously analyze real-time metric data and update knowledge graph 830 of the application. But when the application service becomes compromised, for example due to a bug in the application code, an error in the supply chain, or a compromised DevOp pipeline, the application may attempt to dial the application's host server or attacker's server to download the malicious code or malware. In addition, the application may attempt to steal other external services by initiating an egress connection to the other external service.

When a compromised service establishes malicious egress connections, TALE unit 820 may receive metric data regarding these new connections. Using the received metric data, TALE unit 820 can update knowledge graph 830 with new graph nodes and edges. When knowledge graph 830 is updated, TALE unit 820 may analyze the updated knowledge graph to determine any changes in the topology. When knowledge graph 830 changes, TALE unit 820 may generate network anomalies regarding the details of any new egress connection.

FIG. 27 is a conceptual diagram illustrating an example updated knowledge graph, according to one or more aspects of the present disclosure. For example, TALE unit 820 may update knowledge graph 830 to generate updated knowledge graph 840. As shown in fig. 27, when the censoring service 850 of the e-commerce application is compromised, an attacker may use the censoring service 850 to initiate an egress connection to the attacker's server 852 and external payment gateway service 854. TALE unit 820 identifies these new egress connections as abnormal connections because TALE unit 820 has learned that only accounting service 856 has made an egress connection.

When TALE unit 820 generates network anomaly events for the new egress connection(s), action services (e.g., action unit 818) running within intelligent NIC controller 816 receive these events. The action service may handle the exception event and configure the firewall policy to mitigate the attack, for example, by stopping the traffic of all compromised services (e.g., audit service 850).

Fig. 28 is a timing diagram illustrating an example technique for configuring firewall policies to mitigate exit attacks in accordance with one or more aspects of the disclosure. For example, a compromised service (e.g., audit service 850) may establish an egress connection 860 to clustered network provider 808. The clustered network provider 808 may provide connection data 862 to the node network metric exporter 802 and route the connection 864 to a distributed firewall, which may include a firewall 812. The firewall 812 may provide the connection data 866 to the firewall metric deriver 810. Telemetry metric collector 814 may collect node network metrics 868 from node network metric deriver 802 and collect firewall connection metrics 870 from firewall metric deriver 810. Although shown as originating from telemetry metric collector 814, in some examples telemetry metric collector 814 may passively receive metrics from node network metric deriver 802 and/or firewall metric deriver 810. Telemetry metric collector 814 may feed back metric data 872 to TALE unit 820. TALE unit 820 may construct knowledge graph 874, e.g., updated knowledge graph 840.TALE unit 820 may analyze knowledge graph 876 and notify action services of exception event 878 (e.g., action unit 818 of fig. 25). The action service may create (e.g., generate) a new firewall policy 880 or modify an existing firewall policy. The action service may apply new firewall policies or modified firewall policies 882 to the distributed firewall (e.g., firewall 812) to mitigate the exit attack.

Fig. 29 is a flow diagram illustrating an example technique for configuring firewall policies to mitigate exit attacks in accordance with one or more aspects of the disclosure. For example, a compromised service (e.g., audit service 850 of fig. 27) may establish an egress connection (900) to an aggressor's server 852. Telemetry metric collector 814 may collect metrics from node network metric deriver 802 and/or firewall metric deriver 810. Telemetry metric collector 814 may derive network metrics to TALE unit 820 (902). TALE unit 820 may analyze metric data (904). TALE unit 820 may verify the egress connection (906). For example, TALE unit 820 may determine whether an egress connection is allowed based on an existing knowledge graph. TALE unit 820 may determine whether the connection is an abnormal connection (908). For example, TALE unit 820 may determine whether a connection is allowed based on an existing knowledge graph. If TALE unit 820 determines that the connection is not an abnormal connection (the "NO" path from block 908), TALE unit 820 may ignore the connection (910). If TALE unit 820 determines that the connection is an abnormal connection ("Yes" path from block 908), TALE unit 820 may report an abnormal event to action unit 818 (912). Action unit 818 may handle the exception and create a firewall policy (914). Action unit 818 may apply firewall policies (916) to the distributed firewalls that may use the firewall policies.

FIG. 30 is a flow diagram illustrating an example technique for responding to a malicious egress connection in accordance with one or more aspects of the present disclosure. The technique of fig. 30 is discussed as being performed by different portions of the system with respect to fig. 6. Each portion of the system may perform those techniques that belong to the respective portion, or in some examples, one or more techniques that belong to any one portion of the system may be performed by other portions of the system or a combination of portions of the system.

The smart NIC 606A may configure 920 the egress connection of the application service from the application. For example, the smart NIC 606A may establish a connection from the auditing service 850 to the payment gateway 854 (fig. 27).

The smart NIC 606A may send connection data regarding the egress connection to the computing device (924). For example, the smart NIC 606A may generate, record, and send connection data to the controller 602, and the controller 602 may be a smart NIC controller, such as the smart NIC controller 816 of fig. 25.

The controller 602 may receive connection data related to an egress connection of an application service of an application (926). For example, the controller 602 may receive connection data from the smart NIC 606A. The controller 602 may send the connection data to the computing device (928). For example, the controller 602 may send connection data to the PATE analysis unit 604. In some examples, the controller 602 may process the connection data before sending the connection data to the PATE analysis unit 604.

The PATE analysis unit 604 may obtain connection data related to the egress connection of the application service of the application (930). For example, the PATE analysis unit 604 may retrieve or receive connection data from the controller 602.

The PATE analysis unit 604 may analyze the connection data to determine that the egress connection is an abnormal connection (932). For example, the PATE analysis unit 604 may determine that the egress connection is not an expected egress connection and is therefore anomalous.

The PATE analysis unit 604 may generate a notification indicating that the egress connection is an abnormal connection (934). The PATE analysis unit 604 may send a notification to the computing device (936). For example, the PATE analysis unit 604 may send a notification to the controller 602.

The controller 602 may receive a notification from the computing device indicating that the egress connection is an abnormal connection in response to sending the connection data (938). For example, the controller 602 may receive a notification from the PATE analysis unit 604 indicating that the egress connection is an abnormal connection.

The controller 602 may generate a notification to apply the firewall policy based on the notification indicating that the egress connection is abnormal (940). For example, the controller 602 may determine that firewall policies should be applied by the smart NIC 606A to resolve the abnormal connection and may generate a notification therefrom.

The controller 602 may send an application firewall policy notification to at least one network interface card (942). For example, the controller 602 may send a notification to the smart NIC 606A to apply the firewall policy.

The smart NIC 606A may receive a notification from the computing device to apply the firewall policy in response to sending the connection data (944). For example, the smart NIC 606A may receive a notification from the controller 602 to apply a firewall policy. The smart NIC 606A may apply a firewall policy (946). For example, the smart NIC 606A may extract the firewall policy from the notification and execute the firewall policy, or may determine an identification of the firewall policy based on the notification, load the firewall policy from memory, and execute the firewall policy.

In some examples, the PATE analysis unit 604 may analyze the connection data via a machine learning model (e.g., machine learning model 645). In some examples, the machine learning model is trained using previous connection data of the application.

In some examples, the PATE analysis unit 604 may generate a prior knowledge graph (e.g., the knowledge graph 830 of FIG. 26) based on the prior connection data of the application. In some examples, the previous knowledge graph indicates each application service of the application for which the egress connection has been previously made. In some examples, as part of analyzing the connection data, the PATE analysis unit 604 may generate a first knowledge graph (e.g., knowledge graph 840 of FIG. 27) indicating the application service making the egress connection based on the connection data. The PATE analysis unit 604 may compare the first knowledge graph with the previous knowledge graph.

In some examples, the connection data includes node network metrics and firewall metrics. In some examples, at least a portion of the node network metrics are associated with the cluster nodes, and wherein the node network metrics include at least one of: a source IP address, a destination IP address, a source port number, a destination port number, a source workload name, a connection protocol, and a direction of an egress connection, or a cluster node identifier. In some examples, at least a portion of the firewall metrics are associated with instances of distributed firewalls running on the NIC, and wherein the firewall metrics include at least one of: source IP address, destination IP address, source port number, destination port number, or direction of egress connection.

In some examples, the notification indicating that the egress connection is an abnormal connection includes information associated with the egress connection.

In some examples, the firewall policy is a new firewall policy. In some examples, the notification to apply the firewall policy includes the new firewall policy.

In some examples, as part of applying the firewall policy, the smart NIC 606A may at least one of: drop the egress connection or block further egress connections from the application service. In some examples, controller 602 may generate a new firewall policy as part of generating a notification to apply the firewall policy. In some examples, the new firewall policy is configured to cause the at least one network interface card to at least one of: drop the egress connection or block further egress connections from the application service.

The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. The various features described as modules, units, or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices or other hardware devices. In some cases, various features of the electronic circuit may be implemented as one or more integrated circuit devices, such as an integrated circuit chip or chip set.

If implemented in hardware, the present disclosure may be directed to an apparatus, such as a processor or integrated circuit device, such as an integrated circuit chip or chipset. Alternatively or additionally, the techniques may be implemented, at least in part, by a computer-readable data storage medium comprising instructions that, when executed, cause a processor to perform one or more of the methods described above. For example, a computer-readable data storage medium may store such instructions for execution by a processor.

The computer readable medium may form part of a computer program product, which may include packaging material. The computer readable medium may include a computer data storage medium such as Random Access Memory (RAM), read Only Memory (ROM), non-volatile random access Memory (Non-Volatile Random Access Memory, NVRAM), electrically Erasable Programmable Read Only Memory (EEPROM), flash Memory, magnetic or optical data storage media, and the like. In some examples, an article of manufacture may comprise one or more computer-readable storage media.

In some examples, the computer-readable storage medium may include a non-transitory medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagated signal. In some examples, a non-transitory storage medium may store data (e.g., in RAM or cache) that is capable of changing over time.

The code or instructions may be software and/or firmware executed by a processing circuit comprising one or more processors, such as one or more digital signal processors (DIGITAL SIGNAL processors, DSPs), general purpose microprocessors, application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. Furthermore, in some aspects, the functionality described in this disclosure may be provided within software modules or hardware modules.

Claims

1. A network system, comprising:

A processing circuit; and

One or more memories coupled to the processing circuitry and configured to store instructions that, when executed by the processing circuitry, cause the network system to:

Acquiring first service session measurement data;

executing a machine learning model to determine a business prediction based on the first business session metric data;

acquiring second service session measurement data;

Determining anomalies in traffic based on a comparison of the traffic prediction and the second traffic session metric data; and

Based on the determination of the anomaly, an indication of the anomaly is generated.

2. The network system of claim 1, wherein the instructions further cause the network system to send the indication of the anomaly to a controller of a distributed firewall.

3. The network system of claim 1, wherein the anomaly is indicative of a domain name service, DNS, attack or transmission control protocol, TCP, flooding attack.

4. The network system of claim 3, wherein the second traffic session metric data comprises a number of domain name service requests of a virtual network endpoint of a host device over a period of time, wherein the traffic forecast comprises an expected number of expected domain name service requests during the period of time, and wherein as part of determining the anomaly, the instructions cause the processing circuitry to determine that the number of domain name service requests of the virtual network endpoint of the host device over the period of time is greater than the number of expected domain name service requests.

5. The network system of claim 1, wherein the machine learning model comprises an unsupervised random cut forest machine learning model.

6. The network system of any of claims 1 to 5, wherein the first traffic session metric data and the second traffic session metric data are indicative of a service grid traffic.

7. The network system of any of claims 1-5, wherein the first traffic session metric data and the second traffic session metric data are associated with one or more network interface card NICs.

8. The network system of claim 7, wherein the one or more NICs implement a distributed firewall.

9. The network system of any of claims 1-5, wherein the instructions further cause the network system to generate or select a firewall policy based on the indication of the anomaly.

10. The system network of claim 9, wherein the instructions further cause the network system to:

identifying a NIC of the one or more NICs associated with the host device; and

And sending the firewall policy or the identification of the firewall policy to the NIC.

11. A network system, comprising:

A processing circuit; and

monitoring service grid traffic leaving each of a plurality of network interface cards, NICs, each of the NICs including NIC processing circuitry;

monitoring service grid business to enter each NIC;

Determining traffic session metric data based on the mesh traffic exiting each NIC and the mesh traffic entering each NIC;

Receiving an indication of an anomaly in a business from a machine learning model based on the business session metric data;

Determining a firewall policy based on the indication of the anomaly; and

An indication of a firewall policy is sent to at least one NIC of the one or more NICs.

12. The network system of claim 11, wherein the indication of the firewall policy comprises at least one of the firewall policy or an identification of the firewall policy.

13. The network system of claim 11, wherein the instructions further cause the network system to store the traffic session metric data in a time series database.

14. The network system of any of claims 11 to 13, wherein the traffic session metric data comprises a number of domain name service requests of a virtual network endpoint of a host device over a period of time, wherein the anomaly indicates that the number of domain name service requests of the virtual network endpoint of the host device over the period of time is greater than a number of expected domain name service requests determined by the machine learning model.

15. The network system of any of claims 11 to 13, wherein to determine the firewall policy, the instructions cause the network system to select the firewall policy or generate the firewall policy.

16. The network system of any of claims 11 to 13, wherein the one or more NICs implement a distributed firewall.

17. A network system, comprising:

A network interface card, NIC, the NIC comprising processing circuitry; and

transmitting telemetry data to a controller, the telemetry data comprising traffic session metric data;

receiving an indication of a firewall policy from the controller based on a determination of anomalies in traffic, the anomalies in traffic being determined based on the traffic session metric data; and

And realizing the firewall policy.

18. The network system of claim 17, wherein the traffic session metric data comprises a number of domain name service requests of virtual network endpoints of a host device over a period of time, and wherein the anomaly is determined by a machine learning model.

19. The network system of claim 18, wherein implementing the firewall policy causes the NIC to terminate sending domain name service requests from the virtual endpoint of the host device.

20. The network system of any of claims 17 to 19, wherein the NIC implements an instance of a distributed firewall.

21. A network method performed by the network system of any one of claims 1 to 20.

22. A computer-readable storage medium encoded with instructions for causing one or more programmable processors to perform the method of claim 21.