WO2017058137A1

WO2017058137A1 - Latency tracking metadata for a network switch data packet

Info

Publication number: WO2017058137A1
Application number: PCT/US2015/052561
Authority: WO
Inventors: Claudio Enrique Viquez; Diego VALVERDE; Jose Daniel HERNANDEZ; Osvaldo Andres SANCHEZ
Original assignee: Hewlett Packard Enterprise Development Lp
Priority date: 2015-09-28
Filing date: 2015-09-28
Publication date: 2017-04-06

Abstract

In some examples, a network switch includes a first Application-Specific Integrated Circuit (ASIC) processing resource and an ASIC memory resource storing machine readable instructions. The instructions are to cause the processing resource to: (1) remove, with the first ASIC, latency tracking metadata from the data packet that identifies internal switch state and link latency information of an upstream network switch; (2) build, with the first ASIC, a network configuration database based on the removed latency tracking metadata; (3) forward the data packet without the latency tracking metadata to a downstream node; and (4) forward the network configuration database to an external computer for analysis of the network.

Description

LATENCY TRACKING METADATA FOR A NETWORK SWITCH DATA PACKET

BACKGROUN D

[0001] Computer networks can be used to allow networked devices, such as personal computers, servers, and data storage devices to exchange data. Computer networks often include intermediary datapath devices such as network switches, gateways, and routers, to flow traffic along selected datapaths for routing data between networked devices.

BRIEF DESCRIPTION OF DRAWINGS

[0002] FIG. 1 is a diagram of a network, according to an example.

[0003] FIG. 2 is a flowchart for a method, according to an example.

[0004] FIG. 3 is a flowchart for a method, according to another example.

[0005] FIG. 4 is a flowchart for a method, according to another example.

[0006] FIG. 5 is a diagram of network switch, according to an example.

[0007] FIG. 6 is a diagram of machine-readable storage medium, according to an example.

DETAILED DESCRIPTION

[0008] The following discussion is directed to various examples of the disclosure. Although one or more of these examples may be preferred, the examples disclosed herein should not be interpreted, or otherwise used, as limiting the scope of the disclosure, including the claims. In addition, the following description has broad application, and the discussion of any example is meant only to be descriptive of that example, and not intended to intimate that the scope of the disclosure, including the claims, is limited to that example. Throughout the present disclosure, the terms "a" and "an" are intended to denote at least one of a particular element. In addition, as used herein, the term "includes" means includes but not limited to, the term "including" means including but not limited to. The term "based on" means based at least in part on.

[0009] Certain implementations of the present disclosure are directed to the preparation, addition, removal, analysis, processing, and/or other use of latency tracking metadata of a network switch data packet. For example, an implementation in the form of a method performed by a network switch in a network can include: (a) receiving, with a first latency metadata tracking programmable Application-Specific Integrated Circuit (ASIC), a data packet; (b) determining, with the first ASIC, whether the received data packet is to be routed to a second network switch in the network that includes a second latency metadata tracking programmable ASIC; and if so, then (c) adding, with the first ASIC, metadata to the data packet that identifies internal switch state and link latency information of the first network switch. Many other implementations relating to the use of latency tracking metadata are also described.

[0010] The use of latency tracking metadata can, in some implementations and situations, allow for a network administrator or other individual, team, software, etc., to quickly debug and resolve certain network issues and/or allow for greater network performance. Certain implementations described herein can, for example, lead to improved topology design, network device configuration, latency between connections, network flow performance issues, etc., by providing a more complete picture of the network compared to other network analysis systems. This metadata can be used by itself or along with other data to build a database and/or graph network information in order to reduce or troubleshoot network issues. For example, by using certain implementations of the present disclosure, a network administrator can receive performance data relating to: (1) how certain links in a network may have reduced performance compared to other links, (2) the path a specific network flow is using, (3) time spent between each of the nodes, and (4) the configuration of each of the nodes of the topology.

[0011] As described in further detail below, the data used by the network administrator can, for example, be provided as latency metadata inserted into a data packet by a latency metadata tracking programmable ASIC of a network switch. The latency tracking metadata can, for example, be made up of a variety of different fields, and the list of fields can be chosen programmatically and can be changed without interruptions to the traffic flow in the switch. Such metadata fields can, for example, include both internal switch state as well as link latency information that can be communicated to one or more CPUs in order to dynamically build a map of the switching network that constantly reflects an estimate of the link latency and/or congestion of the network. In some situations, certain metadata (e.g., certain internal switch state information) may not normally be entirely visible to devices or software external to the network switch. As a result, the use of one or more implementations of the present disclosure can provide greater visibility of underlying switch configurations than existing techniques for network monitoring. The implementations presented herein can include additional and/or alternative advantages, many of which will be apparent upon review of the description and figures.

[0012] FIG. 1 is a diagram of an example network 100 including a first example network switch 102 that houses a switch CPU 104 and a programmable ASIC 106 that has various combined hardware and software modules (such as routing determination module 108, metadata adding module 110, and metadata removing module 112, which are described in further detail below). It is appreciated that the term "ASIC" as used herein can, for example, include related technologies such as application- specific field-programmable gate arrays (FPGAs), which can, for example contain an array of programmable logic blocks, and a hierarchy of reconfigurable interconnects that allow the blocks to be wired together. Suitable ASICs for use with the present disclosure can, for example, allow for logic blocks to be configured to perform complex combinational functions as well as simple logic gates like AND and XOR. Suitable ASICs for use with the present disclosure can, for example, also include memory elements, which may be simple flip-flops or more complete blocks of memory.

[0013] Network 100 further includes a second example network switch 114 also housing a switch CPU 104 and a programmable ASIC 106 having various combined hardware and software modules (such as routing determination module 108, metadata adding module 110, and metadata removing module 112, which are described in further detail below). In order to avoid inconsistency in description, the same reference numerals are used for certain components of switches 102 and 114 (e.g. switch CPU 104). However, it is appreciated that these components are distinct equipment for each switch. For example, although switch CPU 104 of switch 114 may (or may not) be the same model of CPU as switch CPU 104 of switch 102, the two CPU's are different pieces of hardware installed on their respective switches.

[0014] FIG. 1 depicts network traffic along a datapath between an example source node 116 and example destination node 118, the datapath being defined by network nodes 102, 120, 122 and 114. Other network nodes, such as nodes 124 and 126 are included within network 100 but are not used in this datapath. It is appreciated that the datapath can be determined by a network controller 128 (or another entity, such as by a network administrator, by datapath nodes themselves, etc.) based on one or more static parameters (e.g., link speeds, number of hops between nodes, etc.) and can further (or alternatively) be based on one or more dynamic parameters (e.g., Quality of Service (QoS), network latency, network throughput, network power consumption, etc.).

[0015] The various network nodes of network 100 can forward traffic along a datapath based on metadata within the traffic. For example, traffic in the form of a data packet 130 can be received at network switch 102 (or another suitable intermediary network node). In the example network of FIG. 1, packet 130 can include some "original" data (shown as "A" in FIG. 1 for illustration). As described in further detail below, in some implementations, switch 102 can be used to insert latency tracking metadata into packet 130 (shown as "B" in FIG. 1 for illustration) and switch 114 can be used to remove the latency tracking metadata from packet 130 before sending packet 130 to destination node 118.

[0016] Network controller 128 can, for example, be used to send and receive traffic control or other operation instructions or information to one or more nodes in network 100. As described in further detail below, network controller 128 can include one or more combined hardware and software modules (such as metadata tracking module 132, which is described in further detail below). Source node 116 and destination node 118 can, for example, be in the form of network hosts or other types of network nodes. For example, one or both of source node 116 and destination node 118 can be in the form of suitable servers, desktop computers, laptops, printers, etc. For example, source node 116 can be in the form of a desktop computer including a monitor for presenting information to an operator and a keyboard and mouse for receiving input from an operator, and destination node 118 can be in the form of a standalone storage server appliance. It is appreciated that source node 116 and destination node 118 can be endpoint nodes on network 100, intermediate nodes between endpoint nodes, or positioned at other logical or physical locations within network 100.

[0017] The various intermediary nodes (including switches 102 and 114) within network 100 can, for example, be in the form of switches or other multi-port network bridges that process and forward data at the data link layer. In some implementations, one or more of the nodes can be in the form of multilayer switches that operate at multiple layers of the Open Systems Connection (OSI) model (e.g., the data link and network layers). Although the term "network switch" is used throughout this description, it is appreciated that this term can refer broadly to other suitable network data forwarding devices. For example, a general purpose computer can include suitable hardware and machine-readable instructions that allow the computer to function as a network switch. It is appreciated that the term "switch" can include other network datapath elements in the form of suitable routers, gateways and other devices that provide switch-like functionality for network 100.

[0018] In some implementations, a given network switch in a network (e.g., switch 102) can rely on flow rules stored on the switch (or otherwise accessible by the switch) for forwarding or otherwise handling traffic. Flow rules can, for example, contain information such as: (1) match fields to match against packets (e.g., an ingress port and specific packet header fields), (2) a priority value for the flow rule to allow prioritization over other flow entries, (3) counters that are updated when packets are matched, (4) instructions to modify the action set or pipeline processing, (5) timeouts indicating a maximum amount of time or idle time before a flow is expired by the switch, and (6) a cookie value which can be used by a network controller or other entity to filter flow statistics, flow modification, and flow deletion. As described in further detail below, such flow rules can be transmitted to the switch via a network controller, directly by an administrator or other entity via a command line interface (CLI), a graphical user interface (GUI), or through another suitable input. [0019] The various nodes within network 100 are connected via one or more data channels, which can, for example be in the form of data cables or wireless data channels. Although a single link (i.e., a single line in FIG. 1) between each network node is illustrated in FIG. 1, it is appreciated that each single link may include multiple wires or other wired or wireless data channels. Moreover, FIG. 1 further depicts network controller 128 as being connected to each network nodes via broken lines, which is intended to illustrate logical control channels between network controller 128 and respective nodes. However, it is appreciated that network controller 128 may be directly connected to only one or a few network nodes, while being indirectly connected to other nodes of network 100. As but one example, network controller 128 can be directly connected to node 122 via an Ethernet cable, while being indirectly connected to node 124 (e.g., by relying on node 122 as an intermediary for communication with node 124). In such a situation, the control channel can be considered a direct logical channel between network controller 122 and node 124 and is formed by a first physical channel (e.g., a first Ethernet cable) that connects network controller 128 to node 122 and by a second physical channel (e.g., a second Ethernet cable) that connects node 122 to node 124.

[0020] Network 100 can, in some implementations, be implemented as a Software-Defined Network (SDN). Software-defined networking can allow for the decoupling of traffic routing control decisions (e.g., which port of a network switch should be used to forward traffic en route to a given destination) from the network's physical infrastructure. For example, in an SDN, such traffic routing control decisions can be determined by an entity (e.g., a network controller) that is different from the routing device itself (e.g., the network switch tasked with forwarding the traffic). A network controller used in implementing an SDN (i.e., an SDN controller) can, for example, be programmed to: (1) receive dynamic parameters of the network from intermediary datapath devices (e.g., network switches), (2) decide how to route packets over the network, and (3) inform the devices about these decisions. SDN controllers can, for example, be configured to access and control multiple devices within the SDN via a network communication channel. Such a network communication channel can be referred to as a "control channel," an "OpenFlow channel" (for SDN's implemented using the OpenFlow protocol), a "communication channel," an "interface channel," etc. In some networks, a network controller can use such a control channel to configure devices (e.g., configure flows stored on devices), receive data packets, send packets using the device, gather state and statistics from devices, and/or other uses. In some networks, SDN applications can be run on a network controller or on other devices on the network (or otherwise in communication with the network) and interfaced with the network controller to meet customer use cases, such as to achieve a desired throughput (or another QoS) over the SDN, enforce security provisions for the SDN, or provide another suitable service or functionality.

[0021] The functionality of network controller 128 can, for example, be implemented in part via a software program on a standalone machine, such as a standalone server. In some implementations, network controller 128 can be implemented on a multipurpose machine, such as a suitable desktop computer, laptop, tablet, or the like. In some implementations, network controller 128 can be implemented on a suitable non-host network node, such as certain types of network switches. It is appreciated that the functionality of network controller 128 may be split among multiple controllers or other devices. For example, network 100 is described and illustrated as including only one network controller 128. However, it is appreciated that the disclosure herein can be implemented in SDNs or other networks with multiple controllers. For example, in some networks, network devices are in communication with multiple controllers such that control of the network can be smoothly handed over from a first controller to a second controller if a first controller fails or is otherwise out of operation. As another example, multiple controllers can work together to concurrently control certain networks. In such networks, a first controller can, for example, control certain network devices while a second controller can control other network devices. In view of the above, reference in this application to a single network controller 128 that controls the operation of network 100 is intended to include such multiple controller configurations (and other suitable multiple controller configurations).

[0022] Within the context of an SDN, controlled network nodes (e.g., switch 102) can be used as sensors in the network as they may have information about dynamic network parameters. When polled via standard SDN interfaces, the devices can report this information to network controller 128. Network 100 can, for example, be implemented through the use of network controller 128 that interfaces with various SDN-compatible devices via a suitable Application Program Interface ("API") or a suitable SDN protocol (e.g., OpenFlow) or other protocol.

[0023] As used herein, the term "controlled" and similar terminology in the context of SDN- compatible network nodes, such as "controlled switches," is intended to include devices within the control domain of network controller 128 or otherwise controllable by network controller 128. Such a controlled node can, for example, communicate with network controller 128 and network controller 128 is able to manage the node in accordance with an SDN protocol, such as the OpenFlow protocol. For example, an OpenFlow-compatible switch controlled by network controller 128 can permit network controller 128 to add, update, and delete flow entries in flow tables of the switch using suitable SDN commands.

[0024] In the example network 100 depicted in FIG. 1, the various network nodes are in the form of intermediary nodes (e.g., controlled network switches 102 and 114) and host devices (source node 116 and destination node 118). It is appreciated however, that the implementations described herein can be used or adapted for networks including more or fewer devices, different types of devices, and different network arrangements. It is further appreciated that the disclosure herein can apply to suitable SDNs (e.g., certain hybrid or heterogeneous SDNs) in which some devices are controlled by a network controller (e.g., network controller 128) and some devices are not controlled by the network controller (e.g., network controller 128 or any other network controller). For example, in some implementations, at least one node (e.g., node 114) along a given datapath is controlled by network controller 128 and at least one node along the given datapath (node 122) is not controlled by network controller 128.

[0025] FIG. 2 illustrates a flowchart for a method 134 according to an example of the

present disclosure. For illustration, the description of method 134 and its component steps make reference to example network 100 and elements thereof, such as for example network switch 102, etc. However, it is appreciated that method 134 or aspects thereof can be used or otherwise applicable for any suitable network or network element described herein or otherwise. For example, method 134 can be applied to computer networks with different network topologies than those illustrated in FIG. 1.

[0026] In some implementations, method 134 can be implemented or otherwise executed through the use of executable instructions stored on a memory resource (e.g., the memory resource of the network switch of FIG. 5), executable machine readable instructions stored on a storage medium (e.g., the medium of FIG. 6), in the form of electronic circuitry (e.g., on an ASIC), and/or another suitable form. Although the description of method 134 herein primarily refers to steps performed on network switch 102 for purposes of illustration, it is appreciated that in some

implementations, method 134 can be executed on another computing device within network 100 and/or in data communication with network switch 102.

[0027] Method 134 of FIG. 2 includes receiving (at block 136), with a first latency metadata tracking programmable ASIC (e.g., ASIC 106 of switch 102) of a first network switch (e.g., switch 102) in network 100, a data packet 130. Data packet 130 can, for example be received through a physical, virtual, and/or logical port of network switch 102. For consistency, the industry term "packet" is used throughout this description, however, it is appreciated that the term "packet" as used herein can refer to any suitable protocol data unit (PDU). Such a packet can, for example, include payload data as well as metadata in the form of control data. Control data can, for example, provide data to assist network switch 102 with reliably delivering payload data. For example, control data can include network addresses for source node 116 and destination node 118, error detection codes, sequencing information, packet size of the packet, a time-to-live (TTL) value, etc. In contrast, payload data can include data carried on behalf of an application for use by source node 116 and destination node 118.

[0028] Method 134 of FIG. 2 includes determining (at block 138), with ASIC 106, whether packet 130 is to be routed to a second network switch (e.g., switch 114) that includes a second latency metadata tracking programmable ASIC (e.g., ASIC 106 of switch 102). With reference to FIG. 1, network switch 102 and network switch 114 both include latency metadata tracking ASICs as they both include respective routing determination modules 108, metadata adding modules 110, and metadata removing modules 112. Such switches can also be referred to as "compliant" or latency metadata tracking compliant (LMTC) switches in the context of the present disclosure. It is appreciated that such compliant switches may include additional or alternative modules or programming than switch 102 and 114 and the term

"compliant" is not intended to require modules 108, 110, and 112. For example, in some implementations, depending on a switch's position within network 100, it may be deemed "compliant" if is it able to add latency tracking metadata even if it is unable to remove the metadata. Likewise, depending on its position within network 100, it may be deemed "compliant" if is it able to remove latency tracking metadata even if it is unable to add metadata.

[0029] In certain implementations, block 138 can, for example, allow for the use of the present disclosure in networks where not every network switch is programmed to track latency metadata. That is, block 138 can, for example, be used to determine whether there are any downstream network switches programmed to track latency metadata. If there are no downstream network switches (e.g., when the packet is to exit a network of monitored switches), then as described below, any previously added latency tracking metadata can be removed to allow the original payload to reach its destination.

[0030] Although the present disclosure allows for the use of switches that are not

programmed to track latency metadata, such switches may, in some cases, fail to produce useful topology or latency information. In such a case, the latency tracking metadata information will be transparent to switches that do not allow for latency tracking addition or removal, and will instead be interpreted as merely part of the payload of the data packet and can be ignored by such switches.

[0031] Method 134 of FIG. 2 includes adding (at block 140), with ASIC 106 of switch 102, metadata to data packet 130 that identifies internal switch state and link latency information of switch 102 when it is determined that packet 130 is to be routed to a second network switch 114 that includes second latency metadata tracking programmable ASIC 106. As described in further detail below, the added metadata can, in some implementations, be used to build a map of the network configuration based on the metadata or other operations. The added metadata can, for example, be metadata specific to a given ingress and/or egress port of switch 102 or another characteristic of switch 102 or packet 130. For example, in some implementations, the added metadata can include Media Access Control (MAC) addresses for the source node and destination node of the data packet. The added metadata can include other fields of interest from packet 130. In some implementations, network controller 128 can indicate to switch 102 that certain metadata fields of packet 130 are of interest for a specific control task. For example, for some operations, such as an L2 learning/move operation or Deep Packet Inspection (DPI) operation, network controller 128 (or another entity, such as a network administrator, host machine, etc.), may be interested in information about a packet's VLAN, the packet's source and destination MAC, as well as the port of network switch 102 that received the data packet. It is appreciated that the above is just an example of certain fields that may be of interest to a specific operation and that other fields or information may be of interest for the same operations or different operations. For example, in some implementations, payload data of the packet may be of interest to the network controller 128 (e.g., for certain DPI operations). In such an implementation, network controller 128 can identify the payload as being of interest to controller 128 and should be forwarded to controller 128 in the form of metadata added to packet 130. In some implementations, switch 102 can compute a list of nodes directly connected to it. Using this information, switch 102 can communicate its state and approximation of the internal switch latency/delay or link latency/delay, as well as accumulated latency/delay on per packet, per trunk, or per VLAN basis. Packet link/latency/delay determination across link can, for example, be calculated via switch 102. Moreover, ASIC 106 or another component of switch 102 can provide a per-packet time-stamp value, and in some switches, can further allow for programming ASIC 106 or another component of switch 102 to act as a "chronometer," that can register values on per packet, per port, per VLAN, only packets matching certain criteria (header fields with specific values that can be chosen programmatically), time of ingress, time of egress, etc. In some implementations, ASIC 106 or another component of switch 102 can incorporate custom and granular latency/delay parameters. Furthermore, the choosing of the parameters can be changed and tailored based of network state or user choices, and this can be done dynamically without interruption of the traffic.

[0033] In some implementations, an SDN protocol can be used to provide control

instructions to switch 102 and/or programmable ASIC 106. As provided above, network controller 128 can, in some implementations, be in the form of an SDN controller. In such implementations, the SDN controller can prepare and/or send instructions relating to which latency tracking metadata should be added to packet 130 by switch 102. In such an implementation, the latency tracking metadata can, for example, be selected in accordance with an objective of an SDN application running on an SDN controller (e.g., network controller 128 in certain

implementations).

[0034] Internal state information can, for example, include internal states and variables of switch 102, such as switch logs for switch 102, VLAN information, QoS parameters and the packet time stamp as well as internal switch states, such as routing table information stored on switch (e.g., L2 table information, L3 table information, etc.), ACL's, debug information, statistics, counter, meters, etc. Link latency information can, for example, refer to latency caused due to processing of packet 130 through switch 102. In some implementations, link latency information can include historical information for previous packets processed by switch 102 or other nodes of network 100. In some implementations, link latency information can include time stamp information for packet 130 that can be used by switch 102 or another computing device to determine latency information relating to traffic through switch 102.

[0035] In some implementations, ASIC 106 can alter the original packet size and

encapsulations/decapsulations programmatically such that the modified packet with latency tracking metadata is still a valid packet and the bridging/routing information is preserved across the network. As described in further detail below, certain implementations can allow for latency tracking metadata to be inserted into the original packet on packet ingress to a network of programmable ASICs, and once the packet exits the network of programmable ASICs, then the metadata is removed such that original payload reaches its destination.

[0036] Although the flowchart of FIG. 2 shows a specific order of performance, it is

appreciated that this order may be rearranged into another suitable order, may be executed concurrently or with partial concurrence, or a combination thereof.

Likewise, suitable additional and/or comparable steps may be added to method 134 or other methods described herein in order to achieve the same or comparable functionality. In some implementations, one or more steps are omitted. For example, in some implementations, block 140 of adding metadata to packet 130 can be omitted from method 134 (e.g., if there are no downstream switches that are programmed to remove latency tracking metadata). Blocks corresponding to additional or alternative functionality of other implementations described herein can be incorporated in method 134. For example, and as described in further detail below with respect to the method of FIG. 3, in some implementations, method 134 can include removing metadata from packet 130. It is appreciated that blocks corresponding to the functionality of various aspects of switch 102 otherwise described herein can be incorporated in method 134 even if such functionality is not explicitly characterized herein as a block in a method.

[0037] FIG. 3 illustrates another example of method 134 in accordance with the present disclosure. For illustration, FIG. 3 reproduces various blocks from method 134 of FIG. 2, however it is appreciated that method 134 of FIG. 3 can include additional, alternative, or fewer steps, functionality, etc., than method 134 of FIG. 2 and is not intended to be limited by the diagram of FIG. 1 (or vice versa) or the related disclosure thereof. It is further appreciated that method 134 of FIG. 3 can incorporate one or more aspects of method 134 of FIG. 2 and vice versa. For example, in some implementations, method 134 of FIG. 2 can include the additional step described below with respect to method 134 of FIG. 3.

[0038] Method 134 of FIG. 3 includes removing (at block 142), with the first ASIC, metadata from the data packet that identifies internal switch state and link latency information of one or more other network switches when it is determined that the received packet is not to be routed to a second network switch that includes a second latency metadata tracking programmable ASIC. The removing operation of block 142 can, for example, be based on instructions previously communicated to switch 102 that identifies a location of metadata within packet 130 for removal or another criteria that can allow switch 102 to reliably determine metadata for removal. In some implementations, this metadata can be further processing by switch 102 before being sent to another computing device, such as network controller 128, for analysis. In some implementations, switch 102 is also programmed to analyze and/or process the removed metadata.

[0039] FIG. 4 illustrates another example of method 134 in accordance with the present disclosure. For illustration, FIG. 4 reproduces various blocks from method 134 of FIG. 2, however it is appreciated that method 134 of FIG. 4 can include additional, alternative, or fewer steps, functionality, etc., than method 134 of FIG. 2 and is not intended to be limited by the diagram of FIG. 1 (or vice versa) or the related disclosure thereof. It is further appreciated that method 134 of FIG. 4 can incorporate one or more aspects of method 134 of FIG. 2 and vice versa. For example, in some implementations, method 134 of FIG. 2 can include the additional step described below with respect to method 134 of FIG. 4.

[0040] Method 134 of FIG. 4 includes sending (at block 144), with ASIC 106 of switch 102, removed metadata to a processing unit to build a map of the network configuration based on the metadata. The map can, for example be in the form of a suitable network topology or latency map. The map can, for example, be built dynamically based on real-time data from one or more switches within network 100 (e.g., switch 102 and switch 102). In some implementations, a map of the network can be built that reflects latency and congestion information of the various links of the network. A network map produced by latency tracking metadata can, in some

implementations, be designed to allow a network maintenance department to solve issues of network connectivity, configuration or even defective hardware. It is appreciated that the present disclosure is not intended to dictate a specific way to display a topology/congestion map, (e.g., whether per VLAN, per application port/ link, etc.). Instead, the map is created or updated during the network operation, and can be done by a separate software component, or by an existing software visualization component that can be adapted in order to display, process or do further analysis with the information collected by the network.

[0041] In some implementations, before sending the metadata, switch 102 can compile the metadata into a data structure. In some implementations, only metadata is included in the compiled data structure. That is, the data structure does not store the payload data of the data packet. In other implementations, both metadata and payload data is stored in the data structure. A data structure for use with the present disclosure can include any suitable structure for organizing data for use by a computer. An example data structure can include fixed-length or resizable arrays, which can for example list a number of elements in a specific order that are accessible using an integer index to specify which element is requested. In some implementations, associative arrays, such as hash tables, may also be used as suitable data structures. In some implementations, aggregated data structures, such as records that contain other elements in the forms of fields or members, can be used. It is appreciated that any suitable data structure may be used. As another example, a data packet itself may be considered a data structure. For example, in some implementations, a compressed data packet that eliminates certain irrelevant metadata fields and/or payload data from the original data packet can be compiled by switch 102 and sent to another computing device, such as network controller 128, or can remain on switch 102 for processing and/or analysis of the latency tracking metadata.

[0042] In some implementations, an SDN protocol can be used to send the removed

metadata to one or more nodes of network 100 to a network controller 128 or other entity. For example, network controller 128 can, in some implementations, be in the form of an SDN controller and can receive the removed metadata for further processing and/or analysis. In such an implementation, the removed metadata can, for example, be used in accordance with an objective of an SDN application running on the SDN controller. In some implementations, network controller 128 can, for example, assign an identifier (ID) to each switch and can further assign to each switch a special value or cookie that can be added to the latency tracking metadata of the data packet. In implementations, where a given switch has an SDN pipeline, then a number of tables and an identifier for each table can be incorporated in the latency tracking metadata.

[0043] In some implementations, method 134 can include other operations, such as for example, performing one or more actions based on the latency tracking metadata. In some implementations, actions can be applied for a predefined amount of time (e.g., by associating timers to the action) or a predefined number of bytes (e.g., by associating bytes counters to the action), and/or other conditions. In some implementations, actions can be performed on the packet level (e.g., forward the packet to a given egress port of switch 102 or modify a packet header) or at another level of network management or routing.

[0044] For example, in some implementations, method 134 can include updating a routing table stored on network switch 102 based on the latency tracking metadata. It is appreciated that other actions may be performed based on the latency tracking metadata. For example, in some implementations, a Deep Packet Inspection (DPI) operation can be performed for the data packet based on the latency tracking metadata. For example, ASIC 106 can be configured to analyze the data packet to search for a suspicious DPI signature. In such an implementation, ASIC 106 can then send any suspicious signatures and/or the data packet itself to network controller 128 (or another entity in network 100 or elsewhere) for further analysis and/or processing.

[0045] FIG. 5 is a diagram of an example network switch 102 in accordance with the present disclosure. As described in further detail below, switch 102 includes a CPU processing resource 146, an ASIC processing resource 148, and a memory resource 150 that stores machine-readable instructions 152, 154, 156, and 158. For illustration, the description of switch 102 of FIG. 5 makes reference to various aspects of the diagram of FIG. 1 as well as method 134 of FIGs. 2-4. Indeed, for consistency, the same reference number for the switch of FIG. 1 is used for the switch of FIG. 5. However it is appreciated that switch 102 of FIG. 5 can include additional, alternative, or fewer aspects, functionality, etc., than the implementation described with respect to method 134 as well as the switch of FIG. 1 and is not intended to be limited by the related disclosure thereof. [0046] Instructions 152 stored on memory resource 150 are, when executed by CPU processing resource 146, to remove, with the first ASIC, latency tracking metadata from the data packet that identifies internal switch state and link latency information of an upstream network switch. Instructions 152 can incorporate one or more aspects of blocks of method 134 or another suitable aspect of other

implementations described herein (and vice versa). Instructions 154 stored on memory resource 150 are, when executed by CPU processing resource 146, to build, with the first ASIC, a network configuration database based on the removed latency tracking metadata. Instructions 154 can incorporate one or more aspects of blocks of method 134 or another suitable aspect of other implementations described herein (and vice versa).

[0047] Instructions 156 stored on memory resource 150 are, when executed by CPU

processing resource 146, to forward the data packet without the latency tracking metadata to a downstream node. Instructions 156 can incorporate one or more aspects of blocks of method 134 or another suitable aspect of other

implementations described herein (and vice versa). Instructions 158 stored on memory resource 150 are, when executed by CPU processing resource 146, to forward the network configuration database to an external computer for analysis of the network. Instructions 158 can incorporate one or more aspects of blocks of method 134 or another suitable aspect of other implementations described herein (and vice versa).

[0048] In some implementations, instructions stored on memory resource 150 are, when executed by CPU processing resource 146, to add metadata to the data packet that identifies internal switch state and link latency information of the first network switch when it is determined that the received packet is to be routed to a downstream network switch that includes a latency metadata tracking

programmable ASIC. Such instructions can incorporate one or more aspects of blocks of method 134 or another suitable aspect of other implementations described herein (and vice versa).

[0049] Each processing resource 146 and 148 of network switch 102 can, for example, be in the form of a central processing unit (CPU), a semiconductor-based microprocessor, a digital signal processor (DSP) such as a digital image processing unit, other hardware devices or processing elements suitable to retrieve and execute instructions stored in memory resource 150, or suitable combinations thereof. Each processing resource 146 and 148 can, for example, include single or multiple cores on a chip, multiple cores across multiple chips, multiple cores across multiple devices, or suitable combinations thereof. Each processing resource 146 and 148 can be functional to fetch, decode, and execute instructions as described herein. As an alternative or in addition to retrieving and executing instructions, each processing resource 146 and 148 can, for example, include at least one integrated circuit (IC), other control logic, other electronic circuits, or suitable combination thereof that include a number of electronic components for performing the functionality of instructions stored on memory resource 150. The term "logic" can, in some implementations, be an alternative or additional processing resource to perform a particular action and/or function, etc., described herein, which includes hardware, e.g., various forms of transistor logic, application specific integrated circuits (ASICs), etc., as opposed to machine executable instructions, e.g., software firmware, etc., stored in memory and executable by a processor. Each processing resource 146 and 148 can, for example, be implemented across multiple processing units and instructions may be implemented by different processing units in different areas of network switch 102. Memory resource 150 of switch 102 can, for example, be in the form of a non- transitory machine-readable storage medium, such as a suitable electronic, magnetic, optical, or other physical storage apparatus to contain or store information such as machine-readable instructions 152, 154, 156, and 158. Such instructions can be operative to perform one or more functions described herein, such as those described herein with respect to method 134 or other methods described herein. Memory resource 150 can, for example, be housed within the same housing as one or more processing resources 146 and 148 for network switch 102, such as within a computing tower case for network switch 102 (in

implementations where network switch 102 is housed within a computing tower case). In some implementations, memory resource 150 and processing resources 146 and 148 are housed in different housings. As used herein, the term "machine- readable storage medium" can, for example, include Random Access Memory (RAM), flash memory, a storage drive (e.g., a hard disk), any type of storage disc (e.g., a Compact Disc Read Only Memory (CD-ROM), any other type of compact disc, a DVD, etc.), and the like, or a combination thereof. In some implementations, memory resource 150 can correspond to a memory including a main memory, such as a Random Access Memory (RAM), where software may reside during runtime, and a secondary memory. The secondary memory can, for example, include a nonvolatile memory where a copy of machine-readable instructions are stored. It is appreciated that both machine-readable instructions as well as related data can be stored on memory mediums and that multiple mediums can be treated as a single medium for purposes of description.

[0051] Memory resource 150 can be in communication with processing resources 146 and 148 via suitable communication links 160 and 162. Each communication link 160 and 162 can be or remote to a machine (e.g., a computing device) associated with one or both processing resources 146 and 148. Examples of communication links can include an electronic bus internal to a machine (e.g., a computing device) where memory resource 150 is one of volatile, non-volatile, fixed, and/or removable storage medium in communication with processing resources 146 and 148 via respective electronic busses.

[0052] In some implementations, one or more aspects of network switch 102 (as well as network controller 128 or other devices of network 100) can be in the form of functional modules that can, for example, be operative to execute one or more processes of instructions 152, 154, 156, and 158, and/or other functionality described herein relating to other implementations of the disclosure. As used herein, the term "module" refers to a combination of hardware (e.g., a processor such as an integrated circuit or other circuitry) and software (e.g., machine- or processor-executable instructions, commands, or code such as firmware, programming, or object code). A combination of hardware and software can include hardware only (i.e., a hardware element with no software elements), software hosted at hardware (e.g., software that is stored at a memory and executed or interpreted at a processor), or hardware and software hosted at hardware. It is further appreciated that the term "module" is additionally intended to refer to one or more modules or a combination of modules. Each module of a network switch 102 can, for example, include one or more machine-readable storage mediums and one or more computer processors. In view of the above, it is appreciated that the various instructions of network switch 102 described above can correspond to separate and/or combined functional modules. For example, certain instructions can correspond to a routing

determination module (e.g., module 108 of FIG. 1), to determine whether packet 130 is to be routed to a second network switch (e.g., switch 114) in network 100 that includes a second latency metadata tracking programmable ASIC. Likewise, certain instructions can correspond to a metadata adding module (e.g., module 110 of FIG. 1) to add metadata to packet 130 that identifies internal switch state and link latency information of switch 102 when it is determined that packet 130 is to be routed to a second network switch that includes a second latency metadata tracking

programmable ASIC. Likewise, certain instructions can correspond to a metadata removing module (e.g., module 112 of FIG. 1) to remove metadata from packet 130 that identifies internal switch state and link latency information of one or more other network switches when it is determined that packet 130 is not to be routed to a second network switch (e.g., switch 114) that includes a second latency metadata tracking programmable ASIC. Moreover, network controller 128 can include or more modules, such as metadata tracking module 132, which can, for example allow for tracking by controller 128 of latency tracking metadata received from one or more nodes in network 100. Network controller 128 can further include other functional modules relating to building a network map based on received latency tracking metadata, generating field of inquiry for latency tracking metadata, sending instructions to controlled nodes within network 100, or other modules

corresponding to aspects of controller 128 described herein. It is further appreciated that a given module can be used for multiple functions. In some implementations, a single module can be used to add metadata and remove metadata. [0054] One or more nodes within network 100 (e.g., network controller 128, network switch 102, etc.) can further include a suitable communication module to allow networked communication between network controller 128, network switch 102, and/or other elements of network 100. Such a communication module can, for example, include a network interface controller having an Ethernet port and/or a Fibre Channel port. In some implementations, such a communication module can include wired or wireless communication interface, and can, in some implementations, provide for virtual network ports. In some implementations, such a communication module includes hardware in the form of a hard drive, related firmware, and other software for allowing the hard drive to operatively communicate with other hardware of network controller 128, network switch 102, or other network equipment. The

communication module can, for example, include machine-readable instructions for use with communication the communication module, such as firmware for implementing physical or virtual network ports.

[0055] FIG. 6 illustrates a machine-readable storage medium 164 having stored thereon machine readable instructions that can be executed by a computer processor of a first latency metadata tracking compliant (LMTC) network switch (e.g., switch 102) in network 100. In some implementations, medium 164 can be housed within a network controller, such as network controller 128, or on another computing device within network 100 or in or remote wired or wireless data communication with network 100.

[0056] For illustration, the description of machine-readable storage medium 164 provided herein makes reference to various aspects of network switch 102 (e.g., processing resources such as CPU processing resource 146 and ASIC processing resource 148) and other implementations of the disclosure (e.g., method 134). Although one or more aspects of network switch 102 (as well as instructions such as instructions 152, 154, 156, and 158) can be applied or otherwise incorporated with medium 164, it is appreciated that in some implementations, medium 164 may be stored or housed separately from such a system. For example, in some implementations, medium 164 can be in the form of Random Access Memory (RAM), flash memory, a storage drive (e.g., a hard disk), any type of storage disc (e.g., a Compact Disc Read Only Memory (CD-ROM), any other type of compact disc, a DVD, etc.), and the like, or a combination thereof.

[0057] Medium 164 includes machine-readable instructions 166 stored thereon to cause ASIC processing resource 148 to determine whether a data packet is to be routed from the first network switch to a second network switch previously identified to the first LMTC network switch as being a LMTC network switch. Instructions 166 can, for example, incorporate one or more aspects of one or more blocks of method 134 or instructions of network switch 102 or another suitable aspect of other

implementations described herein (and vice versa). Medium 164 further includes machine-readable instructions 168 stored thereon to cause ASIC processing resource 148 to determine whether the data packet includes latency tracking metadata from a LMTC network switch upstream from the first LMTC network switch. Instructions 168 can, for example, incorporate one or more aspects of one or more blocks of method 134 or instructions of network switch 102 or another suitable aspect of other implementations described herein (and vice versa).

[0058] Medium 164 further includes machine-readable instructions 170 stored thereon to cause ASIC processing resource 148 to send, with the first ASIC, latency tracking metadata removed from the data packet to a processing unit when it is determined that the data packet includes latency tracking metadata and when it is determined that the data packet is not to be routed to a second LMTC network switch.

Instructions 170 can, for example, incorporate one or more aspects of one or more blocks of method 134 or instructions of network switch 102 or another suitable aspect of other implementations described herein (and vice versa).

[0059] While certain implementations have been shown and described above, various changes in form and details may be made. For example, some features that have been described in relation to one implementation and/or process can be related to other implementations. In other words, processes, features, components, and/or properties described in relation to one implementation can be useful in other implementations. Furthermore, it should be appreciated that the systems and methods described herein can include various combinations and/or subcombinations of the components and/or features of the different implementations described. Thus, features described with reference to one or more implementations can be combined with other implementations described herein. As used herein, "logic" is an alternative or additional processing resource to perform a particular action and/or function, etc., described herein, which includes hardware, e.g., various forms of transistor logic, application specific integrated circuits (ASICs), etc., as opposed to machine executable instructions, e.g., software firmware, etc., stored in memory and executable by a processor. Further, as used herein, "a" or "a number of" something can refer to one or more such things. For example, "a number of widgets" can refer to one or more widgets. Also, as used herein, "a plurality of" something can refer to more than one of such things.

Claims

CLAIMS What is claimed is:

1. A method comprising: receiving, with a first latency metadata tracking programmable Application-Specific Integrated Circuit (ASIC) of a first network switch in a network, a data packet; determining, with the first ASIC, whether the received data packet is to be routed to a second network switch in the network that includes a second latency metadata tracking programmable ASIC; adding, with the first ASIC, metadata to the data packet that identifies internal switch state and link latency information of the first network switch when it is determined that the received packet is to be routed to a second network switch that includes a second latency metadata tracking programmable ASIC.

2. The method of claim 1, wherein the internal switch state information includes switch logs for the first network switch.

3. The method of claim 1, further comprising: removing, with the first ASIC, metadata from the data packet that identifies internal switch state and link latency information of one or more other network switches when it is determined that the received packet is not to be routed to a second network switch that includes a second latency metadata tracking programmable ASIC.

4. The method of claim 3, further comprising:

sending, with the first ASIC, removed metadata to a processing unit to build a map of the network configuration based on the metadata.

5. The method of claim 4, wherein the map is in the form of a network topology map.

6. The method of claim 1, wherein the processing unit is housed within the first network switch.

7. The method of claim 1, wherein the map is in the form of a network latency map.

8. The method of claim 1, wherein the map is built dynamically based on real-time data from the first network switch.

9. The method of claim 1, wherein the added metadata includes Media Access Control (MAC) addresses for the source node and destination node of the data packet.

10. The method of claim 1, wherein the metadata that identifies internal switch state information includes routing table information stored on the first network switch.

11. The method of claim 1, wherein the metadata that identifies link latency information includes time stamp information for the data packet.

12. A non-transitory machine readable storage medium having stored thereon machine readable instructions to cause a computer processor of a first latency metadata tracking compliant (LMTC) network switch in a network to:

determine, with a first Application-Specific Integrated Circuit (ASIC) separate from a Central Processing Unit (CPU) of the first LMTC network switch, whether a data packet is to be routed from the first network switch to a second network switch previously identified to the first LMTC network switch as being a LMTC network switch;

determine, with the first ASIC, whether the data packet includes latency tracking metadata from a LMTC network switch upstream from the first LMTC network switch; send, with the first ASIC, latency tracking metadata removed from the data packet to a processing unit when it is determined that the data packet includes latency tracking metadata and when it is determined that the data packet is not to be routed to a second LMTC network switch.

13. The medium of claim 12, wherein the processing unit is external to the first network switch.

14. A network switch comprising:

a first Application-Specific Integrated Circuit (ASIC) processing resource; and an ASIC memory resource storing machine readable instructions to cause the processing resource to:

remove, with the first ASIC, latency tracking metadata from the data packet that identifies internal switch state and link latency information of an upstream network switch; build, with the first ASIC, a network configuration database based on the removed latency tracking metadata;

forward the data packet without the latency tracking metadata to a downstream node; and

forward the network configuration database to an external computer for analysis of the network.

15. The network switch of claim 14, wherein the ASIC memory resource stores machine readable instructions to cause the processing resource to:

add metadata to the data packet that identifies internal switch state and link latency information of the first network switch when it is determined that the received packet is to be routed to a downstream network switch that includes a latency metadata tracking programmable ASIC.