US20190013954A1 - Elastic timestamping - Google Patents
Elastic timestamping Download PDFInfo
- Publication number
- US20190013954A1 US20190013954A1 US15/850,502 US201715850502A US2019013954A1 US 20190013954 A1 US20190013954 A1 US 20190013954A1 US 201715850502 A US201715850502 A US 201715850502A US 2019013954 A1 US2019013954 A1 US 2019013954A1
- Authority
- US
- United States
- Prior art keywords
- timestamp
- response
- packet
- telemetry
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
- H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
- H04Q11/0062—Network aspects
- H04Q11/0067—Provisions for optical access or distribution networks, e.g. Gigabit Ethernet Passive Optical Network (GE-PON), ATM-based Passive Optical Network (A-PON), PON-Ring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2315—Optimistic concurrency control
- G06F16/2322—Optimistic concurrency control using timestamps
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9014—Indexing; Data structures therefor; Storage structures hash tables
-
- G06F17/30353—
-
- G06F17/30551—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J14/00—Optical multiplex systems
- H04J14/02—Wavelength-division multiplex systems
- H04J14/0227—Operation, administration, maintenance or provisioning [OAMP] of WDM networks, e.g. media access, routing or wavelength allocation
- H04J14/0254—Optical medium access
- H04J14/0267—Optical signaling or routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0673—Clock or time synchronisation among packet nodes using intermediate nodes, e.g. modification of a received timestamp before further transmission to the next packet node, e.g. including internal delay time or residence time into the packet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4641—Virtual LANs, VLANs, e.g. virtual private networks [VPN]
- H04L12/4645—Details on frame tagging
- H04L12/465—Details on frame tagging wherein a single frame includes a plurality of VLAN tags
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/38—Flow based routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0816—Key establishment, i.e. cryptographic processes or cryptographic protocols whereby a shared secret becomes available to two or more parties, for subsequent use
- H04L9/085—Secret sharing or secret splitting, e.g. threshold schemes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
- H04L9/3242—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3297—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving time stamps, e.g. generation of time stamps
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M11/00—Telephonic communication systems specially adapted for combination with other electrical systems
- H04M11/002—Telephonic communication systems specially adapted for combination with other electrical systems with telemetering systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
- H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
- H04Q11/0062—Network aspects
- H04Q11/0066—Provisions for optical burst or packet networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
- H04Q9/02—Automatically-operated arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4641—Virtual LANs, VLANs, e.g. virtual private networks [VPN]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4604—LAN interconnection over a backbone network, e.g. Internet, Frame Relay
- H04L2012/4629—LAN interconnection over a backbone network, e.g. Internet, Frame Relay using multilayer switching, e.g. layer 3 switching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/24—Multipath
- H04L45/245—Link aggregation, e.g. trunking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
- H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
- H04Q11/0005—Switch and router aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
- H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
- H04Q11/0062—Network aspects
- H04Q2011/0079—Operation or maintenance aspects
- H04Q2011/0083—Testing; Monitoring
Definitions
- the disclosure relates generally to a system and method for variable size elastic timestamping for use in computing and networking applications.
- Timestamping involves recording digital timestamp values, which may be used to create a temporal order among a set of events. Timestamping techniques are used in a variety of computing fields, such as network management, computer security, and concurrency control. Timestamping is often used in network telemetry, which involves the use of automated tools and processes designed to collect measurements and other data at points throughout a network, which can then be used for network monitoring and performance analysis. Timestamping for telemetry can be used to verify the freshness of telemetry data collected and temporal ordering of telemetry data within a flow.
- the time at which the queue length was measured is needed to correlate the queue length information to the flows present.
- the temporal information may be used to analyze bottlenecks in the network for the specific flows.
- a method and system for elastic timestamping for use in computing and networking applications including telemetry is disclosed herein.
- a device that is part of a system may initially generate a variable size timestamp or elastic n-dimensional timestamp (ENTS) (a first timestamp) with n time dimensions fields for a corresponding event in the system for which timing or temporal order information is needed.
- the device may select a subset of the n time dimensions fields of the ENTS based on a relevant time granularity of the corresponding event to generate a compact ENTS (a second timestamp) with a reduced size.
- the device may communicate the compact ENTS for further processing.
- the ENTS may be generated for a device-specific action performed to gather telemetry data in response to received telemetry intent at the device, and the compact ENTS may be communicated with a corresponding telemetry response.
- the disclosed elastic timestamping system and method is used in a packet-optical in-band telemetry (POINT) framework designed for gathering multi-layer telemetry data in packet-optical networks.
- POINT packet-optical in-band telemetry
- FIG. 1 is a high-level diagram of an example packet-optical in-band telemetry (POINT) framework implemented in an example packet-optical network, in accordance with the disclosures herein;
- POINT packet-optical in-band telemetry
- FIG. 2 is a flow diagram of an example telemetry processing procedure that meets data freshness requirements and includes ENTS in the telemetry responses, in accordance with the disclosures herein;
- FIG. 4 is a block diagram of a computing system in which one or more disclosed embodiments may be implemented.
- a method and system for elastic timestamping with variable and adjustable size for use in computing and networking systems are disclosed, where an entity or device in the system generates an elastic n-dimensional timestamp (ENTS) for a corresponding event or action in the system for which timing or temporal order information is needed.
- Each of the n fields or dimensions (time dimension fields) of the ENTS provides a different granularity in time (e.g., hours, minutes, seconds, milliseconds, etc.) and each of the n fields has a length that may be different from that of the other fields.
- the device selects a subset of the n dimensions of ENTS to generate a reduced or compact ENTS (i.e., a compact timestamp) for communication (e.g., reporting to a receiving entity for processing, recording to storage, displaying to an operator etc.) by determining the relevant time granularity for the corresponding event.
- the device may also communicate an indication of which dimensions are included in and/or excluded from the ENTS.
- the device may maintain and periodically report a mapping from ENTS to a local clock value at the device.
- the device may receive event or data freshness criteria or thresholds and may use the ENTS to ensure that the reported system events or actions meet the data freshness criteria, and drop information for system events that do not meet the data freshness criteria.
- any of the disclosed timestamping mechanisms may be used in any application that makes use of timestamping, including, but not limited to any computing or network application that requires record keeping, network administration and management applications, computer security applications, concurrency control applications, and synchronization applications.
- Providing temporal information as part of telemetry data may be used to verify freshness of the telemetry information collected and temporal ordering of telemetry data within a flow.
- observation entails the act of actual telemetry data measurement by the hardware (e.g., measuring bit error rate (BER), measuring signal-to-noise ratio (SNR)) in response to a telemetry request, and recording the data involves collecting, logging, and/or displaying the response to the telemetry request. Consequently, there may be a delay between the observation of telemetry data and recording of telemetry data.
- BER bit error rate
- SNR signal-to-noise ratio
- the timestamp at which observations were made may not be available and recording of the observation may happen at periodic intervals (e.g., every 15 minutes, once a day, etc.) instead of at the actual time of observation.
- the timestamp associated with the collection of telemetry data may not have the fine detail needed to order relevant network-wide events to effectively serve the purpose of telemetry to troubleshoot network issues across a network. This problem may be even more exaggerated in multi-layer networks comprising both packet and optical devices where delays between observation and recording of data may be greater.
- data plane timestamping i.e., where the timestamps are added to the telemetry data and transmitted in packets/frames, for example in the header, across the network
- the timestamping information may be included with the telemetry data
- the collected telemetry data may then be properly interpreted and correlated at a processing device (e.g., at a sink node) and put in a time sequence in order, even if the data arrives out of order.
- the receipt of telemetry data without temporal information may be unreliable because the data may be received out-of-order from how it was originally observed and/or transmitted.
- Data plane timestamping may also be used to provide temporal ordering of network events that may occur across different devices, for example when network telemetry is used to analyze the impact of transient fiber failures in an optical or packet-optical network.
- Different devices in a network may have different clock values, and thus temporal ordering across devices is essential to be able to correlate the pieces of data collected from different network devices.
- a timestamp will include detailed time information such as year, month, day, hour, second, and fraction of a second.
- NTP network time protocol
- a timestamp occupies 8 bytes (64 bits) with two 32-bit fields: an integer part of the number of seconds since the base date (i.e., a reference start time for the local clock) and the fractional part of the number of seconds (or nanoseconds field).
- the timestamp may only be needed at the ingress node and the egress node, for example to measure the overall delay across the network.
- certain network measurements require data and associated timestamp information to be collected at each hop along the network path (e.g., when measuring delay across particular links, or when trying to locate a particular link failure). In these cases, the accumulation of timestamp information for each recorded piece of telemetry data generates a substantial amount of overhead data.
- differential timestamps where a time difference or “delta” time value from the timestamp from the timestamp of the previous event is communicated to reduce overhead.
- differential timestamps require a state be maintained at various points in the network and are limited in their use to timestamping on a per-flow basis only.
- the disclosed elastic timestamping system uses a timestamping format that can be adjusted in size, in terms of number of fields and/or total bits, to generate a compact timestamp that meets the desired time granularity needed by telemetry applications (or other computing and networking applications) while reducing overhead.
- the term “elastic” refers to the fact that the timestamp can be dynamically adjusted while still effectively conveying the relevant time information.
- the communication of other dimensions e.g., higher dimensions such as date, day, or hour
- a timestamp is generated at a device to record the time at which a telemetry event occurred (e.g., by reading the value of a clock running locally at the device).
- the timestamp has an elastic n-dimensional timestamp (ENTS) format with up to n fields: [T 11 . . . T 1m :T 21 . . . T 2p : . . . T n1 . . . T nq ].
- Each field may provide a different unit or granularity of time (also referred to as dimension or time dimension field of ENTS).
- the fields may represent the hours, minutes, seconds, and milliseconds of the timestamp (e.g., HH:MM:SS:MSS).
- Each field has a respective length in bits, m, p, . . . , q, such that the lengths may or may not be the same (e.g., m ⁇ p ⁇ . . . ⁇ q).
- the number of fields n and the lengths of the fields m, p, . . . q for the ENTS may be provided or fixed in advance for the participating devices in the network or configured dynamically during network operation.
- each participating device may derive the ENTS from a local clock value.
- the local clock value may be read from the local or system clock at the device.
- the local clock value may be a subset of bits of the local clock.
- a selected subset of dimensions (or fields) of ENTS are communicated for each telemetry data collected according to the relevant granularity for the time context of the event measured by the telemetry data.
- a total number of dimensions s ⁇ n may be communicated as a compact or compressed ENTS (i.e., compact timestamp), such that the selected dimensions may include one or more ENTS dimensions closest to the granularity relevant to the telemetry observation.
- the receiver that processes the telemetry data e.g., the sink node
- events that need to be distinguished in time at the millisecond (ms) granularity need not send the hourly values (hourly field) of the ENTS.
- events that need to be distinguished in time at the hourly granularity may not need to send the ms values (ms field) of the ENTS.
- the relevant ENTS dimension are the day and hour in the day that the temperature readings were averaged over. Second and millisecond information is not needed in this case, and thus the second and millisecond ENTS dimensions may not be communicated with the telemetry data.
- an ENTS of 22 bits in length can be used to monitor events at millisecond granularity within an overall period of one hour, thus removing the higher and lower time dimensions compared to an NTP timestamp of 64 bits, and thus reducing the size (length) in bits of the timestamp by over 65%.
- the device may communicate the selected ENTS dimensions associated with particular telemetry data along with the telemetry data and may insert the selected ENTS dimensions in the packet or frame header carrying the corresponding telemetry intent or may communicate the selected ENTS dimensions with the corresponding telemetry data via another route such as in an out-of-band channel.
- an indication (e.g., a few bits) may be included with the selected ENTS dimensions and telemetry data to indicate the presence or absence of particular dimensions in the selected ENTS.
- the indication may indicate with the bit sequence “0011” that the hour and minute dimensions are absent and the second and millisecond dimensions are present in the ENTS carried in the header.
- each device may periodically communicate, to the entity that processes the telemetry data (e.g., the sink node), a mapping from its ENTS (or compressed ENTS) to the local clock value at the device.
- a device's local mapping from ENTs to its local clock value is useful in a network where it is cannot be assumed that the clocks across the network devices are synchronized or maintain strict synchronization. Because a packet (data flow) carrying telemetry data will travel across multiple devices, the ENTS values gathered at the devices may be out of synch relative to one another because the respective device clocks are not necessarily synchronized.
- the periodic mapping of ENTS to local clock value at each device along a path can be used at the sink node to determine the relative timing and temporal ordering of the events across the multiple devices.
- mechanisms may be provided to ensure data freshness of the telemetry data.
- an intermediate device along a network path may be provided with a required or desired data freshness criteria for telemetry data that is used to validate the freshness of collected data before sending it as a response to an intent request.
- the desired data freshness criteria may be that telemetry data be locally measured within a time frame, for example within the last 10 ms (e.g., when collecting packet loss information).
- the data freshness criteria may be that telemetry data be gathered or received within a time period (e.g., the last 15 ms), in cases where the measurements is measured remotely but collected locally at the device (e.g., where measurements are performed at a layer 0 (L0) device but the corresponding data is processed and inserted into the data flow by a layer (L1) device such as at a packet-optical layer boundary device in a packet-optical network).
- Data freshness criteria may be communicated to a device in a number of ways.
- the data freshness criteria may be communicated along with the telemetry intent (e.g., in the packet or frame header), or via an out-of-band channel.
- the data freshness criteria may be programmed into individual devices.
- data freshness may not be stipulated as a requirement. Instead, the device may include information regarding the freshness of the telemetry data in the telemetry response.
- telemetry applications are described in more detail and examples are given of the disclosed elastic timestamping system and method as used in telemetry applications.
- the disclosed elastic timestamping system and methods may be used in any application that uses timestamping, including, but not limited to any computing or network application that requires record keeping, network administration and management applications, computer security applications, concurrency control applications, and synchronization applications.
- Streaming telemetry mechanisms such as OpenConfig, are designed to streamline the notification of network state by having the network elements stream the telemetry data up to a central management entity where the data gets stored and processed. While streaming telemetry mechanisms employ extensive offline algorithms to process telemetry data, they are not designed to inherently improve the quality of the data collected.
- INT In-band Network Telemetry
- the data plane refers to the part of a device's architecture that makes forwarding decisions for incoming packets. For example, routing may be determined by the device using a locally stored table in which the device looks up the destination address of the incoming packet and retrieves the information needed for forwarding.
- the INT framework relies on programmable data planes to bring flexibility to telemetry data collection.
- Devices with programmable data planes include network processors or general-purpose central processing units (CPUs) at the low end, and data path programmable switch chips at the high end.
- a source switch or more generally, a source network device incorporates an instruction header to collect network state information as a part of the data packet.
- Intermediate INT-capable switches interpret the instruction header and collect and insert the desired network state information in the data packet, which eventually reaches a sink switch and can be used as needed to monitor and evaluate the operation of the network.
- INT includes real-time telemetry rates, low CPU and operating system (O/S) overhead, and the flexibility to programmatically instrument packets to carry useful telemetry data.
- the programmable data planes used in INT have been explicitly designed for packet networks; however extending INT mechanisms into optical networks, where there is no notion of data packets, is far from straightforward due to factors such as layering and the presence of purely analog devices.
- packet-optical networks such as those interconnecting data centers
- packet-optical networks see additional challenges when it comes to network telemetry because of the different types of telemetry data collected in packet versus optical networks.
- the telemetry data collected in a packet layer of a packet network such as packet loss and latency, on a per-flow basis cannot be easily attributed to or correlated with data collected in the optical layer of an optical networks, such as bit error rates (BERs) and quality factor (Q-factor).
- BERs bit error rates
- Q-factor quality factor
- the optical network lacks the digital constructs used by telemetry solutions such as INT, and the packet layer does not have access to measurements in the optical network.
- a further challenge occurs in associating packet flow telemetry data with the corresponding data from optical transport network (OTN) layers, which involves piecing together telemetry data from many devices.
- OTN optical transport network
- Optical parameters may affect traffic flows. For example, if a link experiences degradation in Q-without link failure, operators can use the knowledge of that information to proactively move critical applications away from that particular link. Thus, it is useful for network operators to be able to monitor optical parameters over time to use in routing and other applications.
- a source device inserts an intent (POINT intent) for telemetry data collection along with the data flow.
- the intent communicates the parameters of data collection such as conditions for data collection, entities being monitored, and the type of data to be collected for that flow.
- Intermediate devices on that data flow process the high-level intent if it is targeted towards them, translate the intent into a suitable device-specific action for data collection and execute that action to collect an intent response.
- a layer boundary such as packet to optical, or across optical layers such as a hierarchy of optical data units (ODUs)
- intermediate devices translate the intent and response using a layer-appropriate mechanism.
- the intent and response may be encapsulated using IP options or VXLAN metadata header.
- the intent can be retrieved from the packet header, and translated and encapsulated as ODU layer metadata, which remain accessible to all nodes along the end-to-end path of the ODU.
- the POINT intent can be translated into an appropriate query for telemetry data collection via the management plane of the optical devices. As soon as the response of data collection is ready, it is communicated through the optical network and translated appropriately into a packet or packet header at the packet-optical boundary and forwarded to the sink for analysis. For example, the response communication may be out-of-band using the optical supervisory channel (OSC).
- OSC optical supervisory channel
- the POINT framework also supports adding response metadata for incorporating deployment-specific reliability mechanisms.
- the POINT framework provides hierarchical layering with intent and response translation at each layer boundary, and mapping of the intent to layer-specific data collection mechanism, such that the POINT framework can be deployed across a network layer hierarchy.
- the POINT framework also provides for fate sharing of telemetry intent and data flow. Telemetry data for a specific data flow can be collected in-band as the data traverses the network layers.
- intent responses can be out-of-band to accommodate scenarios such as troubleshooting networks when there is no connectivity between the source and the sink.
- intents are high level instructions for data collection and can be mapped to existing data collection mechanisms between two POINT capable intermediate network devices.
- FIG. 1 is a high-level diagram of an example POINT framework 100 implemented in an example packet-optical network 102 , in accordance with the disclosures herein.
- the example packet-optical network 102 includes packet devices 110 , 112 , 114 , 116 , 118 and 120 , and an optical network 104 segment that includes optical devices 122 , 124 , 126 , 128 , 130 and 132 .
- the POINT framework 100 can operate over an optical network 104 with Layer0 (L0) and/or Layer-1 (L1) circuits.
- the packet devices include a POINT source device 110 and a POINT sink device 120 , as well as packet optical gateways (POGs) 114 and 116 located at the interfaces between the packet segments and optical network 104 segment of the packet-optical network 102 .
- the packet devices 110 , 120 , 114 and 116 can operate at the packet layer, for example at layer 2 (L2)/layer 3 (L3) (e.g., L2 may be a data link layer and L3 may be a network layer, which exist above a physical layer).
- POGs 114 and 116 are also connected via lower layer devices, such as L1/L0 devices 122 , 124 , 126 , 128 , 130 and 132 .
- POGs 114 and 116 and optical devices 126 and 128 are configured as POINT intermediate devices (i.e., devices with digital capability to interpret POINT intent, translate it across layers, and aggregate and propagate the telemetry state information in the packet-optical network 102 ).
- telemetry information for a packet-optical traffic flow 105 such as intent or POINT data (e.g., intent and response), in the packet-optical network 102 is gathered in the data plane 140 as part of the information carried in the network 102 , as described below.
- the telemetry plane 160 represents the telemetry information for the packet optical flow 105 being mapped and correlated across network layers, constructs (e.g., secure network communications (SNC), label-switched path (LSP), or virtual local area network (ULAN)) and devices operating at different layers in the networking stack to give the end user (e.g., at the POINT sink) a unified view of the operation of the entire packet-optical network 102 .
- SNC secure network communications
- LSP label-switched path
- ULAN virtual local area network
- a POINT source device 110 may initiate a network telemetry data collection for a packet-optical flow 105 along a packet-optical data path from the source device 110 to a sink device 120 .
- POINT intermediate devices such as POGs 114 , 116 , and optical devices 126 , 128 , may interpret the intent, collect the desired telemetry data, and encode it back into the packet (flow) 142 , which eventually gets forwarded to the sink device 120 .
- packet (frame) 142 traverses the packet-optical network 102 across devices and layers (e.g., packet layers L2/L3 and optical layers L1/L0), in the data plane 140 intent information is transferred into other layers, translated into device-specific actions, and responses are collected (e.g., added to POINT data in packet 142 ) for use at the POINT sink device 120 .
- the collected telemetry data for the packet-optical flow 105 (collected from POINT source device 110 to POINT sink device 120 ) is processed as needed by the intended applications. Examples of telemetry data processing may include triggering a report to a management entity (e.g., using mechanisms like OpenConfig) or archiving collected data in a storage device.
- a management entity e.g., using mechanisms like OpenConfig
- FIG. 2 is a flow diagram of an example telemetry processing procedure 200 that meets data freshness requirements and includes ENTS in the telemetry responses, in accordance with the disclosures herein.
- the telemetry processing procedure 200 may be performed in an optical layer by an intermediate optical or packet-optical device in a packet-optical network, although the procedure is similar in a packet layer, as performed by a packet device, involving packets instead of ODU frames.
- the device may read POINT intent from the ODU frame 210 , for example from the ODU-L1POINT-OH 214 .
- a POINT intent may include a corresponding data freshness requirement that indicates a maximum time duration for which telemetry data is valid (e.g., Intenti has a 1 minute data freshness requirement, and Intent. has a 200 ms data freshness requirement).
- the received intents may be added to the associative map 224 (also referred to as mapping table or table) while responses are generated.
- the device may check if the response for a particular intent is ready by looking it up in the associative map 224 that keeps a mapping of intent and corresponding response.
- Response 2 and Response 3 are available for corresponding Intent 2 and Intent 3 , whereas a response is not yet available for Intenti.
- the device may also verify that the ENTS of each response (shown in this example to include three fields: (hour: minute: second)) meets the data freshness requirements for the corresponding intent.
- the intent and response may be added to the ODU frame 210 , for example into the ODU-L1POINT-OH 214 .
- the intent may be added to an intent queue 220 for intent translation and processing.
- the corresponding response may be removed from the intent queue 220 , translated, processed, and inserted in the associative map 224 to be inserted with the response in an ODU frame 210 (in this case a subsequent ODU frame from when the corresponding intent was received) for downstream forwarding.
- the response may be communicated via an out-of-band channel.
- ENTS in telemetry applications may be used to provide an ordering of events on the same device.
- the ENTS dimensions and value within each dimension may be used to order events, for example in increasing or decreasing order in time (“dictionary order”).
- ENTS in telemetry applications may be used to provide an ordering of network events across different devices.
- ENTs may be used to provide ordering of telemetry data within a telemetry (e.g., POINT) flow.
- an ENTS value pertaining to an event is not the exact time value of the event.
- the event may occur in hardware whereas the recording of the ENTS value may occur in software, such that there may be some delay between the event and the ENTS recording.
- latency bounds for local ENTS values may be generated, and may be provided to the sink node to provide a bound on the accuracy of the ENTS value and may be particularly useful when evaluating data freshness.
- ENTS values may be treated as logical clock values. ENTS in telemetry applications may be used to ensure data freshness, for example by communicating data freshness requirements (e.g., with intent data) as described herein.
- FIG. 3 is a flow diagram of an example elastic timestamping procedure 300 with event freshness for use in a computing or networking system, in accordance with the disclosures herein.
- the elastic timestamping procedure 300 may be performed by an entity or device that is part of the computing or network system.
- an elastic ENTS is generated for a corresponding event or action in the system for which timing or temporal order information is needed.
- the ENTS is evaluated to verify if the event meets a freshness threshold.
- the event and corresponding ENTS may be disregarded (e.g., data discarded). Otherwise, if the event meets the freshness threshold, at 308 , a subset of the dimensions of ENTS are selected to generate a compact ENTS by determining the relevant time granularity for the corresponding event.
- the compact ENTS is communicated (e.g., reported to a receiving entity for processing, recorded to storage, displayed to an operator, etc.), thus achieving reduced overhead.
- the disclosed elastic timestamping methods and system may be implemented using software and/or hardware and may be partially or fully implemented by computing devices, such as the computing device 400 shown in FIG. 4 .
- FIG. 4 is a block diagram of a computing system 400 in which one or more disclosed embodiments may be implemented.
- the computing system 400 may include, for example, a computer, a switch, a router, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer.
- the computing device 400 may include a processor 402 , a memory 404 , a storage 406 , one or more input devices 408 , and/or one or more output devices 410 .
- the input devices 408 and output devices 410 may be generally referred to as interfaces for the computing device 400 .
- the device 400 may include an input driver 412 and/or an output driver 414 .
- the device 400 may include additional components not shown in FIG. 4 .
- the processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core may be a CPU or a GPU.
- the memory 404 may be located on the same die as the processor 402 , or may be located separately from the processor 402 .
- the memory 404 may include a volatile or non-volatile memory, for example, random access memory (RAM), dynamic RAM, or a cache.
- the storage 406 may include a fixed or removable storage, for example, a hard disk drive, a solid state drive, an optical disk, or a flash drive.
- the input devices 408 may include a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals).
- the output devices 410 may include a display, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals).
- the input driver 412 may communicate with the processor 402 and the input devices 408 , and may permit the processor 402 to receive input from the input devices 408 .
- the output driver 414 may communicate with the processor 402 and the output devices 410 , and may permit the processor 402 to send output to the output devices 410 .
- the output driver 416 may include an accelerated processing device (“APD”) 416 which may be coupled to a display device 418 .
- the APD may be configured to accept compute commands and graphics rendering commands from processor 402 , to process those compute and graphics rendering commands, and to provide pixel output to display device 418 for display.
- the point source 110 may be implemented, at least in part, with the components of computing device 400 shown in FIG. 4 .
- the procedures shown in FIGS. 2 and 3 may be implemented, at least in part, with the components of computing device 400 shown in FIG. 4 .
- Suitable processing devices include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine.
- DSP digital signal processor
- ASICs Application Specific Integrated Circuits
- FPGAs Field Programmable Gate Arrays
- Such processors may be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediary data including netlists (such instructions capable of being stored on a computer readable media).
- HDL hardware description language
- netlists such instructions capable of being stored on a computer readable media.
- the results of such processing may be maskworks that are then used in a semiconductor manufacturing process to manufacture a processor which implements aspects of the embodiments.
- non-transitory computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
- ROM read only memory
- RAM random access memory
- register cache memory
- semiconductor memory devices magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Power Engineering (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- This application claims the benefit of U.S. Provisional Application No. 62/528,964, filed Jul. 5, 2017, which is incorporated by reference as if fully set forth herein.
- The disclosure relates generally to a system and method for variable size elastic timestamping for use in computing and networking applications.
- Timestamping, as may be used in computing and networking systems, involves recording digital timestamp values, which may be used to create a temporal order among a set of events. Timestamping techniques are used in a variety of computing fields, such as network management, computer security, and concurrency control. Timestamping is often used in network telemetry, which involves the use of automated tools and processes designed to collect measurements and other data at points throughout a network, which can then be used for network monitoring and performance analysis. Timestamping for telemetry can be used to verify the freshness of telemetry data collected and temporal ordering of telemetry data within a flow. For example, when using telemetry to analyze network congestion by looking at queue lengths, the time at which the queue length was measured is needed to correlate the queue length information to the flows present. In another example, for route tracing telemetry applications for specific flows, the temporal information may be used to analyze bottlenecks in the network for the specific flows. Thus, a sense of time, or “time plus context”, provided by timestamping is frequently needed for telemetry data to be useful.
- A method and system for elastic timestamping for use in computing and networking applications including telemetry is disclosed herein. A device that is part of a system may initially generate a variable size timestamp or elastic n-dimensional timestamp (ENTS) (a first timestamp) with n time dimensions fields for a corresponding event in the system for which timing or temporal order information is needed. The device may select a subset of the n time dimensions fields of the ENTS based on a relevant time granularity of the corresponding event to generate a compact ENTS (a second timestamp) with a reduced size. The device may communicate the compact ENTS for further processing. In an example, the ENTS may be generated for a device-specific action performed to gather telemetry data in response to received telemetry intent at the device, and the compact ENTS may be communicated with a corresponding telemetry response. In an example, the disclosed elastic timestamping system and method is used in a packet-optical in-band telemetry (POINT) framework designed for gathering multi-layer telemetry data in packet-optical networks.
-
FIG. 1 is a high-level diagram of an example packet-optical in-band telemetry (POINT) framework implemented in an example packet-optical network, in accordance with the disclosures herein; -
FIG. 2 is a flow diagram of an example telemetry processing procedure that meets data freshness requirements and includes ENTS in the telemetry responses, in accordance with the disclosures herein; -
FIG. 3 is a flow diagram of an example elastic timestamping procedure with event freshness for use in a computing or networking system, in accordance with the disclosures herein; and -
FIG. 4 is a block diagram of a computing system in which one or more disclosed embodiments may be implemented. - A method and system for elastic timestamping with variable and adjustable size for use in computing and networking systems are disclosed, where an entity or device in the system generates an elastic n-dimensional timestamp (ENTS) for a corresponding event or action in the system for which timing or temporal order information is needed. Each of the n fields or dimensions (time dimension fields) of the ENTS provides a different granularity in time (e.g., hours, minutes, seconds, milliseconds, etc.) and each of the n fields has a length that may be different from that of the other fields. The device selects a subset of the n dimensions of ENTS to generate a reduced or compact ENTS (i.e., a compact timestamp) for communication (e.g., reporting to a receiving entity for processing, recording to storage, displaying to an operator etc.) by determining the relevant time granularity for the corresponding event. The device may also communicate an indication of which dimensions are included in and/or excluded from the ENTS. The device may maintain and periodically report a mapping from ENTS to a local clock value at the device. The device may receive event or data freshness criteria or thresholds and may use the ENTS to ensure that the reported system events or actions meet the data freshness criteria, and drop information for system events that do not meet the data freshness criteria.
- Examples of the disclosed timestamping system are given herein with reference to telemetry applications; however, any of the disclosed timestamping mechanisms, alone or in combination, may be used in any application that makes use of timestamping, including, but not limited to any computing or network application that requires record keeping, network administration and management applications, computer security applications, concurrency control applications, and synchronization applications.
- Providing temporal information as part of telemetry data may be used to verify freshness of the telemetry information collected and temporal ordering of telemetry data within a flow. For telemetry applications, observation entails the act of actual telemetry data measurement by the hardware (e.g., measuring bit error rate (BER), measuring signal-to-noise ratio (SNR)) in response to a telemetry request, and recording the data involves collecting, logging, and/or displaying the response to the telemetry request. Consequently, there may be a delay between the observation of telemetry data and recording of telemetry data. In some cases, the timestamp at which observations were made may not be available and recording of the observation may happen at periodic intervals (e.g., every 15 minutes, once a day, etc.) instead of at the actual time of observation. Thus, the timestamp associated with the collection of telemetry data may not have the fine detail needed to order relevant network-wide events to effectively serve the purpose of telemetry to troubleshoot network issues across a network. This problem may be even more exaggerated in multi-layer networks comprising both packet and optical devices where delays between observation and recording of data may be greater.
- In an example, data plane timestamping (i.e., where the timestamps are added to the telemetry data and transmitted in packets/frames, for example in the header, across the network) may be used to provide the temporal ordering of events on the same device, where the events include repeated readings of specific telemetry data from the device (e.g., link utilization, power levels, temperature, etc.). When the timestamping information is included with the telemetry data, the collected telemetry data may then be properly interpreted and correlated at a processing device (e.g., at a sink node) and put in a time sequence in order, even if the data arrives out of order. The receipt of telemetry data without temporal information may be unreliable because the data may be received out-of-order from how it was originally observed and/or transmitted.
- Data plane timestamping may also be used to provide temporal ordering of network events that may occur across different devices, for example when network telemetry is used to analyze the impact of transient fiber failures in an optical or packet-optical network. Different devices in a network may have different clock values, and thus temporal ordering across devices is essential to be able to correlate the pieces of data collected from different network devices. Moreover, in many networks, it may not be realistic to assume that clocks across devices are synchronized. Clock synchronization may be coordinated across different devices but is typically a resource-consuming task. Therefore, temporal ordering is particularly useful when absolute clock values are not consistent.
- While data plane timestamping has a lot of practical and essential uses for telemetry applications, it may also be very expensive to implement because it adds large amounts of overhead to the telemetry data. A timestamp will include detailed time information such as year, month, day, hour, second, and fraction of a second. For example, according to the network time protocol (NTP) timestamp format, a timestamp occupies 8 bytes (64 bits) with two 32-bit fields: an integer part of the number of seconds since the base date (i.e., a reference start time for the local clock) and the fractional part of the number of seconds (or nanoseconds field). In some use cases, the timestamp may only be needed at the ingress node and the egress node, for example to measure the overall delay across the network. However, certain network measurements require data and associated timestamp information to be collected at each hop along the network path (e.g., when measuring delay across particular links, or when trying to locate a particular link failure). In these cases, the accumulation of timestamp information for each recorded piece of telemetry data generates a substantial amount of overhead data.
- An existing solution proposes using a differential timestamp where a time difference or “delta” time value from the timestamp from the timestamp of the previous event is communicated to reduce overhead. However, differential timestamps require a state be maintained at various points in the network and are limited in their use to timestamping on a per-flow basis only.
- To address the above issues, the disclosed elastic timestamping system uses a timestamping format that can be adjusted in size, in terms of number of fields and/or total bits, to generate a compact timestamp that meets the desired time granularity needed by telemetry applications (or other computing and networking applications) while reducing overhead. The term “elastic” refers to the fact that the timestamp can be dynamically adjusted while still effectively conveying the relevant time information. With the disclosed compact timestamps, only the time dimensions that are most relevant to the telemetry data being collected are to be communicated with a response. The communication of other dimensions (e.g., higher dimensions such as date, day, or hour) may be amortized over several responses and thus not communicated with each response to reduce overhead. For example, if bit errors are being collected over the course of minutes, then there is no need to communicate the millisecond (or microsecond) timestamp values, and/or the date values may be amortized over several responses sent during the course of a particular day.
- In an example, a timestamp is generated at a device to record the time at which a telemetry event occurred (e.g., by reading the value of a clock running locally at the device). In accordance with the disclosed variable size elastic timestamping system, the timestamp has an elastic n-dimensional timestamp (ENTS) format with up to n fields: [T11 . . . T1m:T21 . . . T2p: . . . Tn1 . . . Tnq]. Each field may provide a different unit or granularity of time (also referred to as dimension or time dimension field of ENTS). For example, for n=4 fields (dimensions), the fields may represent the hours, minutes, seconds, and milliseconds of the timestamp (e.g., HH:MM:SS:MSS). Each field has a respective length in bits, m, p, . . . , q, such that the lengths may or may not be the same (e.g., m≠p≠ . . . ≠q). In an example, the number of fields n and the lengths of the fields m, p, . . . q for the ENTS may be provided or fixed in advance for the participating devices in the network or configured dynamically during network operation. In an example, each participating device may derive the ENTS from a local clock value. For example, the local clock value may be read from the local or system clock at the device. In another example, the local clock value may be a subset of bits of the local clock.
- According to the disclosed elastic timestamping system, a selected subset of dimensions (or fields) of ENTS are communicated for each telemetry data collected according to the relevant granularity for the time context of the event measured by the telemetry data. For example, a total number of dimensions s<n may be communicated as a compact or compressed ENTS (i.e., compact timestamp), such that the selected dimensions may include one or more ENTS dimensions closest to the granularity relevant to the telemetry observation. In some cases, the receiver that processes the telemetry data (e.g., the sink node) may have additional time information to expand the compact ENTS into an expanded timestamp.
- In an example, events that need to be distinguished in time at the millisecond (ms) granularity (e.g., alarm indication for loss of packet or frame) need not send the hourly values (hourly field) of the ENTS. Similarly, events that need to be distinguished in time at the hourly granularity may not need to send the ms values (ms field) of the ENTS. For example, for telemetry data that gathers average temperature readings at each device averaged over an hour time window, the relevant ENTS dimension are the day and hour in the day that the temperature readings were averaged over. Second and millisecond information is not needed in this case, and thus the second and millisecond ENTS dimensions may not be communicated with the telemetry data. As an example of overhead savings, an ENTS of 22 bits in length can be used to monitor events at millisecond granularity within an overall period of one hour, thus removing the higher and lower time dimensions compared to an NTP timestamp of 64 bits, and thus reducing the size (length) in bits of the timestamp by over 65%.
- Once the dimensions for the ENTS are selected, then only the selected ENTS dimensions for the corresponding telemetry data are used to generate a compact (compressed) ENTS. The compressed ENTS may be communicated to the responsible entity (e.g., the sink node) for further telemetry processing and analysis. In an example, the device may communicate the selected ENTS dimensions associated with particular telemetry data along with the telemetry data and may insert the selected ENTS dimensions in the packet or frame header carrying the corresponding telemetry intent or may communicate the selected ENTS dimensions with the corresponding telemetry data via another route such as in an out-of-band channel. In another example, an indication (e.g., a few bits) may be included with the selected ENTS dimensions and telemetry data to indicate the presence or absence of particular dimensions in the selected ENTS. For example, the indication may indicate with the bit sequence “0011” that the hour and minute dimensions are absent and the second and millisecond dimensions are present in the ENTS carried in the header.
- In another example, each device may periodically communicate, to the entity that processes the telemetry data (e.g., the sink node), a mapping from its ENTS (or compressed ENTS) to the local clock value at the device. A device's local mapping from ENTs to its local clock value is useful in a network where it is cannot be assumed that the clocks across the network devices are synchronized or maintain strict synchronization. Because a packet (data flow) carrying telemetry data will travel across multiple devices, the ENTS values gathered at the devices may be out of synch relative to one another because the respective device clocks are not necessarily synchronized. Thus, the periodic mapping of ENTS to local clock value at each device along a path can be used at the sink node to determine the relative timing and temporal ordering of the events across the multiple devices.
- In another example, in accordance with the disclosed elastic timestamping system, mechanisms may be provided to ensure data freshness of the telemetry data. For example, an intermediate device along a network path may be provided with a required or desired data freshness criteria for telemetry data that is used to validate the freshness of collected data before sending it as a response to an intent request. For example, the desired data freshness criteria may be that telemetry data be locally measured within a time frame, for example within the last 10 ms (e.g., when collecting packet loss information). In another example, the data freshness criteria may be that telemetry data be gathered or received within a time period (e.g., the last 15 ms), in cases where the measurements is measured remotely but collected locally at the device (e.g., where measurements are performed at a layer 0 (L0) device but the corresponding data is processed and inserted into the data flow by a layer (L1) device such as at a packet-optical layer boundary device in a packet-optical network). Data freshness criteria may be communicated to a device in a number of ways. For example, the data freshness criteria may be communicated along with the telemetry intent (e.g., in the packet or frame header), or via an out-of-band channel. In another example, the data freshness criteria may be programmed into individual devices. In another example, data freshness may not be stipulated as a requirement. Instead, the device may include information regarding the freshness of the telemetry data in the telemetry response.
- In the following, telemetry applications are described in more detail and examples are given of the disclosed elastic timestamping system and method as used in telemetry applications. Although examples herein refer to telemetry applications, the disclosed elastic timestamping system and methods (including ENTS format, ENTS dimension selection, and mechanisms for data freshness) may be used in any application that uses timestamping, including, but not limited to any computing or network application that requires record keeping, network administration and management applications, computer security applications, concurrency control applications, and synchronization applications.
- Streaming telemetry mechanisms, such as OpenConfig, are designed to streamline the notification of network state by having the network elements stream the telemetry data up to a central management entity where the data gets stored and processed. While streaming telemetry mechanisms employ extensive offline algorithms to process telemetry data, they are not designed to inherently improve the quality of the data collected.
- To improve extensibility and bring flexibility into telemetry data collection, the In-band Network Telemetry (INT) framework was developed for packet networks. INT is implemented in the data plane such that telemetry information is carried in data packets (e.g., in the header of data packets) and can get modified with each hop. The data plane refers to the part of a device's architecture that makes forwarding decisions for incoming packets. For example, routing may be determined by the device using a locally stored table in which the device looks up the destination address of the incoming packet and retrieves the information needed for forwarding.
- The INT framework relies on programmable data planes to bring flexibility to telemetry data collection. Devices with programmable data planes include network processors or general-purpose central processing units (CPUs) at the low end, and data path programmable switch chips at the high end. With INT, a source switch (or more generally, a source network device) incorporates an instruction header to collect network state information as a part of the data packet. Intermediate INT-capable switches (devices) interpret the instruction header and collect and insert the desired network state information in the data packet, which eventually reaches a sink switch and can be used as needed to monitor and evaluate the operation of the network. Advantages of INT include real-time telemetry rates, low CPU and operating system (O/S) overhead, and the flexibility to programmatically instrument packets to carry useful telemetry data. The programmable data planes used in INT have been explicitly designed for packet networks; however extending INT mechanisms into optical networks, where there is no notion of data packets, is far from straightforward due to factors such as layering and the presence of purely analog devices.
- The emergence of integrated packet and optical networks, or “packet-optical networks”, such as those interconnecting data centers, see additional challenges when it comes to network telemetry because of the different types of telemetry data collected in packet versus optical networks. For example, the telemetry data collected in a packet layer of a packet network, such as packet loss and latency, on a per-flow basis cannot be easily attributed to or correlated with data collected in the optical layer of an optical networks, such as bit error rates (BERs) and quality factor (Q-factor). Moreover, the optical network lacks the digital constructs used by telemetry solutions such as INT, and the packet layer does not have access to measurements in the optical network. A further challenge occurs in associating packet flow telemetry data with the corresponding data from optical transport network (OTN) layers, which involves piecing together telemetry data from many devices.
- Optical parameters may affect traffic flows. For example, if a link experiences degradation in Q-without link failure, operators can use the knowledge of that information to proactively move critical applications away from that particular link. Thus, it is useful for network operators to be able to monitor optical parameters over time to use in routing and other applications.
- Thus, the packet-optical in-band telemetry (POINT) framework was developed (as described in U.S. patent application Ser. No. 15/801,526, which is incorporated herein by reference in its entirety) to achieve end-to-end correlation of collected network state data in mixed networks with multiple network layers, such as packet-optical networks.
- According to the POINT framework, a source device inserts an intent (POINT intent) for telemetry data collection along with the data flow. The intent communicates the parameters of data collection such as conditions for data collection, entities being monitored, and the type of data to be collected for that flow. Intermediate devices on that data flow process the high-level intent if it is targeted towards them, translate the intent into a suitable device-specific action for data collection and execute that action to collect an intent response. At a layer boundary, such as packet to optical, or across optical layers such as a hierarchy of optical data units (ODUs), intermediate devices translate the intent and response using a layer-appropriate mechanism. For example, in the packet network, the intent and response may be encapsulated using IP options or VXLAN metadata header. At the packet-optical boundary, the intent can be retrieved from the packet header, and translated and encapsulated as ODU layer metadata, which remain accessible to all nodes along the end-to-end path of the ODU.
- In another example, the POINT intent can be translated into an appropriate query for telemetry data collection via the management plane of the optical devices. As soon as the response of data collection is ready, it is communicated through the optical network and translated appropriately into a packet or packet header at the packet-optical boundary and forwarded to the sink for analysis. For example, the response communication may be out-of-band using the optical supervisory channel (OSC). The POINT framework also supports adding response metadata for incorporating deployment-specific reliability mechanisms.
- Thus, the POINT framework provides hierarchical layering with intent and response translation at each layer boundary, and mapping of the intent to layer-specific data collection mechanism, such that the POINT framework can be deployed across a network layer hierarchy. The POINT framework also provides for fate sharing of telemetry intent and data flow. Telemetry data for a specific data flow can be collected in-band as the data traverses the network layers. By design, intent responses can be out-of-band to accommodate scenarios such as troubleshooting networks when there is no connectivity between the source and the sink. Additionally, intents are high level instructions for data collection and can be mapped to existing data collection mechanisms between two POINT capable intermediate network devices.
-
FIG. 1 is a high-level diagram of anexample POINT framework 100 implemented in an example packet-optical network 102, in accordance with the disclosures herein. The example packet-optical network 102 includespacket devices optical network 104 segment that includesoptical devices POINT framework 100 can operate over anoptical network 104 with Layer0 (L0) and/or Layer-1 (L1) circuits. The packet devices include aPOINT source device 110 and aPOINT sink device 120, as well as packet optical gateways (POGs) 114 and 116 located at the interfaces between the packet segments andoptical network 104 segment of the packet-optical network 102. Thepacket devices POGs L0 devices optical network 102,POGs optical devices - According to the
POINT framework 100, telemetry information for a packet-optical traffic flow 105, such as intent or POINT data (e.g., intent and response), in the packet-optical network 102 is gathered in thedata plane 140 as part of the information carried in thenetwork 102, as described below. Thetelemetry plane 160 represents the telemetry information for the packetoptical flow 105 being mapped and correlated across network layers, constructs (e.g., secure network communications (SNC), label-switched path (LSP), or virtual local area network (ULAN)) and devices operating at different layers in the networking stack to give the end user (e.g., at the POINT sink) a unified view of the operation of the entire packet-optical network 102. - In accordance with the disclosed
POINT framework 100, aPOINT source device 110 may initiate a network telemetry data collection for a packet-optical flow 105 along a packet-optical data path from thesource device 110 to asink device 120. Along the packet-optical data path, POINT intermediate devices, such asPOGs optical devices sink device 120. For example, as packet (frame) 142 traverses the packet-optical network 102 across devices and layers (e.g., packet layers L2/L3 and optical layers L1/L0), in thedata plane 140 intent information is transferred into other layers, translated into device-specific actions, and responses are collected (e.g., added to POINT data in packet 142) for use at thePOINT sink device 120. At thesink device 120, the collected telemetry data for the packet-optical flow 105 (collected fromPOINT source device 110 to POINT sink device 120) is processed as needed by the intended applications. Examples of telemetry data processing may include triggering a report to a management entity (e.g., using mechanisms like OpenConfig) or archiving collected data in a storage device. -
FIG. 2 is a flow diagram of an exampletelemetry processing procedure 200 that meets data freshness requirements and includes ENTS in the telemetry responses, in accordance with the disclosures herein. Thetelemetry processing procedure 200 may be performed in an optical layer by an intermediate optical or packet-optical device in a packet-optical network, although the procedure is similar in a packet layer, as performed by a packet device, involving packets instead of ODU frames. The device may read POINT intent from theODU frame 210, for example from the ODU-L1POINT-OH 214. A POINT intent may include a corresponding data freshness requirement that indicates a maximum time duration for which telemetry data is valid (e.g., Intenti has a 1 minute data freshness requirement, and Intent. has a 200 ms data freshness requirement). - The received intents may be added to the associative map 224 (also referred to as mapping table or table) while responses are generated. The device may check if the response for a particular intent is ready by looking it up in the
associative map 224 that keeps a mapping of intent and corresponding response. At the point in time shown in the example ofFIG. 2 , Response2 and Response3 are available for corresponding Intent2 and Intent3, whereas a response is not yet available for Intenti. The device may also verify that the ENTS of each response (shown in this example to include three fields: (hour: minute: second)) meets the data freshness requirements for the corresponding intent. If a response is ready and meets the corresponding data freshness requirement, then the intent and response may be added to theODU frame 210, for example into the ODU-L1POINT-OH 214. However, if the response is not ready or does not meet the data freshness requirement, the intent may be added to anintent queue 220 for intent translation and processing. Once the corresponding response is determined to be ready (e.g., Response2 for Intent2), it may be added to theassociative map 224, the corresponding intent may be removed from theintent queue 220, translated, processed, and inserted in theassociative map 224 to be inserted with the response in an ODU frame 210 (in this case a subsequent ODU frame from when the corresponding intent was received) for downstream forwarding. In an alternate example, the response may be communicated via an out-of-band channel. - ENTS in telemetry applications may be used to provide an ordering of events on the same device. For example, the ENTS dimensions and value within each dimension may be used to order events, for example in increasing or decreasing order in time (“dictionary order”). ENTS in telemetry applications may be used to provide an ordering of network events across different devices. For example, ENTs may be used to provide ordering of telemetry data within a telemetry (e.g., POINT) flow.
- It may be the case that an ENTS value pertaining to an event is not the exact time value of the event. For example, the event may occur in hardware whereas the recording of the ENTS value may occur in software, such that there may be some delay between the event and the ENTS recording. Thus, in an example, latency bounds for local ENTS values may be generated, and may be provided to the sink node to provide a bound on the accuracy of the ENTS value and may be particularly useful when evaluating data freshness. In another example, ENTS values may be treated as logical clock values. ENTS in telemetry applications may be used to ensure data freshness, for example by communicating data freshness requirements (e.g., with intent data) as described herein.
-
FIG. 3 is a flow diagram of an exampleelastic timestamping procedure 300 with event freshness for use in a computing or networking system, in accordance with the disclosures herein. Theelastic timestamping procedure 300 may be performed by an entity or device that is part of the computing or network system. At 302, an elastic ENTS is generated for a corresponding event or action in the system for which timing or temporal order information is needed. At 304, the ENTS is evaluated to verify if the event meets a freshness threshold. - If the event does not meet the freshness threshold, then at 306 the event and corresponding ENTS may be disregarded (e.g., data discarded). Otherwise, if the event meets the freshness threshold, at 308, a subset of the dimensions of ENTS are selected to generate a compact ENTS by determining the relevant time granularity for the corresponding event. At 310, the compact ENTS is communicated (e.g., reported to a receiving entity for processing, recorded to storage, displayed to an operator, etc.), thus achieving reduced overhead.
- In an example, the disclosed elastic timestamping methods and system, and any subset or one or more component(s) thereof, may be implemented using software and/or hardware and may be partially or fully implemented by computing devices, such as the
computing device 400 shown inFIG. 4 . -
FIG. 4 is a block diagram of acomputing system 400 in which one or more disclosed embodiments may be implemented. Thecomputing system 400 may include, for example, a computer, a switch, a router, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Thecomputing device 400 may include aprocessor 402, amemory 404, astorage 406, one ormore input devices 408, and/or one ormore output devices 410. Theinput devices 408 andoutput devices 410 may be generally referred to as interfaces for thecomputing device 400. Thedevice 400 may include aninput driver 412 and/or anoutput driver 414. Thedevice 400 may include additional components not shown inFIG. 4 . - The
processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core may be a CPU or a GPU. Thememory 404 may be located on the same die as theprocessor 402, or may be located separately from theprocessor 402. Thememory 404 may include a volatile or non-volatile memory, for example, random access memory (RAM), dynamic RAM, or a cache. - The
storage 406 may include a fixed or removable storage, for example, a hard disk drive, a solid state drive, an optical disk, or a flash drive. Theinput devices 408 may include a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals). Theoutput devices 410 may include a display, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmission and/or reception of wireless IEEE 802 signals). - The
input driver 412 may communicate with theprocessor 402 and theinput devices 408, and may permit theprocessor 402 to receive input from theinput devices 408. Theoutput driver 414 may communicate with theprocessor 402 and theoutput devices 410, and may permit theprocessor 402 to send output to theoutput devices 410. The output driver 416 may include an accelerated processing device (“APD”) 416 which may be coupled to adisplay device 418. The APD may be configured to accept compute commands and graphics rendering commands fromprocessor 402, to process those compute and graphics rendering commands, and to provide pixel output to displaydevice 418 for display. - In an example, with reference to
FIG. 1 , thepoint source 110,packet devices 112ad 118, optical devices 122-132, and/orPOGs 114, may be implemented, at least in part, with the components ofcomputing device 400 shown inFIG. 4 . Similarly, the procedures shown inFIGS. 2 and 3 may be implemented, at least in part, with the components ofcomputing device 400 shown inFIG. 4 . - It should be understood that many variations are possible based on the disclosure herein. Although features and elements are described above in particular combinations, each feature or element may be used alone without the other features and elements or in various combinations with or without other features and elements.
- The methods and elements disclosed herein may be implemented in/as a general purpose computer, a processor, a processing device, or a processor core. Suitable processing devices include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine. Such processors may be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediary data including netlists (such instructions capable of being stored on a computer readable media). The results of such processing may be maskworks that are then used in a semiconductor manufacturing process to manufacture a processor which implements aspects of the embodiments.
- The methods, flow charts and elements disclosed herein may be implemented in a computer program, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general purpose computer or a processor. Examples of non-transitory computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/850,502 US20190013954A1 (en) | 2017-07-05 | 2017-12-21 | Elastic timestamping |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762528964P | 2017-07-05 | 2017-07-05 | |
US15/850,502 US20190013954A1 (en) | 2017-07-05 | 2017-12-21 | Elastic timestamping |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190013954A1 true US20190013954A1 (en) | 2019-01-10 |
Family
ID=64902985
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/801,526 Active 2037-12-20 US10341748B2 (en) | 2017-07-05 | 2017-11-02 | Packet-optical in-band telemetry (POINT) framework |
US15/840,606 Active 2037-12-21 US10455303B2 (en) | 2017-07-05 | 2017-12-13 | Packet-optical in-band telemetry (POINT) flow tracing and proof-of-transit |
US15/850,502 Abandoned US20190013954A1 (en) | 2017-07-05 | 2017-12-21 | Elastic timestamping |
US15/947,198 Abandoned US20190014395A1 (en) | 2017-07-05 | 2018-04-06 | Reliable telemetry |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/801,526 Active 2037-12-20 US10341748B2 (en) | 2017-07-05 | 2017-11-02 | Packet-optical in-band telemetry (POINT) framework |
US15/840,606 Active 2037-12-21 US10455303B2 (en) | 2017-07-05 | 2017-12-13 | Packet-optical in-band telemetry (POINT) flow tracing and proof-of-transit |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/947,198 Abandoned US20190014395A1 (en) | 2017-07-05 | 2018-04-06 | Reliable telemetry |
Country Status (1)
Country | Link |
---|---|
US (4) | US10341748B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10999216B2 (en) * | 2018-07-13 | 2021-05-04 | EMC IP Holding Company LLC | Resource allocation and provisioning in a multi-tier edge-cloud virtualization environment |
US11356342B2 (en) | 2020-01-16 | 2022-06-07 | Cisco Technology, Inc. | Methods and apparatus for optimizing bandwidth consumption in support of intense network-wise health assessment |
Families Citing this family (38)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9542391B1 (en) | 2013-11-11 | 2017-01-10 | Amazon Technologies, Inc. | Processing service requests for non-transactional databases |
US10599753B1 (en) | 2013-11-11 | 2020-03-24 | Amazon Technologies, Inc. | Document version control in collaborative environment |
US11336648B2 (en) | 2013-11-11 | 2022-05-17 | Amazon Technologies, Inc. | Document management and collaboration system |
US10691877B1 (en) | 2014-02-07 | 2020-06-23 | Amazon Technologies, Inc. | Homogenous insertion of interactions into documents |
US9942631B2 (en) * | 2015-09-25 | 2018-04-10 | Intel Corporation | Out-of-band platform tuning and configuration |
US10656987B1 (en) * | 2017-04-26 | 2020-05-19 | EMC IP Holding Company LLC | Analysis system and method |
US10341748B2 (en) * | 2017-07-05 | 2019-07-02 | Infinera Corporation | Packet-optical in-band telemetry (POINT) framework |
US10764148B2 (en) * | 2017-11-29 | 2020-09-01 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for network traffic statistics collection |
ES2950584T3 (en) * | 2018-02-19 | 2023-10-11 | Telefonica Sa | Procedure and system to validate the ordered transit test of traffic packets on a network |
US20190260657A1 (en) * | 2018-02-21 | 2019-08-22 | Cisco Technology, Inc. | In-band performance loss measurement in ipv6/srv6 software defined networks |
US11184235B2 (en) * | 2018-03-06 | 2021-11-23 | Cisco Technology, Inc. | In-band direct mode performance loss measurement in software defined networks |
US10805215B2 (en) * | 2018-03-20 | 2020-10-13 | Cisco Technology, Inc. | Intra-host and end-to-end packet path and treatment tracing using in-situ OAM in container networking architecture |
US20190297017A1 (en) * | 2018-03-23 | 2019-09-26 | Cisco Technology, Inc. | Managing network congestion using segment routing |
US10771182B2 (en) * | 2018-04-25 | 2020-09-08 | Cisco Technology, Inc. | Enhancing routing metrics |
US12034593B1 (en) | 2018-07-10 | 2024-07-09 | Cable Television Laboratories, Inc. | Systems and methods for advanced core network controls |
US20200021490A1 (en) * | 2018-07-10 | 2020-01-16 | Cable Television Laboratories, Inc | Systems and methods for advanced core network controls |
US11968548B1 (en) | 2018-07-10 | 2024-04-23 | Cable Television Laboratories, Inc. | Systems and methods for reducing communication network performance degradation using in-band telemetry data |
US10880197B2 (en) | 2018-07-13 | 2020-12-29 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for testing a network node using source code for programming a packet forwarding plane of the network node |
US20200067792A1 (en) * | 2018-08-21 | 2020-02-27 | Argela Yazilim Ve Bilisim Teknolojileri San Ve Tic A S | System and method for in-band telemetry target selection |
US10686671B1 (en) * | 2018-11-05 | 2020-06-16 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for testing network elements of an in-band network telemetry capable network |
US11438371B2 (en) | 2018-11-09 | 2022-09-06 | Cisco Technology, Inc. | Distributed denial of service remediation and prevention |
US11323340B2 (en) * | 2019-01-07 | 2022-05-03 | Vmware, Inc. | Packet flow monitoring in software-defined networking (SDN) environments |
US11032151B1 (en) | 2019-02-06 | 2021-06-08 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for providing dynamically configurable, distributed network visibility device |
US10733088B1 (en) | 2019-03-01 | 2020-08-04 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for testing a network node or a related application programming interface using source code metadata |
CN110048912B (en) * | 2019-04-26 | 2022-07-15 | 中国科学技术大学 | Photoelectric cross-layer network monitoring system, data processing method and device |
US11093376B2 (en) | 2019-06-19 | 2021-08-17 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for configuring a test system using source code of a device being tested |
US11563771B2 (en) | 2019-11-25 | 2023-01-24 | Cisco Technology, Inc. | Network telemetry collection with packet metadata filtering |
CN111147403B (en) * | 2019-12-27 | 2021-08-31 | 苏州盛科通信股份有限公司 | Message processing method and device, storage medium and electronic device |
US11258684B2 (en) | 2020-01-09 | 2022-02-22 | Arista Networks, Inc. | Interval flow-based inband telemetry |
US11425020B2 (en) | 2020-06-01 | 2022-08-23 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for storage, retrieval, and use of programmable pipeline device profiles |
US11474823B2 (en) | 2020-06-15 | 2022-10-18 | Keysight Technologies, Inc. | Methods, systems, and computer readable media for on-demand, on-device compiling and use of programmable pipeline device profiles |
US11212219B1 (en) * | 2020-06-26 | 2021-12-28 | Lenovo Enterprise Solutions (Singapore) Pte. Ltd. | In-band telemetry packet size optimization |
US11258719B1 (en) | 2020-08-24 | 2022-02-22 | Keysight Technologies, Inc. | Methods, systems and computer readable media for network congestion control tuning |
CN112637705B (en) * | 2020-11-26 | 2022-05-27 | 新华三技术有限公司合肥分公司 | Method and device for forwarding in-band remote measurement message |
CN112491489B (en) * | 2020-11-27 | 2022-07-29 | 清华大学 | Method, device and system for time synchronization based on in-band telemetry |
FI4037204T3 (en) * | 2021-01-29 | 2024-04-23 | Jio Platforms Ltd | System and method for automated identification of fiber capacity |
US11956136B2 (en) * | 2021-03-24 | 2024-04-09 | Arista Networks, Inc. | System and method for scalable and accurate flow rate measurement |
US11757769B1 (en) * | 2022-05-13 | 2023-09-12 | Arista Networks, Inc. | End-to-end path detection and management for inter-branch communication in a wide area network (WAN) |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050120115A1 (en) * | 2003-12-02 | 2005-06-02 | Alcatel | Tracing active connection modify failures |
WO2007145552A1 (en) * | 2006-06-15 | 2007-12-21 | Telefonaktiebolaget Lm Ericsson (Publ) | Name-address management in communication networks |
KR20130085543A (en) * | 2011-12-19 | 2013-07-30 | 한국전자통신연구원 | Method and system for domain based packet forwarding |
US8996533B2 (en) * | 2012-06-21 | 2015-03-31 | Breakingpoint Systems, Inc. | Systems and methods multi-key access to data |
WO2014179533A1 (en) * | 2013-05-01 | 2014-11-06 | Adc Telecommunications, Inc. | Enhanced route tracing |
US9413634B2 (en) * | 2014-01-10 | 2016-08-09 | Juniper Networks, Inc. | Dynamic end-to-end network path setup across multiple network layers with network service chaining |
US9633100B2 (en) * | 2014-01-15 | 2017-04-25 | Dell Products, L.P. | System and method for data structure synchronization |
US10198325B2 (en) * | 2016-05-24 | 2019-02-05 | Mastercard International Incorporated | Method and system for desynchronization recovery for permissioned blockchains using bloom filters |
US10367733B2 (en) * | 2017-03-30 | 2019-07-30 | Nicira, Inc. | Identifier-based virtual networking |
US10341748B2 (en) * | 2017-07-05 | 2019-07-02 | Infinera Corporation | Packet-optical in-band telemetry (POINT) framework |
-
2017
- 2017-11-02 US US15/801,526 patent/US10341748B2/en active Active
- 2017-12-13 US US15/840,606 patent/US10455303B2/en active Active
- 2017-12-21 US US15/850,502 patent/US20190013954A1/en not_active Abandoned
-
2018
- 2018-04-06 US US15/947,198 patent/US20190014395A1/en not_active Abandoned
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10999216B2 (en) * | 2018-07-13 | 2021-05-04 | EMC IP Holding Company LLC | Resource allocation and provisioning in a multi-tier edge-cloud virtualization environment |
US11356342B2 (en) | 2020-01-16 | 2022-06-07 | Cisco Technology, Inc. | Methods and apparatus for optimizing bandwidth consumption in support of intense network-wise health assessment |
US11870665B2 (en) | 2020-01-16 | 2024-01-09 | Cisco Technology, Inc. | Methods and apparatus for optimizing bandwidth consumption in support of intense network-wise health assessment |
Also Published As
Publication number | Publication date |
---|---|
US20190014394A1 (en) | 2019-01-10 |
US20190014036A1 (en) | 2019-01-10 |
US10455303B2 (en) | 2019-10-22 |
US20190014395A1 (en) | 2019-01-10 |
US10341748B2 (en) | 2019-07-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20190013954A1 (en) | Elastic timestamping | |
US10516551B2 (en) | In-band telemetry with limited extra bytes | |
US20200336360A1 (en) | Triggered in-band operations, administration, and maintenance in a network environment | |
JP5646090B2 (en) | Traceroute_delay diagnostic command | |
KR101911579B1 (en) | Controller driven oam for openflow | |
US9450846B1 (en) | System and method for tracking packets in a network environment | |
US9847922B2 (en) | System and method for continuous measurement of transit latency in individual data switches and multi-device topologies | |
KR101597255B1 (en) | Performing a time measurement in a communication network | |
CN109617743B (en) | Network performance monitoring and service testing system and testing method | |
CN106789430B (en) | A kind of point-to-point link fault detection method | |
US20230171175A1 (en) | Real-time network-wide link latency monitoring with in-network int sampling and aggregation | |
US20240195519A1 (en) | Tsn operation management system with time capture location protocol | |
Marques et al. | Intsight: Diagnosing slo violations with in-band network telemetry | |
Wu et al. | Lossdetection: Real-time packet loss monitoring system for sampled traffic data | |
Kompella et al. | Router support for fine-grained latency measurements | |
Liu et al. | SFANT: A SRv6-Based Flexible and Active Network Telemetry Scheme in Programming Data Plane | |
US11483122B2 (en) | Time transfer using passive tapping | |
Gao et al. | Xshot: Light-weight link failure localization using crossed probing cycles in SDN | |
JP5997062B2 (en) | Network system and network management device | |
Kernen et al. | Monitoring and Analysis of SMPTE ST 2059-2 PTP Networks and Media Devices | |
Moradi et al. | On time-stamp accuracy of passive monitoring in a container execution environment | |
Kannan | Improving Network Diagnostics Using Programmable Networks | |
CN112312228B (en) | Method, device and storage medium for detecting medium transmission quality index | |
Wang et al. | Enabling Network Diagnostics in Time-Sensitive Networking: Protocol, Algorithm, and Hardware | |
CN116094962A (en) | Network transmission delay measurement method for any two nodes in programmable network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INFINERA CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ANAND, MADHUKAR;SUBRAHMANIAM, RAMESH;VALIVETI, RADHAKRISHNA;SIGNING DATES FROM 20171213 TO 20171218;REEL/FRAME:044463/0693 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |