WO2013167977A1 - Synchronisation d'horloge à l'aide de multiples chemins de réseau - Google Patents

Synchronisation d'horloge à l'aide de multiples chemins de réseau Download PDF

Info

Publication number
WO2013167977A1
WO2013167977A1 PCT/IB2013/001506 IB2013001506W WO2013167977A1 WO 2013167977 A1 WO2013167977 A1 WO 2013167977A1 IB 2013001506 W IB2013001506 W IB 2013001506W WO 2013167977 A1 WO2013167977 A1 WO 2013167977A1
Authority
WO
WIPO (PCT)
Prior art keywords
path
time synchronization
clock
network
address
Prior art date
Application number
PCT/IB2013/001506
Other languages
English (en)
Inventor
Tal Mizrahi
Original Assignee
Marvell Israel (M.I.S.L.) Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/764,732 external-priority patent/US9172485B2/en
Application filed by Marvell Israel (M.I.S.L.) Ltd. filed Critical Marvell Israel (M.I.S.L.) Ltd.
Priority to CN201380024941.2A priority Critical patent/CN104584467B/zh
Publication of WO2013167977A1 publication Critical patent/WO2013167977A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04J—MULTIPLEX COMMUNICATION
    • H04J3/00—Time-division multiplex systems
    • H04J3/02—Details
    • H04J3/06—Synchronising arrangements
    • H04J3/0635—Clock or time synchronisation in a network
    • H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
    • H04J3/0658—Clock or time synchronisation among packet nodes
    • H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
    • H04J3/0664—Clock or time synchronisation among packet nodes using timestamps unidirectional timestamps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04J—MULTIPLEX COMMUNICATION
    • H04J3/00—Time-division multiplex systems
    • H04J3/02—Details
    • H04J3/06—Synchronising arrangements
    • H04J3/0635—Clock or time synchronisation in a network
    • H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
    • H04J3/0658—Clock or time synchronisation among packet nodes
    • H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
    • H04J3/0667—Bidirectional timestamps, e.g. NTP or PTP for compensation of clock drift and for compensation of propagation delays

Definitions

  • the present disclosure relates generally to devices that maintain a clock and, more particularly, to techniques for synchronizing clocks across a network.
  • Clock synchronization protocols are commonly used in packet-based networks to synchronize clocks maintained at different network devices. In such clock
  • a first network device which maintains a master clock
  • the second network device utilizes the master clock time and an estimated network latency to adjust the slave clock in order to synchronize the slave clock with the master clock.
  • a network device comprises one or more ports coupled to a network, and a time synchronization module.
  • the time synchronization module is configured to process (i) respective path information, and (ii) respective time
  • a method includes receiving a plurality of time synchronization packets from a master clock device over two or more different communication paths via one or more ports coupled to a network, wherein each time synchronization packet includes (i) respective path information, and (ii) respective time synchronization information, and wherein the respective path information indicates a respective communication path in the network via which the respective time
  • the method also includes, for each of at least some of the plurality of time synchronization packets, processing (i) respective path
  • the method includes determining a system time clock responsive to the processing of the path information and the time synchronization information included in the at least some of the plurality of time synchronization packets.
  • a system comprises a first network device, a second network device communicatively coupled to the first network device via a
  • the first network device includes one or more first ports coupled to the communication network, and a master clock module.
  • the master clock module is configured to generate a plurality of time synchronization packets, each time synchronization packet having (i) respective path information, and (ii) respective time synchronization information, wherein the respective path information indicates a respective communication path in the communication network via which the respective time synchronization packet is to be transmitted, and cause the plurality of time synchronization packets to be transmitted (i) via at least one of the one or more first ports, and (ii) via multiple different communication paths in the communication network.
  • the time synchronization module is configured to process (i) respective path information, and (ii) respective time synchronization information included in each of at least some of the plurality of time synchronization packets transmitted by the master clock device via at least one of the one or more ports, and determine a system time clock responsive to the processing of the path information and the time synchronization information included in the at least some of the plurality of time synchronization packets.
  • Fig. 1 A is a block diagram of an example system in which a slave clock is maintained based on a master clock and based on signals transmitted by the master clock via multiple paths in a network, according to an embodiment.
  • Fig. IB is a block diagram of another example system in which a slave clock is maintained based on a master clock and based on signals transmitted by the master clock via multiple paths in a network, according to an embodiment.
  • FIGs. 2A and 2B are a flow diagram of an example method for maintaining a slave clock using multiple clock signals received from a master clock via multiple paths in a network, according to an embodiment.
  • Fig. 3 is a timing diagram illustrating signals exchanged via multiple paths between a slave clock and a master clock, according to an embodiment.
  • Fig. 4 is a flow diagram of an example method for maintaining a slave clock using multiple clock signals received from a master clock via multiple paths in a network, according to an embodiment.
  • Fig. 5 is a flow diagram of an example method for maintaining a slave clock using a packet received from a master clock, according to an embodiment.
  • Liwioj rig. o is a nuw uiagram ui an example iiieuiuu lur laciiiiaung ueieniimmg CIUC values across a network, according to an embodiment.
  • Fig. 7 is a flow diagram of another example method for facilitating determining clock values across a network, according to an embodiment.
  • Fig. 8 A is a block diagram of another example system in which a slave clock is maintained based on a master clock and based on signals transmitted by the master clock via multiple paths in a network, according to an embodiment.
  • Fig. 8B is a block diagram of another example system in which a slave clock is maintained based on a master clock and based on signals transmitted by the master clock via multiple paths in a network, according to an embodiment.
  • Fig. 9 is a flow diagram of another example method for facilitating determining clock values across a network, according to an embodiment.
  • Fig. 1 A is a block diagram of an example system 100, according to an embodiment, having a first network device 104 communicatively coupled to a second network device 108 via a communication network 112.
  • the first network device 104 implements a master clock, and the first network device 104 is sometimes referred to herein as the master clock device 104.
  • the second network device 108 implements a slave clock, and the second network device 108 is sometimes referred to herein as the slave clock device 108.
  • the master clock device 104 includes a clock module 120 configured to maintain the master clock.
  • the master clock is a time-of-day clock.
  • the master clock is a counter not directly linked to a time-of-day.
  • the clock module 120 includes or utilizes a counter circuit that maintains the master clock.
  • the clock module 120 is an application layer module operating at an application layer in a multi-layer communication protocol siau . in an eiiiDuuiineni, ine iiiuiu-iayer cummumcaiiun pruiucui siac ⁇ e.g., me
  • Transport Control Protocol/Internet Protocol (TCP/IP) protocol suite comprises a link layer, an Internet layer, a transport layer, and the application layer, where the application layer is above the transport layer, the Internet layer, and the link layer in the protocol stack.
  • the multi-layer communication protocol stack e.g., corresponding to the Open Systems Interconnection (OSI) model
  • OSI Open Systems Interconnection
  • the clock module 120 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack.
  • the clock module 120 is configured to cause a plurality of time-stamped packets to be transmitted from the master clock device 104 to the slave clock device 108 via respective different communication paths through the network 112.
  • the plurality of time-stamped packets are time-stamped with one or more values of the master clock maintained by the clock module 120, in an embodiment.
  • the network 112 comprises one or more of a private network, a public network, a backbone network, a provider network, the Internet, etc.
  • the network 112 comprises a plurality of network switches and/or routers 124 (referred to hereinafter as switches merely for brevity), in an embodiment, and a first path 128 corresponds to a link from the switch 124a to the switch 124b via the switch 124c.
  • a second path 132 corresponds to a link from the switch 124d to the switch 124e via the switch 124f.
  • the clock module 120 is configured to cause a first time-stamped packet to be transmitted from the master clock device 104 to the slave clock device 108 via the path 128, and to cause a second time-stamped packet to be transmitted from the master clock device 104 to the slave clock device 108 via the path 132.
  • the master CIUC uevice I V mciuues iiiuiupie puns I H-W, anu me UIUCK iiiuuuuie u is unngureu ⁇ cause the first time-stamped packet to be transmitted via the port 140a and the second time-stamped packet to be transmitted via the port 140b.
  • each of at least some of the time-stamped packets are transmitted via the same port 140 (not shown in Fig. 1A).
  • the master clock device 104 includes another suitable number of ports 140 (e.g., only one port or more than two ports).
  • the master clock device 104 includes only a single port 140, and path diversity is entirely provided by the network 112.
  • switches/routers can be configured to utilize different paths when transmitting packets destined for the same endpoint device but including different header information (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • public and/or provider networks at least some switches/routers in the network 112 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • the master clock device 104 varies header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • two or more of the ports 140 are coupled to a single switch 124.
  • at least some of the different network paths pass through different ports of the single switch 124 coupled to the two or more ports 140 of the master clock device 104.
  • switches/routers can be configured to utilize different paths when transmitting packets destined for the same endpoint device but including different header ln commeauon ⁇ e.g., source auuresses, uesunauon auuresses, .Layer pons, eic anu or when the packets are received via different physical ports of the switch/router, in some embodiments.
  • At least some switches/routers in the network 112 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.) and/or based on the physical ports of the switch router via which the packets are received, in some embodiments.
  • load balancing techniques in which packets destined for the same endpoint device are directed along different paths based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.) and/or based on the physical ports of the switch router via which the packets are received, in some embodiments.
  • the master clock device 104 varies header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.) and/or transmits packets via different physical ports of the master clock device 104, where the different physical ports of the master clock device 104 are respectively coupled to different physical ports of the switch 124, in some embodiments.
  • a clock synchronization module 150 of the slave clock device 108 is configured to determine different clock results corresponding to the plurality of time-stamped packets transmitted by the master clock device 104, and corresponding to the plurality of different communication paths through the network 112. For example, the clock synchronization module 150 is configured to determine a first clock result corresponding to the first time-stamped packet transmitted via the first path 128, and a second clock result corresponding to the second time-stamped packet transmitted via the second path 132.
  • the slave clock device 108 includes multiple ports 154, and the first time-stamped packet is received via the port 154a and the second time-stamped packet is received via the port 154b.
  • each of at least some of the time-stamped packets are received via the same port 154 (not shown in Fig. 1A).
  • the slave clock device 108 includes another suitable number of ports 154 (e.g., only one or more than two).
  • the slave clock device 108 includes only a single port 154.
  • each clock result generated by the clock synchronization module comprises an offset between the master clock maintained at the master clock device 104 and a slave clock maintained at the slave clock device 108.
  • the clock synchronization module 150 is configured to determine each offset based on (i) the respective time stamp in the respective time-stamped packet transmitted via the respective path, and (ii) a value of the slave clock at a respective time at which the respective time-stamped packet was received by the slave clock device 108, in an embodiment.
  • the clock synchronization module 150 determines a first offset based on (i) a first time stamp in the first time- stamped packet transmitted via the first path 128, and (ii) a first value of the slave clock at a first time at which the first time-stamped packet was received by the slave clock device 108; and the clock synchronization module 150 determines a second offset based on (i) a second time stamp in the second time-stamped packet transmitted via the second path 132, and (ii) a second value of the slave clock at a second time at which the second time-stamped packet was received by the slave clock device 108.
  • the synchronization module 150 comprises a clock value (e.g., a time-of-day value) corresponding to the master clock maintained at the master clock device 104.
  • the clock synchronization module 150 is configured to determine each clock value based on the respective time stamp in the respective time-stamped packet transmitted via the respective path, in an embodiment.
  • the clock synchronization module 150 determines a first clock value based on the first time stamp in the first time-stamped packet transmitted via the first path 128, and determines a second clock value based on the second time stamp in the second time-stamped packet transmitted via the second path 132.
  • the clock synchronization module 150 is an application layer module operating at the application layer in me iiiuiu-iayer cuininumeaiion pruiocoi siac . m oilier eiiiDuuiiiiems, me CIUC synchronization module 150 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack.
  • the clock synchronization module 150 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack.
  • synchronization module 150 is at another suitable layer below an application layer and/or operates at a suitable layer below the fourth layer in the multi-layer communication protocol stack.
  • the clock synchronization module 150 is configured to maintain, for each path, a respective path time data set that includes a respective path-dependent clock, or information for generating the respective clock result such as a respective path delay measurement, a respective clock offset, etc., in some embodiments.
  • the clock synchronization module 150 uses the path time data sets to generate the clock results.
  • the clock results are the path time data sets or are included in the path time data sets.
  • the clock synchronization module 150 includes or utilizes respective counter circuits that maintain the different path-dependent clocks.
  • a clock module 160 in the slave clock device 108 is configured to maintain the slave clock, in some embodiments.
  • the slave clock is a time-of-day clock.
  • the slave clock is a counter not directly linked to a time- of-day.
  • the clock module 160 includes or utilizes a counter circuit that maintains the slave clock.
  • the clock module 160 is an application layer module operating at the application layer in the multi-layer communication protocol stack. In other embodiments, the clock module 160 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack. In some
  • the clock module 160 is configured to select one of the path-dependent clocks as the slave clock rather than maintaining a separate slave clock.
  • liiujjj i ne CIUC inouuie ⁇ is uonngureu LU ueienmne me siave CIOC vaiue using me plurality of clock results generated by the clock synchronization module 150, in some embodiments.
  • the clock module 160 is configured to determine the slave clock value at least by mathematically combining the plurality of clock results generated by the clock synchronization module 150.
  • the clock module 160 is configured to determine the slave clock value at least by selecting one of the plurality of clock results generated by the clock synchronization module 150. For example, one of the clock results is determined to be most accurate based on
  • the clock module 160 is configured to determine the slave clock value at least by (i) combining at least some of the plurality of clock results generated by the clock synchronization module 150 in some situations, and (ii) selecting one of the plurality of clock results generated by the clock synchronization module 150 in other situations.
  • the clock module 160 is configured to determine the slave clock value at least by (i) selecting a subset of the plurality of clock results generated by the clock synchronization module 150, and (ii) combining the subset of clock results.
  • the selected subset is the clock result(s) that are determined to be suitably accurate based on independently determined accuracy metrics for each path such as path length, path delay variability (or noisiness) etc., in some embodiments.
  • the clock module 160 is configured to determine the slave clock value at least by mathematically combining two or more of the plurality of offsets to generate a combined offset, and then use the combined offset to set the slave clock value. In some embodiments in which the clock results generated by the clock synchronization module 150 comprise offsets, the clock module 160 is configured to select one of the plurality of offsets, and then use the selected offset to set the slave clock value.
  • one of the offsets is determined to be most accurate based on lnuepenuenuy ueiemimeu accuracy meirics lur eacn pain suuii as pain iengui, pain ueiay variability (or noisiness) etc., in some embodiments.
  • the clock module 160 is configured to, (i) in some situations, mathematically combine the plurality of offsets to generate the combined offset, and then use the combined offset to set the slave clock value, and (ii) in other situations, select one of the plurality of offsets, and then use the selected offset to set the slave clock value.
  • the clock module 160 is configured to determine the slave clock value at least by (i) selecting a subset of the plurality offsets, and (ii) combining the subset of offsets.
  • the selected subset is the offset(s) that are determined to be suitably accurate based on independently determined accuracy metrics for each path such as path length, path delay variability (or noisiness) etc., in some embodiments.
  • the clock module 160 is configured to determine the slave clock value at least by mathematically combining two or more of the different path clock values to generate a combined clock value, and then use the combined clock value to set the slave clock value.
  • the clock module 160 is configured to select one of the path clock values, and then use the selected path clock value to set the slave clock value. For example, one of the clock values is determined to be most accurate based on independently determined accuracy metrics for each path such as path length, path delay variability (or noisiness) etc., in some embodiments.
  • the clock module 160 is configured to, (i) in some situations, mathematically combine at least some of the path clock values to generate the combined clock value, and then use the combined clock value to set the slave clock vaiue, anu yii) m ouier siiuauuns, seieci une ui me pain CIUC values, anu men use me selected path clock value to set the slave clock value.
  • the clock module 160 is configured to calculate an average of at least some of the clock results generated by the clock synchronization module 150, and to determine the slave clock value based on the average clock result. In some embodiments in which the clock results generated by the clock synchronization module 150 comprise offsets, the clock module 160 is configured to calculate an average of at least some of the offsets, and to determine the slave clock value based on the average offset. In some embodiments in which the clock results generated by the clock
  • the synchronization module 150 comprise different clock values corresponding to different paths (path clock values), the clock module 160 is configured to calculate an average of at least some of the path clock values, and to determine the slave clock value based on the average clock value.
  • the clock module 160 is configured to determine the slave clock value at least by (i) selecting a subset of the plurality path clock values, and (ii) combining the subset of path clock values.
  • the selected subset is the path clock value(s) that are determined to be suitably accurate based on independently determined accuracy metrics for each path such as path length, path delay variability (or noisiness) etc., in some embodiments.
  • the clock module 160 is configured to calculate a weighted average of at least some of the clock results.
  • weights utilized in calculating the weighted average are determined based on independently determined accuracy metrics for each path such as path length, path delay variability (or noisiness) etc.,. For example, relatively lesser weight is given to a path with a relatively longer length, whereas relatively greater weight is given to a path with a relatively shorter length, in an embodiment.
  • relatively lesser weight is given to a path with a relatively higher variability (or noisiness) in path delay
  • relatively greater weigin is given io a pain wiui a reiauveiy luwer vanaDiiuy ⁇ ur noismess m pain ueiay, m an embodiment.
  • the clock synchronization module 150 includes a path delay variability determination module 170 configured to generate respective measures of path delay variability for the different paths through the network 112.
  • the clock synchronization module 150 is configured to utilize the measures of path delay variability for the different paths through the network 112 to determine the different weights utilized in calculating the weighted average. For example, relatively lesser weight is given to a path with a relatively higher variability (or noisiness) in path delay, whereas relatively greater weight is given to a path with a relatively lower variability (or noisiness) in path delay, in an embodiment.
  • the path delay variability determination module 170 is configured to generate respective path delay measures for the different paths through the network 112.
  • each path delay measure is a measure of a one-way delay from the master clock device 104 to the slave clock device 108 via the respective path.
  • the path delay variability determination module 170 is configured to generate, for each path, the one-way delay by (i) determining a round-trip delay from the slave clock device 108 to the master clock device 104 and back to the slave clock device 108 via the respective path, and (ii) calculating the one-way delay based on the round trip delay.
  • the one-way delay is calculated as one half of the round-trip delay.
  • the path delay variability determination module 170 is configured to generate respective average path delays for the different paths through the network 112. In an embodiment, the path delay variability determination module 170 is configured to generate, for each path, the respective measure of path delay variability based on the respective average path delay and a respective recent (e.g., most recent) path delay measure. In an embodiment, the path delay variability determination module 170 is cuniigureu ⁇ generate, lor eacn pain, ine respective measure ⁇ pain ueiay variaDiiiiy based on calculating a difference between the respective average path delay and the respective recent (e.g., most recent) path delay measure.
  • the magnitude of the difference calculated for a path generally indicates a degree of variability in the path delay for the path, in an embodiment. For example, if a magnitude of a first difference calculated for a first path is greater than a magnitude of a second difference calculated for a second path, it is assumed that the path delay in the first path has a greater degree of variability than the path delay in the second path, in an embodiment.
  • the clock module 160 is configured to select one of the offsets based on measures of path delay variability for the different paths, and then use the selected offset to set the slave clock value. For example, in an embodiment, the clock module 160 is configured to select an offset corresponding to a path with a lowest measure of path delay variability, and then use the selected offset to set the slave clock value.
  • the clock module 160 is configured to select a set of the offsets that are each associated with a measure of path delay variability that meets one or more criteria, and to determine an offset using the set of selected offsets. For example, in an embodiment, the clock module 160 is configured to select a set of the offsets that are each associated with a measure of path delay variability that meets a threshold (e.g., is less than the threshold, is less than or equal to the threshold, etc.), and to determine an offset using the set of selected offsets.
  • a threshold e.g., is less than the threshold, is less than or equal to the threshold, etc.
  • the clock module 160 is configured to calculate an average of the selected offsets, and to determine the slave clock value using the average offset.
  • paths with high variability are not considered when calculating the average such that only paths with a measured variability less than a suuaoie vanaDiniy inresnuiu are useu wnen calcul ting me average, in an einDuuiiiieni, if there is only one path with a measured variability less than the suitable variability threshold, the corresponding one offset is used to set the slave clock value (i.e., an average of multiple offsets is not calculated).
  • the clock module 160 is configured to set the slave clock using the selected offset.
  • the clock module 160 is configured to select one of the path clock values based on measures of path delay variability for the different paths, and then use the selected path clock value to set the slave clock value. For example, in an embodiment, the clock module 160 is configured to select a path clock value
  • the clock module 160 is configured to select a set of the path clock values that are each associated with a measure of path delay variability that meets one or more criteria, and to determine an average path clock value using the set of selected path clock values.
  • the clock module 160 is configured to select a set of the path clock values that are each associated with a measure of path delay variability that meets a threshold (e.g., is less than the threshold, is less than or equal to the threshold, etc.), and to determine slave clock value using the set of selected path clock values.
  • the clock module 160 is configured to calculate an average of the selected path clock values, and to determine the slave clock value using the average path clock value.
  • me UIOC inouuie iou is coniigureu iu sei me siave IUC using me seiecieu path clock value.
  • the clock synchronization module 150 is configured to measure respective path characteristics for each path such as path length, path delay variability (or noisiness), etc., and/or measured characteristics of respective path clock values, such as degree of jitter.
  • the path length is measured in terms of a number of hops.
  • Traceroute-based path discovery techniques are utilized to determine the number of hops in a path.
  • the clock synchronization module 150 is configured to measure respective path characteristics for each path such as path length, path delay variability (or noisiness), etc., and/or measured characteristics of respective path clock values, such as degree of jitter.
  • Fig. IB is a block diagram of another example system 180 which the master clock device 104 includes only a single port 140a and the slave clock device 108 includes only a single port 154a.
  • Path diversity is entirely provided by the network 112.
  • switches 124 are configured to utilize different paths 184, 188 when transmitting packets destined for the same endpoint device but including different header information (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • switches 124 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths 184, 188 based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • the master clock device 104 varies header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • LUU4 1 ⁇ rigs, ⁇ anu zr are a now diagram ui an example iiieuiuu ⁇ ior maintaining a slave clock using multiple clock signals received from a master clock via multiple paths in a network, according to an embodiment.
  • the slave clock device 108 of Fig. 1A or Fig. IB is configured to implement the method 200, in an embodiment, and the method 200 is discussed with reference to Figs. 1A and IB for explanatory purposes. In other embodiments, however, the method 200 is implemented by another suitable device.
  • a state variable is initialized to a value COMBINED.
  • the state variable generally indicates how the slave clock is to be set. For example, when the state variable is a value COMBINED, the slave clock is to be set according to a first technique (as described below), and when the state variable is a value SWITCHING, the slave clock is to be set according to a second technique (as described below), in an embodiment.
  • a respective clock (e.g., a respective time of day clock (TOD;)) is determined for each of a plurality of paths via which clock signals are received from the master clock device 104.
  • a TOD t is determined for clock signals from the master clock device 104 received via path 128, and TOD 2 is determined for clock signals from the master clock device 104 received via path 132.
  • Block 208 comprises
  • a most recent measure of a delay from the master clock device 104 to the slave clock device 108 via the path i is determined for each path i.
  • the slave clock device 108 transmits a path delay measurement packet (e.g., an Internet Control Message Protocol (ICMP) echo request packet) to the master clock device 104, which prompts the master clock device 104 to immediately transmit a response packet (e.g., an ICMP response) to the slave clock device 108 via the path i, in an embodiment.
  • ICMP Internet Control Message Protocol
  • the slave clock device 108 measures the round-trip delay corresponding to the path i based on when the ICMP echo request packet was transmitted and when the ICMP response was re eiveu vm me pain 1, anu ueieniimes ine as one nan ine ruunu-inp ueiay, m an embodiment. Such measurements are made periodically, according to a schedule, etc., in various embodiments.
  • an average delay from the master clock device 104 to the slave clock device 108 via the path i is determined for each path i.
  • the AvgDelayj is calculated by averaging multiple LastDelayi values over time, in an embodiment.
  • the AvgDelayi is updated with the new LastDelayi value, in an embodiment.
  • respective measures of path delay variability for the different paths are calculated. For example, an absolute value of the difference (DelayDiffj) between the AvgDelayi and the LastDelay; is calculated for each path i, in an
  • a larger DelayDiffj indicates relatively more variability as compared to a smaller DelayDiffj, in an embodiment.
  • the AvgDelayi is updated, as discussed above, and then a new
  • DelayDiffi is calculated, in an embodiment.
  • the respective measures of path delay variability for the different paths also provide a measure of noisiness of the respective TODj's. For example, as the path delay variability for the i-th path increases, jitter (or noisiness) of the TOD for the i-th path will also increase, in an embodiment. In other embodiments, respective measures of noisiness of the respective TODi's are determined by analyzing the respective TODi's over time.
  • a primary path is selected as the path with the lowest measure of path delay variability.
  • the primary path is selected as the paths with the lowest DelayDiffj.
  • the primary path is selected based on one or more suitable measured characteristics of the individual TODs, such as degree of jitter.
  • a first TOD (SwitchingTOD) is set to the TOD; (determined at block 208) corresponding to the primary path (determined at block 220).
  • the switchingTOD corresponds to the TOD ; of the path with the lowest measure of path delay variability, in an embodiment. In other embodiments, the
  • SwitchingTOD corresponds to the TODi of a path selected based on measured
  • characteristics of the individual TODs such as degree of jitter.
  • a second TOD (CombinedTOD) is set to an average of the TODi's for all of the paths (determined at block 208). As will be explained below, the
  • block 228 comprises calculating a weighted average.
  • weights for calculating the weighted average are determined based on the DelayDiffj values calculated at block 216. Thus, in some embodiments, the weights for calculating the weighted average are configurable and are determined based on measured path
  • characteristics such as path length, path delay variability (or noisiness), etc., and/or measured characteristics of the individual TODs, such as degree of jitter.
  • the absolute value of a difference between the SwitchingTOD and the CombinedTOD is compared to a threshold.
  • the threshold is the DelayDiff of the primary path multiplied by a suitable scalar a. If it is determined that the absolute value of the difference between the SwitchingTOD and the CombinedTOD does not meet the threshold (e.g., is less than or equal to the threshold), the state variable remains set to COMBINED, and the flow returns to block 208.
  • block 244 helps ensure continuity of the DynamicTOD.
  • the state variable is set to SWITCHING.
  • the state variable is set to SWITCHING because the comparison at block 240 indicates that path delay variability in one or more of the paths has increased, which may adversely impact the CombinedTOD.
  • the flow proceeds to block 260.
  • the slave clock (DynamicTOD) is set to the SwitchingTOD determined at block 224.
  • the state variable remains set to SWITCHING, and the flow returns to block 208.
  • the flow proceeds to block 268.
  • the TODi for each path i is set to the SwitchingTOD.
  • block 268 helps ensure continuity of the DynamicTOD.
  • the state variable is set to COMBINED. After block 272, the flow returns to block 208.
  • the TOD; for each path i, the SwitchingTOD, and the CombinedTOD are calculated offsets between the master clock and the slave clock.
  • the slave clock (DynamicTOD) is updated using either the CombinedTOD offset (block 236) or the SwitchingTOD offset (block 260).
  • the slave clock is updated by adding the offset to a current value of the slave clock, in an embodiment.
  • Fig. 3 is a timing diagram 300 illustrating signals exchanged between the slave clock device 108 and the master clock device 104 as part of an example of maintaining me siave CIOC , accurumg iu an eiiiDuuimeni. i ne strignu liiusiraieu m rig. J mvuives two different paths in the network 112: path 0 and path 1. In other scenarios, a suitable number of paths greater than two are utilized, at least in some embodiments.
  • the slave clock device 108 transmits a first path delay measurement packet (e.g., ICMP echo request packet) to the master clock device 104, prompting the master clock device 104 to immediately transmit a first response packet (e.g., an ICMP response) to the slave clock device 108 via the path 0, in an embodiment.
  • a first path delay measurement packet e.g., ICMP echo request packet
  • the master clock device 104 prompts the master clock device 104 to immediately transmit a first response packet (e.g., an ICMP response) to the slave clock device 108 via the path 0, in an embodiment.
  • the first response packet e.g., an ICMP response
  • the slave clock device 108 calculates a delay on path 0 ( ⁇ 0 ) as one half of the round trip delay:
  • the slave clock device 108 transmits a second path delay
  • the master clock device 104 prompting the master clock device 104 to immediately transmit a second response packet (e.g., an ICMP response) to the slave clock device 108 via the path 1, in an embodiment.
  • a second response packet e.g., an ICMP response
  • the second response packet transmitted by the master clock device 104 via the path 1 is received at the slave clock device 108, in an embodiment.
  • the slave clock device 108 calculates a delay on path 1 (A ) as one half of the round trip delay:
  • the master clock device 104 transmits a first time-stamped packet to the slave clock device 108 via the path 0.
  • the first time-stamped packet includes a time stamp equal to (or substantially equal to) a value corresponding to time T 4 .
  • the slave clock device 108 receives the first time-stamped packet via me pain u. ner nine 15, ine siave CIUC uevice ⁇ cuinpums a ciuc unsei corresponding to path 0 based on the first time-stamped packet and the calculated delay on path 0 ( ⁇ 0 ).
  • the clock offset corresponding to path 0 (Offset 0 ) is calculated according to:
  • the master clock device 104 transmits a second time-stamped packet to the slave clock device 108 via the path 1.
  • the second time-stamped packet includes a time stamp equal to (or substantially equal to) a value corresponding to time T 6 .
  • the slave clock device 108 receives the second time-stamped packet via the path 1.
  • the slave clock device 108 computes a clock offset corresponding to path 1 based on the second time-stamped packet and the calculated delay on path 1 (Ai). In an embodiment, the clock offset corresponding to path 1
  • Offset ⁇ is calculated according to:
  • the slave clock is then updated using the clock offset corresponding to path 0 and the clock offset corresponding to path 1.
  • the slave clock is then updated using the clock offset corresponding to path 0 and the clock offset corresponding to path 1 according to techniques such as described above (e.g., computing an average of the path 0 offset and the path 1 offset, selecting one of the path 0 offset and the path 1 offset based on path delay variability, etc.).
  • switches/routers can be configured to utilize different paths when transmitting packets destined for the same endpoint device but including different address information (e.g., source addresses and/or destination addresses), in an embodiment.
  • address information e.g., source addresses and/or destination addresses
  • the routing/forwarding tables across the network can be configured with multiple traffic engineered paths between the master clock and the slave clock. By configuring the routers/switches in such networks diverse paths are created for transmission of packets between the master clock and the slave clock.
  • switches/routers in other types of networks, such as public and provider networks, it not possible or practical to configure switches/routers to provide different paths.
  • some networks 112 at least some switches/routers in the network 112 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths, in some embodiments.
  • the master clock device 104 uses different addresses (e.g., destination address) and/or pairs of addresses (e.g., ⁇ master, slave ⁇ ) in the packet.
  • a plurality of addresses and/or address pairs as a whole correspond to a plurality of different paths
  • two or more of such addresses or address pairs may correspond to exactly the same network path, making them such addresses/address pairs redundant.
  • the master clock device 104 and/or the slave clock device 108 are configured to utilize
  • RaceFlow (Viswanathan et al., “TraceFlow draft-zinjuvadia-traceflow-02.txt,” Internet Draft, Internet Engineering Task Force (IETF), August 2008, available at
  • Traceroute-based filtering is implemented by both the master clock device 104 and the slave clock device 108, in an embodiment. Traceroute-based filtering is implemented only on the slave clock device 108 to reduce overhead on the master clock device 104, in another embodiment. Other suitable tools for discovering the paths between two points in a network are utilized in other
  • the slave clock utilizes multiple IP addresses. In an embodiment, each of at least some of the multiple IP addresses of the slave clock corresponds to a different path. In an embodiment, the master clock utilizes multiple IP addresses. In an embodiment, at least some different ⁇ master IP, slave IP ⁇ address pairs corresponds to different paths. In other embodiments, the master clock utilizes multiple VLAN IDs, where each VLAN ID corresponds to a different path.
  • other suitable fields in the packet header are utilized alternatively or additionally to determine different paths.
  • Other header fields utilized to determine a path include one or more of a user datagram protocol (UDP) port field, a Transmission Control Protocol (TCP) port field, the Flow Label field in an IPv6 header, a VLAN ID field in an Ethernet header, a Label field in a Multiprotocol Label Switching (MPLS) header, etc.
  • UDP user datagram protocol
  • TCP Transmission Control Protocol
  • MPLS Multiprotocol Label Switching
  • any combination of any subset of master IP address, slave IP address, a UDP port field, a TCP port field, the Flow Label field in an IPv6 header, a VLAN ID field in an Ethernet header, a Label field in an MPLS header, are utilized to determine and/or identify a path via which a packet is to be or was transmitted.
  • the slave clock device 108 is configured to note the physical port 154 via which a packet is received, and to determine via which path the packet was transmitted based on at least the physical port 154 via which the packet was received.
  • Fig. 4 is a flow diagram of an example method 400 for maintaining a slave clock using multiple clock signals received from a master clock via multiple paths in a network, according to an embodiment.
  • the system 100 of Fig. 1A and/or the system 180 of Fig. IB is configured to implement the method 400, in an embodiment, and the method 400 is discussed with reference to Figs. 1A and IB for explanatory purposes. In other embodiments, however, the method 400 is implemented by another suitable system.
  • Fig. 4 is described, for explanatory purposes, with reference to messages similar to messages utilized in the precision time protocol (PTP). In other embodiments, messages similar to messages utilized in the network time protocol (NTP) are utilized.
  • PTP precision time protocol
  • NTP network time protocol
  • slave IP addresses are associated with the slave clock, and one of the slave IP addresses is designated as a primary slave IP address, in an embodiment.
  • at least one IP address is associated with the master clock.
  • One of the at least one master IP addresses is designated as a primary master IP address, in an embodiment. If there is only one master IP address, the single master IP address is designated as the primary slave IP address, in an embodiment.
  • the slave clock device 108 periodically transmits multiple announcement messages, each announcement message in the multiple announcement messages having a different slave IP address, in an embodiment.
  • the multiple announcement messages each announcement message in the multiple announcement messages having a different slave IP address, in an embodiment.
  • announcement messages are received by the master clock device 104, and the master clock device 104 notes the different slave IP addresses included in the multiple
  • each slave IP address corresponds to a corresponding one of the different paths, in an embodiment.
  • Each synchronization message includes the primary master IP address, in an embodiment.
  • Each synchronization message includes a timestamp corresponding to a value of the master clock when the synchronization message was transmitted by the master clock device 104, in an embodiment.
  • the slave clock device 108 transmits to the master clock device 104 a delay request message responsive to each synchronization message.
  • Each delay request message includes one of the multiple slave IP addresses.
  • the master clock device 104 transmits to the slave clock device 108 a delay response message responsive to each delay response message from the slave clock device 108.
  • Each delay response message includes one of the multiple slave IP addresses, and each delay response message is transmitted via a different path, in an embodiment.
  • Each delay response message includes the primary master IP address, in an embodiment.
  • the slave clock device 108 receives the multiple delay response messages transmitted by the master clock device 104 at block 416. For each delay response message received at block 420, the slave clock device 108 determines a delay corresponding to the path via which the delay response message was received. In an embodiment, the slave clock device 108 determines the delay based on (i) when the delay response message was received at the slave clock device 108, and (ii) when the corresponding delay request message (block 412) was transmitted by the slave clock device 108. For each delay response message received at block 420, the slave clock uevice luo ueienmnes me pain via wiiicn me message was receiveu oaseu on me siave ir address in the message, in an embodiment.
  • the slave clock device 108 receives multiple synchronization messages periodically transmitted by the master clock device 104 at block 408. For each synchronization message received by the slave clock device 108, the slave clock device 108 determines a clock offset corresponding to the path via which the synchronization message was received. In an embodiment, the slave clock device 108 determines the clock offset based on (i) a value of the slave clock when the synchronization message was received at the slave clock device 108, (ii) the master clock time stamp in the synchronization message, and (iii) the delay for the corresponding path (block 420). For each synchronization message received by the slave clock device 108, the slave clock device 108 determines the path via which the message was received based on the slave IP address in the message, in an embodiment.
  • the slave clock device 108 updates the slave clock as a function of the different offsets, determined at block 424, corresponding to the different paths, in an embodiment.
  • multiple master IP addresses are associated with the master clock, where one of the at least one master IP addresses is designated as a primary master IP address, in an embodiment.
  • respective synchronization messages for each ⁇ master IP, slave IP ⁇ address pair are periodically transmitted by the master clock device 104 at block 408, and the different ⁇ master IP, slave IP ⁇ address pairs correspond to different paths.
  • the slave clock device 108 transmits, at block 412, a corresponding delay request message to the master IP address included in the delay request message.
  • the slave clock device 108 determines the path via which the message was received based on the ⁇ master IP, slave IP ⁇ address pair in the message.
  • me s ive CIOU uevice ⁇ ueiemimes me pain via wm ii me message was received based on the ⁇ master IP, slave IP ⁇ address pair in the message.
  • Fig. 5 is a flow diagram of an example method 500 for setting a slave clock, according to an embodiment.
  • the slave clock device 108 of Fig. 1 A and/or Fig. IB is configured to implement the method 500, in an embodiment, and the method 500 is discussed with reference to Figs. 1A and IB for explanatory purposes. In other embodiments, however, the method 500 is implemented by another suitable device.
  • a packet is received at the slave clock device 108, the packet having been transmitted by the master clock device 104.
  • the packet includes a time stamp corresponding to the master clock.
  • the time stamp corresponds to a value of the master clock when the packet was transmitted by the master clock.
  • the slave clock device 108 determines via which one of multiple paths the packet was received at block 504. In an embodiment, the slave clock device 108 determines via which one of the multiple paths based on an address in the packet. In an embodiment, the slave clock device 108 determines via which one of the multiple paths based on a slave IP address in the packet. In an embodiment, the slave clock device 108 determines via which one of the multiple paths based on a pair of addresses in the packet. In an embodiment, the slave clock device 108 determines via which one of the multiple paths based on a ⁇ master IP, slave IP ⁇ address pair in the packet.
  • the clock synchronization module 150 of the slave clock device 108 is configured to determine via which one of multiple paths the packet was received at block 504. In an embodiment, some other module of the slave clock device 108 is configured to determine via which one of multiple paths the packet was received at block 504. In some embodiments, the module implementing block 508 operates at the application layer in the multi-layer communication protocol stack (e.g., the TCP/IP protocol stack), and/or operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack (e.g., corresponding to the OSI model).
  • the application layer in the multi-layer communication protocol stack e.g., the TCP/IP protocol stack
  • another suitable layer at least at a fourth layer in a multi-layer communication protocol stack (e.g., corresponding to the OSI model).
  • me niuuuuie implementing DIUU JUO operates at a fourth layer or above in a multi-layer communication protocol stack, and also determines via which one of multiple network paths the packet was received at block 504.
  • the module implementing block 508 operates at a layer below the application layer and/or below the fourth layer in the multi-layer protocol stack.
  • the module implementing block 508 operates at the IP layer or at Layer-3 in a multi-layer protocol stack.
  • the module implementing block 508 operates at the MAC layer or at Layer-2 in a multi-layer protocol stack.
  • the slave clock device 108 uses the path information determined at block 508 and the time stamp in the packet received at block 504 to set the slave clock value.
  • block 512 is performed at the application layer in a multi-layer communication protocol stack, and/or operates at another suitable layer at least at the fourth layer in the multi-layer communication protocol stack, and also utilizes network path information to set the slave clock value.
  • block 512 is
  • the module implementing block 512 operates at a layer below the application layer and/or below the fourth layer in the multi-layer protocol stack.
  • the module implementing block 512 operates at the IP layer or at Layer-3 in a multi-layer protocol stack.
  • the module implementing block 512 operates at the MAC layer or at Layer-2 in a multi-layer protocol stack.
  • setting the slave clock value at block 512 is
  • luuiiiij rig. o is a nuw uiagram ui an example iiieinuu ⁇ ior laciiiiaimg ueiennmmg clock values across a network, according to an embodiment.
  • the master clock device 104 of Fig. 1A and/or Fig. IB is configured to implement the method 600, in an embodiment, and the method 600 is discussed with reference to Figs. 1 A and IB for explanatory purposes. In other embodiments, however, the method 600 is implemented by another suitable device.
  • multiple first addresses are associated with the master clock.
  • multiple IP addresses master IP addresses
  • multiple time-stamped packets are transmitted by the master clock device 104 to the slave clock device 108, where each packet includes a different one of the multiple addresses of block 604.
  • the different master IP addresses in the multiple time-stamped packets cause, at least partially, the multiple time-stamped packets to be transmitted over different paths in the network 112.
  • the multiple packets include multiple slave IP addresses, and the different ⁇ master IP, slave IP ⁇ address pairs cause, at least partially, the multiple time-stamped packets to be transmitted over different paths in the network 112.
  • Fig. 7 is a flow diagram of an example method 700 for facilitating determining clock values across a network, according to an embodiment.
  • the slave clock device 108 of Fig. 1A and/or Fig. IB is configured to implement the method 700, in an embodiment, and the method 700 is discussed with reference to Figs. 1A and IB for explanatory purposes. In other embodiments, however, the method 700 is implemented by another suitable device.
  • multiple first addresses are associated with the slave clock.
  • multiple IP addresses slave IP addresses
  • Liwiiioj ⁇ Dio / wo, muiiipie requesis are iransiiiiueu iu me inasier CIUC uevice I VJH-, the multiple requests including the multiple addresses to cause the master clock device 104 to transmit multiple time-stamped packets via different paths in the network 112, so that the multiple time-stamped packets are received by the slave clock device 108 via different paths in the network 112.
  • the different slave IP addresses in the requests cause, at least partially, the multiple time-stamped packets to be received over different paths in the network 112.
  • the multiple packets include multiple master IP addresses, and the different ⁇ master IP, slave IP ⁇ address pairs cause, at least partially, the multiple time-stamped packets to be received over different paths in the network 112.
  • Fig. 8 is a block diagram of another example system 800, according to an embodiment, having a first network device 804 communicatively coupled to a second network device 808 via a communication network 812.
  • the first network device 804 implements a master clock, and the first network device 804 is sometimes referred to herein as the master clock device 804.
  • the second network device 808 implements a slave clock, and the second network device 808 is sometimes referred to herein as the slave clock device 808.
  • the master clock device 804 includes a clock module 820 configured to maintain the master clock.
  • the master clock is a time-of-day clock.
  • the master clock is a counter not directly linked to a time-of-day.
  • the clock module 820 includes or utilizes a counter circuit that maintains the master clock.
  • the clock module 820 is an application layer module operating at an application layer in a multi-layer communication protocol stack.
  • the multi-layer communication protocol stack e.g., the
  • Transport Control Protocol/Internet Protocol (TCP/IP) protocol suite comprises a link layer, an Internet layer, a transport layer, and the application layer, where the application layer is above the transport layer, the Internet layer, and the link layer in the protocol stack.
  • the multi-layer communication protocol stack e.g., currcspunumg ⁇ ine vjpen aysiems miercunnecuun ⁇ ai
  • muueij comprises a pnysicai layer, a data link layer, a network layer, a transport layer, a session layer, a presentation layer, and the application layer, where the application layer is above the presentation layer, the session layer, the transport layer, the network layer, the data link layer, and the physical layer in the protocol stack.
  • the clock module 820 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack. In other embodiments, the clock module 820 operates at another suitable layer in a multi-layer communication protocol stack, such as at a second or third layer.
  • the clock module 820 is configured to cause a plurality of time-stamped packets to be transmitted from the master clock device 804 to the slave clock device 808 via respective different communication paths through the network 812.
  • the plurality of time-stamped packets are time-stamped with one or more values of the master clock maintained by the clock module 820, in an embodiment.
  • the network 812 comprises one or more of a private network, a public network, a backbone network, a provider network, the Internet, etc.
  • the network 812 comprises a plurality of switches 824, in an embodiment, and a first path 828 corresponds to a link from the switch 824a to the switch 824b.
  • a second path 830 corresponds to a link from the switch 824c to the switch 824d.
  • a third path 832 corresponds to a link from the switch 824e to the switch 824f.
  • the clock module 820 is configured to cause a first time-stamped packet to be transmitted from the master clock device 804 to the slave clock device 808 via the path 828, to cause a second time-stamped packet to be transmitted from the master clock device 804 to the slave clock device 808 via the path 830, and to cause a third time- stamped packet to be transmitted from the master clock device 804 to the slave clock device 808 via the path 832.
  • the master clock device 804 includes multiple ports 840, and the clock module 820 is configured to cause the first time- stamped packet to be transmitted via the port 840a, the second time-stamped packet to be iransiiiiueu via me pun OH-WD, anu me Liiiru iiine-siampeu c ei iu oc iransiiiiueu via me port 840c.
  • each of at least some of the time- stamped packets are transmitted via the same port 840.
  • three ports 840 are illustrated in Fig. 8, in other embodiments, the master clock device 804 includes a suitable number of ports 840 greater than or less than three.
  • the master clock device 804 includes only a single port 840, and path diversity is entirely provided by the network 812.
  • switches/routers can be configured to utilize different paths when transmitting packets destined for the same endpoint device but including different header information (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • public and/or provider networks at least some switches/routers in the network 812 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • the master clock device 804 varies header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • the clock module 820 includes a path selection module 844 configured to cause time-stamped packets to be transmitted via selected paths in the network, in an embodiment.
  • the path selection module 844 is configured to cause path information to be added to the time-stamped packets, where the path information indicates the network path via which the packet is transmitted.
  • the path selection module 844 is configured to cause a virtual local area network identifier (VID) to be added to a packet, where the VID indicates the network path via which the packet is transmitted.
  • VIP virtual local area network identifier
  • MSTP Multiple Spanning Tree Protocol
  • 802.1Q-2005 specifies a separate Spanning Tree for each VLAN group.
  • switches 824 in the network 812 direct a packet through a particular path through the network 812 according to a VID in the packet.
  • the VID is an indication, in the packet, of the path via which the packet is transmitted, in an embodiment.
  • the VID is part of a combination including other information in the packet (an indication) of the path via which the packet is transmitted, in another embodiment.
  • switches 824 in the network 812 direct a packet through a particular path in the network 812 according to a VID in the packet.
  • the VID is an indication, in the packet, of the path via which the packet is transmitted, in an embodiment.
  • the VID is part of a combination including other information in the packet (the
  • the VID utilized to indicate the network path is a backbone VID (B-VID), and switches 824 in the network 812 direct a packet through a particular path through the network 812 according to the B-VID, in an embodiment.
  • B-VID backbone VID
  • switches 824 in the network 812 direct a packet through a particular path through the network 812 according to the B-VID, in an embodiment.
  • the B-VID is an indication, in the packet, of the path via which the packet is transmitted, in an embodiment.
  • the VID is part of a combination including other information in the packet (an indication) of the path via which the packet is transmitted, in another embodiment.
  • the path information added to the time-stamped packets includes a VID and one or more addresses corresponding to a layer below an Internet Protocol (IP) layer in a multi-layer protocol stack.
  • IP Internet Protocol
  • the path information added to the time-stamped packets includes a VID and one or more Layer-2 addresses, in an embodiment.
  • the path information added to the time-stamped packets includes one or more IP addresses.
  • the master clock device 804 is associated with multiple IP addresses
  • the slave clock device 808 is associated with multiple IP addresses
  • path selection is achieved, at least in part, by selection of the master IP address and/or the slave IP address.
  • the network 812 is engineered to route packets with different slave IP addresses along different paths, in an embodiment.
  • the network 812 is engineered to route packets with different master IP addresses along different paths.
  • the network 812 is engineered to route packets with different master IP address/slave IP address pairs along different paths.
  • the master clock device 804 and/or the slave clock device 808 are configured to utilize Traceroute-based path discovery to determine paths that correspond to different master IP addresses, slave IP address, and/or master IP address/slave IP address pairs.
  • Traceroute-based path discovery to determine paths that correspond to different master IP addresses, slave IP address, and/or master IP address/slave IP address pairs.
  • Parent traceroute and “TraceFlow” are examples of tools that discover the paths between two points in the network. Other suitable tools for discovering the paths between two points in a network are utilized in other embodiments.
  • the clock module 820 causes a plurality of packets 848 to be transmitted through the network 812 along different paths 828, 830, 832.
  • the packet 848a is transmitted along the path 828; the packet 848b is transmitted along the path 830; and the packet 848c is transmitted along the path 832.
  • Each packet 848 includes respective path information (e.g., Layer-2 information such as a VLAN-ID, one or more addresses, etc., and/or Layer-3 or higher information such as one or more IP addresses, etc.) and respective time information (e.g., a time stamp), in some embodiments.
  • Layer-2 information such as a VLAN-ID, one or more addresses, etc.
  • Layer-3 or higher information such as one or more IP addresses, etc.
  • time information e.g., a time stamp
  • a time synchronization module 850 of the slave clock device 808 is configured to process respective path information, and respective time information included in each of at least some of the packets 848, and to determine a system time clock responsive to the processing of the path information and the time information.
  • the time synchronization module 850 is configured to maintain respective clock information for each path, and to use at least some of the clock information for the different paths to determine the system time clock. For example, the time
  • synchronization module 850 is configured to maintain first clock information
  • the slave clock device 808 includes multiple ports 854, and the packet 848a is received via the port 854a; the packet 848b is received via the port 854b; and the packet 848c is received via the port 854c. In other embodiments and/or scenarios, each of at least some of the packets 848 are received via the same port 854. Although three ports 854 are illustrated in Fig. 8, in other embodiments, the slave clock device 808 includes a suitable number of ports 154 less than or greater than three. In an embodiment, the slave clock device 808 includes only a single port 854.
  • the time synchronization module 850 is configured to update respective clock information for each path based on the time information in the respective time-stamped packet 848 transmitted via the respective path.
  • the clock information for each path maintained by the time synchronization module 850 comprises a respective offset between the master clock maintained at the master clock device 804 and a slave clock maintained at the slave clock device 808.
  • the clock information for each path generated by the time synchronization module 150 comprises a respective clock value (e.g., a time-of-day vaiuej uurresponumg LU ine master CIU iiiamiameu ai me inasier IUC uevi e out.
  • the time synchronization module 850 is configured to determine each clock value based on the respective time stamp in the respective time-stamped packet 848 transmitted via the respective path, in an embodiment.
  • the time synchronization module 850 is an application layer module operating at the application layer in the multi-layer communication protocol stack. In other embodiments, the clock synchronization module 850 operates at another suitable layer at least at a fourth layer in a multi-layer communication protocol stack. In other embodiments, the clock synchronization module 850 is at another suitable layer below an application layer and/or operates at a suitable layer below the fourth layer in the multi-layer communication protocol stack.
  • the time synchronization module includes a path
  • the path identification module 870 is configured to analyze path information in each packet 848 to identify that the packet 848 was transmitted via a particular path. For instance, in an embodiment, the path identification module 870 is configured to analyze Layer-2 information in the packet 848 to identify that the packet 848 was transmitted via a particular path. As an illustrative example, in an embodiment, the path identification module 870 is configured to analyze a VLAN ID in the packet 848 to identify that the packet 848 was transmitted via a particular path. For example, different VLAN IDs indicate a different one of the paths 828, 830, 832, in an
  • the path identification module 870 is configured to analyze a VLAN ID in the packet 848 as well as other Layer-2 information (e.g., one or more addresses) to identify that the packet 848 was transmitted via a particular path.
  • Layer-2 information e.g., one or more addresses
  • the path identification module 870 is configured to analyze Layer-3 information in the packet 848 to identify that the packet 848 was transmitted via a particular path.
  • the path identification module 870 is configured to analyze at least an IP address in the packet 848 ⁇ luenuiy uiai me ac ei OHO was iransiiiiueu via a parucuiar pain, rur example, different source IP addresses, destination IP addresses, or source IP/destination IP paires indicate a different one of the paths 828, 830, 832, in various embodiment.
  • the path identification module 870 is configured to analyze one or more IP addreses in the packet 848 as well as other Layer-3 or higher information to identify that the packet 848 was transmitted via a particular path.
  • the time synchronization module 850 in the slave clock device 808 is configured to maintain the slave clock, in some embodiments.
  • the slave clock is a time-of-day clock.
  • the slave clock is a counter not directly linked to a time-of-day.
  • the time synchronization module 850 includes or utilizes a counter circuit that maintains the slave clock.
  • the time synchronization module 850 is configured to select one of the path-dependent clocks as the slave clock rather than maintaining a separate slave clock.
  • the time synchronization module 850 is configured to determine the slave clock value using the path-dependent clock information using techniques such as described above, in some embodiments. For example, in an embodiment, the time synchronization module 850 is configured to determine the slave clock value at least by mathematically combining at least some of the path-dependent clock information. In another embodiment, the time synchronization module 850 is configured to determine the slave clock value at least by selecting one of a plurality of path-dependent clock results. In another embodiment, the time synchronization module 850 is configured to determine the slave clock value at least by (i) combining at least some of the plurality of
  • path-dependent clock results in some situations, and (ii) selecting one of the plurality of path-dependent clock results.
  • synchronization module 850 is configured to determine the slave clock value at least by l) selecting a suDsei ⁇ me piuraiiiy ui pain-uepenueni C!UC resuiis, anu ⁇ n ; cuiiiDimng the subset of path-dependent clock results.
  • Fig. 8B is a block diagram of another example system 880 which the master clock device 804 includes only a single port 840b and the slave clock device 808 includes only a single port 854b.
  • Path diversity is entirely provided by the network 812.
  • switches 824 are configured to utilize different paths 884, 888, 892 when transmitting packets destined for the same endpoint device but including different header information (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • switches 824 apply load balancing techniques in which packets destined for the same endpoint device are directed along different paths 884, 888, 892 based upon different header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • the master clock device 804 varies header information in the packets (e.g., source addresses, destination addresses, Layer 4 ports, etc.), in some embodiments.
  • Fig. 9 is a flow diagram of an example method 900 for facilitating determining clock values across a network, according to an embodiment.
  • the slave clock device 808 of Fig. 8 A and/or Fig. 8B is configured to implement the method 900, in an embodiment, and the method 900 is discussed with reference to Figs. 8A and 8B for explanatory purposes. In other embodiments, however, the method 900 is implemented by another suitable device, such as the slave clock device 808 of Fig. 8A and/or Fig. 8B.
  • time synchronization packets from a master clock are received via a plurality of communication paths.
  • the time synchronization packets 848 are received via the paths 828, 830, and 832.
  • Each time synchronization packet includes path information and time information.
  • the path information e.g., Layer-2 information such as a VLAN-ID, one or more addresses, etc., and/or Layer-3 or higher lniumiaiion sucii as one or more ir auuresses, eic muicaies me pain via wmuii me packet was transmitted, in an embodiment.
  • the time information is a time stamp, a clock value, etc., corresponding to the master clock, in some embodiments.
  • the path via which the packet was transmitted is identified using the path information in the packet.
  • the path is identified using one or more of (i) a VLAN ID, (ii) a Layer-2 destination address, (iii) a Layer-2 source address, (iv) a source IP address, and/or (v) a destination IP address in the packet, in various embodiments.
  • the packet 848a is identified as being transmitted via path 828 by analyzing path information in the packet 848a
  • the packet 848b is identified as being transmitted via path 830 by analyzing path information in the packet 848b
  • the packet 848c is identified as being transmitted via path 832 by analyzing path information in the packet 848c.
  • respective time synchronization information for the paths identified at block 908 are updated using respective time information in the packets. For example, time synchronization information corresponding to path 828 is updated using time information in the packet 848a; time synchronization information corresponding to path 830 is updated using time information in the packet 848b; and time synchronization information corresponding to path 832 is updated using time information in the packet 848c.
  • a system time clock is determined using at least a subset of the time synchronization information for the different paths determined at block 912.
  • Techniques such as described above are used to determine the system time clock using at least a subset of time synchronization information for the different paths are utilized, in some embodiments.
  • BMCA Best Master Clock Algorithm
  • At least some of the various blocks, operations, and techniques described above may be implemented utilizing hardware, a processor executing firmware instructions, a processor executing software instructions, or any combination thereof.
  • the software or firmware instructions may be stored in any tangible, non-transitory computer readable memory such as a magnetic disk, an optical disk, a random access memory, a read only memory, a flash memory, etc.
  • the software or firmware instructions may include machine readable instructions that, when executed by the processor, cause the processor to perform various acts.
  • the hardware may comprise one or more of discrete components, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, etc.
  • ASIC application-specific integrated circuit

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Synchronisation In Digital Transmission Systems (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

Selon l'invention, un dispositif de réseau comprend un ou plusieurs ports couplés à un réseau, et un module de synchronisation temporelle. Le module de synchronisation temporelle traite (i) des informations de chemin respectives, et (ii) des informations de synchronisation temporelle respectives incluses dans chaque paquet de synchronisation temporelle parmi au moins certains d'une pluralité de paquets de synchronisation temporelle reçus en provenance d'un dispositif d'horloge maître sur deux chemins de communication différents ou plus et par l'intermédiaire d'au moins un du ou des ports, les informations de chemin respectives indiquant un chemin de communication respectif dans le réseau par l'intermédiaire duquel le paquet de synchronisation temporelle respectif a été reçu. Le module de synchronisation temporelle détermine une horloge de temps système en réponse au traitement des informations de chemin et des informations de synchronisation temporelle incluses dans au moins certains de la pluralité de paquets de synchronisation temporelle.
PCT/IB2013/001506 2012-05-09 2013-05-09 Synchronisation d'horloge à l'aide de multiples chemins de réseau WO2013167977A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201380024941.2A CN104584467B (zh) 2012-05-09 2013-05-09 使用多个网络路径的时钟同步

Applications Claiming Priority (16)

Application Number Priority Date Filing Date Title
US201261644722P 2012-05-09 2012-05-09
US61/644,722 2012-05-09
US201261647173P 2012-05-15 2012-05-15
US201261647123P 2012-05-15 2012-05-15
US61/647,123 2012-05-15
US61/647,173 2012-05-15
US201261695367P 2012-08-31 2012-08-31
US201261695371P 2012-08-31 2012-08-31
US61/695,367 2012-08-31
US61/695,371 2012-08-31
US201261706526P 2012-09-27 2012-09-27
US61/706,526 2012-09-27
US201261713818P 2012-10-15 2012-10-15
US61/713,818 2012-10-15
US13/764,732 2013-02-11
US13/764,732 US9172485B2 (en) 2012-02-09 2013-02-11 Clock synchronization using multiple network paths

Publications (1)

Publication Number Publication Date
WO2013167977A1 true WO2013167977A1 (fr) 2013-11-14

Family

ID=49550216

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2013/001506 WO2013167977A1 (fr) 2012-05-09 2013-05-09 Synchronisation d'horloge à l'aide de multiples chemins de réseau

Country Status (2)

Country Link
CN (1) CN104584467B (fr)
WO (1) WO2013167977A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9112630B1 (en) 2012-02-09 2015-08-18 Marvell Israel (M.I.S.L) Ltd. Clock synchronization in the presence of security threats
US9172485B2 (en) 2012-02-09 2015-10-27 Marvell Israel (M.I.S.L) Ltd. Clock synchronization using multiple network paths
US9806835B2 (en) 2012-02-09 2017-10-31 Marvell International Ltd. Clock synchronization using multiple network paths
EP3255820A1 (fr) * 2016-06-06 2017-12-13 Sprecher Automation GmbH Procédé d'enregistrement synchronisé de données de mesure nécessaires à la commande de dispositif de protection différentielle de lignes électriques
CN109218007A (zh) * 2017-07-03 2019-01-15 中兴通讯股份有限公司 路径选择方法、设备及存储介质
WO2019177503A1 (fr) * 2018-03-14 2019-09-19 Telefonaktiebolaget Lm Ericsson (Publ) Procédé et premier dispositif pour une synchronisation d'horloge

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106911414A (zh) * 2015-12-22 2017-06-30 中兴通讯股份有限公司 时钟同步方法和装置
CN113556221A (zh) * 2020-04-23 2021-10-26 西门子股份公司 冗余网络中校时的方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2254267A1 (fr) * 2009-05-22 2010-11-24 Zarlink Semiconductor Inc. Récupération de synchronisation d'entrées multiples dans des réseaux par paquets
WO2012065823A1 (fr) * 2010-11-15 2012-05-24 Alcatel Lucent Procédé pour synchroniser l'horloge maîtresse et l'horloge asservie d'un réseau à commutation de paquets ayant des connexions agrégées entre des noeuds et dispositifs de synchronisation associés
WO2013117997A2 (fr) * 2012-02-09 2013-08-15 Marvell Israel (M.I.S.L) Ltd. Synchronisation par rapport à l'horloge au moyen de plusieurs chemins de réseau

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009056638A1 (fr) * 2007-11-02 2009-05-07 Nortel Networks Limited Synchronisation de nœuds de réseau

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2254267A1 (fr) * 2009-05-22 2010-11-24 Zarlink Semiconductor Inc. Récupération de synchronisation d'entrées multiples dans des réseaux par paquets
WO2012065823A1 (fr) * 2010-11-15 2012-05-24 Alcatel Lucent Procédé pour synchroniser l'horloge maîtresse et l'horloge asservie d'un réseau à commutation de paquets ayant des connexions agrégées entre des noeuds et dispositifs de synchronisation associés
WO2013117997A2 (fr) * 2012-02-09 2013-08-15 Marvell Israel (M.I.S.L) Ltd. Synchronisation par rapport à l'horloge au moyen de plusieurs chemins de réseau

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JEAN-CHARLES TOURNIER ET AL: "Precise time synchronization on a high available redundant ring protocol", PRECISION CLOCK SYNCHRONIZATION FOR MEASUREMENT, CONTROL AND COMMUNICATION, 2009. ISPCS 2009. INTERNATIONAL SYMPOSIUM ON, IEEE, PISCATAWAY, NJ, USA, 12 October 2009 (2009-10-12), pages 1 - 4, XP031570880, ISBN: 978-1-4244-4391-8 *
SHPINER TECHNION-ISRAEL INSTITUTE OF TECHNOLOGY R TSE C SCHELP PMC-SIERRA T MIZRAHI MARVELL A: "Multi-Path Time Synchronization; draft-shpiner-multi-path-synchronization-00.txt", MULTI-PATH TIME SYNCHRONIZATION; DRAFT-SHPINER-MULTI-PATH-SYNCHRONIZATION-00.TXT, INTERNET ENGINEERING TASK FORCE, IETF; STANDARDWORKINGDRAFT, INTERNET SOCIETY (ISOC) 4, RUE DES FALAISES CH- 1205 GENEVA, SWITZERLAND, 15 October 2012 (2012-10-15), pages 1 - 15, XP015085235 *
TAL MIZRAHI: "Slave diversity: Using multiple paths to improve the accuracy of clock synchronization protocols", PRECISION CLOCK SYNCHRONIZATION FOR MEASUREMENT CONTROL AND COMMUNICATION (ISPCS), 2012 INTERNATIONAL IEEE SYMPOSIUM ON, IEEE, 24 September 2012 (2012-09-24), pages 1 - 6, XP032257242, ISBN: 978-1-4577-1714-7, DOI: 10.1109/ISPCS.2012.6336621 *
VISWANATHAN ET AL.: "Internet Draft", August 2008, IETF, article "TraceFlow draft-zinjuvadia-traceflow-02.txt"

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9112630B1 (en) 2012-02-09 2015-08-18 Marvell Israel (M.I.S.L) Ltd. Clock synchronization in the presence of security threats
US9172485B2 (en) 2012-02-09 2015-10-27 Marvell Israel (M.I.S.L) Ltd. Clock synchronization using multiple network paths
US9806835B2 (en) 2012-02-09 2017-10-31 Marvell International Ltd. Clock synchronization using multiple network paths
US10205547B2 (en) 2012-02-09 2019-02-12 Marvell Israel (M.I.S.L) Ltd. Clock synchronization using multiple network paths
EP3255820A1 (fr) * 2016-06-06 2017-12-13 Sprecher Automation GmbH Procédé d'enregistrement synchronisé de données de mesure nécessaires à la commande de dispositif de protection différentielle de lignes électriques
CN109218007A (zh) * 2017-07-03 2019-01-15 中兴通讯股份有限公司 路径选择方法、设备及存储介质
CN109218007B (zh) * 2017-07-03 2022-11-01 中兴通讯股份有限公司 路径选择方法、设备及存储介质
WO2019177503A1 (fr) * 2018-03-14 2019-09-19 Telefonaktiebolaget Lm Ericsson (Publ) Procédé et premier dispositif pour une synchronisation d'horloge
US11329746B2 (en) 2018-03-14 2022-05-10 Telefonaktiebolaget Lm Ericsson (Publ) Method and a first device for clock synchronization

Also Published As

Publication number Publication date
CN104584467B (zh) 2018-04-27
CN104584467A (zh) 2015-04-29

Similar Documents

Publication Publication Date Title
US10205547B2 (en) Clock synchronization using multiple network paths
US9172485B2 (en) Clock synchronization using multiple network paths
WO2013167977A1 (fr) Synchronisation d'horloge à l'aide de multiples chemins de réseau
US9112630B1 (en) Clock synchronization in the presence of security threats
US11133998B2 (en) Method, apparatus, and system for measuring network delay
US11496235B2 (en) Symmetric path/link over LAG interface using LLDP for time synchronization between two nodes using PTP
US11057136B1 (en) Time correction using extension fields
US9698926B2 (en) Distributed two-step clock
JP6009672B2 (ja) 通信ネットワークにおいて、高精度時刻プロトコルによって配信クロックを同期するための方法
US20060203851A1 (en) Applications of multiple time synchronization domains
US20120128011A1 (en) Method on a network element for the purpose of synchronization of clocks in a network
JP2012209791A (ja) ネットワークノード、時刻同期方法及びネットワークシステム
WO2012130629A1 (fr) Procédé, appareil et système de distribution temporelle dans un réseau de télécommunications
EP2422468B1 (fr) Éléments d'addition et d'élimination pour support sur trajet ntp et procédé d'interaction entre les domaines de protocole ntp et iee-1588
JP2013104772A (ja) フレーム伝送装置及び同期方法
WO2014044487A1 (fr) Procédé de synchronisation d'horloge, et appareil électronique
US7688862B2 (en) Systems and methods for the synchronization of a real-time scheduled packet network using relative timing
CN101902291A (zh) 一种精确时间同步方法、设备和系统
EP2458757B1 (fr) Procédé de transmission de paquets de synchronisation IEEE 1588V2 sur Internet dans un mode lien par lien via des éléments de réseau à horloge transparente et dispositif d'aide associé
JP2001036538A (ja) 時刻同期方法及びその装置
Kálmán et al. Time synchronization challenges in industrial networks
JP4408086B2 (ja) クロック同期方法およびクロック同期回路

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13747691

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13747691

Country of ref document: EP

Kind code of ref document: A1