WO2000028683A2

WO2000028683A2 - Emulation for synchronous behavior in a plesiosynchronous environment

Info

Publication number: WO2000028683A2
Application number: PCT/US1999/026486
Authority: WO
Inventors: Paul M. Sweazey
Original assignee: Intera Systems, Inc.
Priority date: 1998-11-09
Filing date: 1999-11-09
Publication date: 2000-05-18
Also published as: WO2000028683A9; WO2000028683A3; AU2146900A

Abstract

A switch fabric (100) is disclosed in which a cycle master node (104) is coupled to destination nodes (108) to distribute isochronous payload among nodes (104, 108). Precise temporal information is distributed throughout the fabric (100) in slots (312) to identify isochronous payload by the payload"s occurrence in a particular place and time. The cycle master (104) distributes a unique symbol called a mark (304) to each destination node (108). The mark (304) identifies the beginning of an isochronous cycle (300) to each destination node (108) which receives the mark (304). The cycle master (104) distributes isochronous symbols (312) which provides the isochronous payload information to the destination node (108). The distribution of slots (312) in combination with a mark symbol (304) allows each node to determine the local time at any given time by counting the number of slots (312) received after the last mark (304) received. The cycle master (104) also distributes gap symbols (320) to the destination nodes (108), which allow the destination nodes (108) to maintain a uniform throughput throughout the fabric (100) regardless of the transmission capacity of each node (104, 108).

Description

EMULATION FOR SYNCHRONOUS BEHAVIOR IN A PLESIOSYNCHRONOUS ENVIRONMENT

Related Applications

This application claims the benefit of the following co-pending provisional applications: U.S. Provisional Patent application 60/124,581, entitled "Emulation Of Synchronous Behavior In A Plesiosynchronus Interconnect", filed on March 16, 1999; U.S. Provisional Patent application 60/107,732, entitled "Fault Tolerant

Synchronization", filed on November 9, 1998; and U.S. Provisional Patent application 60/127,262, entitled "High Speed Switch Fabric", filed on March 31, 1999.

I. Background of the Invention i. Technical Field

This invention pertains to the field of data communications, and, more specifically, maintaining synchronization between interconnected nodes in a network of connected devices. ii. Background Art Time domain multiplexing (TDM) divides a continuous stream of transmitted data into regularly repeating timeslots, and places successive data values of a particular stream within each repetition of a known timeslot. A TDM network is operated using this method. Since data is identified by the time at which it appears, a TDM network requires that all nodes in the network be temporally synchronized with each other. The synchronization information available to nodes in a TDM network is typically derived from the signals that they receive on the network links. An individual link in a TDM network exhibits synchronous behavior, because the transmitting logic, the receiving logic, and the link are all within the same clock domain of the transmitting logic.

It is advantageous if all nodes in the network are able to operate at precisely the same speed, because the logic associated with transmission and reception of data is less complex than when nodes operate at different speeds. In practice, such completely synchronous behavior is not practical or possible, especially in networks that span long distances or whose reliability characteristics are incompatible with usage of a single clock source for the entire network. In contrast, the nodes of a plesiosynchronous network operate by convention at nearly, but not exactly, the same frequency.

An isochronous network is a network that provides a one-to-one correspondence between timeslots on each network link, whether or not the isochronous network is synchronous or plesiosynchronous. Isochronous behavior is required in order to implement a TDM network.

A packet network is a network that transmits data without the use of timeslots; instead data is identified by its occurrence within a contiguous, grouped sequence of data values that constitute a packet. A packet typically contains identifying information, called a header, at the beginning of the packet. The interpretation of the identifying information is established by convention — by conformance to a standardized format for the header. Because the interpretation of a packet is not coupled to any timeslot on the network, the behavior of a packet network is referred to as asynchronous. Packet transmission is often the optimal method for communication between computing systems. TDM transmission is often the optimal method for transmission of human I/O, such as voice and video streams. It is desirable for a single network to support both transmission methods so that the network may adapt to any mixture of the two types of transmissions.

Thus, a new system, method, and apparatus is needed for transmitting TDM (synchronous or isochronous) data and packet (asynchronous) data in the same network.

Disclosure of Invention

A system, method and apparatus for emulating synchronous behavior in a plesiosynchronous environment is disclosed in which a cycle master node (104) is coupled to destination nodes (108) in a fabric (100) to distribute isochronous payload between nodes (104, 108). Precise temporal information is distributed throughout the fabric (100) in slots (312) to identify isochronous payload by the pay load's occurrence in a particular place and time. The cycle master (104) distributes a unique symbol called a mark (304) to each destination node (108). The mark (304) identifies the beginning of an isochronous cycle (300) to each destination node (108) which receives the mark (304). The cycle master (104) distributes slots (312) which provide information about the passage of time to the destination node (108). The distribution of slots (312) in combination with a mark symbol (304) allows each node (104, 108) to determine the local time at any given time by counting the number of slots (312) received after the last mark (304) received. Gap symbol periods (308) containing gaps (320) are interspersed among the slots (312) so that the number of slots (312) occurring between successive marks (304) is identical throughout the fabric (100) for any particular isochronous cycle (300). Thus, synchronous data may be distributed in accordance with the present invention due to the distribution of the mark symbol (304), slot symbol periods (312), and gap symbol periods (308).

The cycle master (104) controls the number of slots (312) in each isochronous cycle (300). In one embodiment, the cycle master (104) avoids creating more slots (312) than may be handled by the slowest node (108) based on the information available to it about the speed of the slowest node (108), and of its own speed relative to the slowest node (108). In a preferred embodiment, the gaps (320) are evenly distributed throughout the isochronous cycle (300) to allow the receipt of slots (312) to most closely replicate the passage of time. In a further embodiment, the isochronous nature of the fabric (100) is used to provide error-checking.

Brief Description of the Drawings

These and other more detailed and specific objects and features of the present invention are more fully disclosed in the following specification, reference being had to the accompanying drawings, in which:

Figure 1 illustrates a preferred embodiment of a fabric 100. Figure 2A-E illustrate the passage of symbols through a fabric 100 in accordance with a preferred embodiment of the present invention.

Figure 3 illustrates an isochronous cycle 300 in accordance with a preferred embodiment of the present invention. Figures 4a and 4b illustrate alternate embodiments of an isochronous cycle 300 in accordance with a preferred embodiment of the present invention.

Figure 5 is a block diagram illustrating a node 108 in accordance with the present invention. Figure 6 illustrates a method of error detection in accordance with a preferred embodiment of the present invention.

Detailed Description of the Preferred Embodiments Figure 1 illustrates a switch fabric 100 in accordance with the present invention. In this embodiment, the fabric 100 is an interconnection of client device nodes 104, 108 designed to provide high speed aggregate data stream delivery. The fabric 100 illustrated in Figure 1 is of a toroidal topology; however, the topology of the fabric 100 is limited only by the number of ports 1 12, 120 implemented in a particular switching design. The fabric 100 of Figure 1 is specifically designed for short-haul interconnections, for chip-to-chip connections, or for local area networks. The nodes 104, 108 are attached to the client devices 512 (shown in Figure 5) which are connected within the fabric 100. Client devices may include storage devices, processor/memory subsystems, network interface adapters, and/or television cameras. The connection lengths may be several inches or could be many meters. The client devices 512 typically communicate using asynchronous protocols. When synchronous data, for example, voice or video data, is required to be transmitted, the incompatibility of the two types of protocols leads to inefficiency and high complexity in conventional systems. Each node 104, 108 has a node processing unit 500 shown in Figure 5 which controls the interconnection protocol and procedures. The node 104, 108 which contains the node processing unit 500 is typically an integrated circuit, manufactured using conventional semiconductor processes. Each node 104, 108 is connected to the other nodes 104, 108 through a link 116. The links 116 preferably contain two identical groups of signals, called lanes, that propagate information in opposing directions, thus allowing bidirectional communication between each connected node 104, 108 in the fabric 100. Links 116 are preferably implemented as single-ended or differential electrical backplane conductors, as short, intra-chassis copper cable or flex circuits, or as inter-chassis fiber-optic ribbon cable. Each lane may include data signals and a clock signal, or may consist of one or more serial signals with embedded clock information. The fabric 100 uses the links 116 to deliver client data between the nodes 104, 108 of the fabric 100. In accordance with the present invention, the client data can be transmitted asynchronously, in the usual manner, or isochronously, to allow the transmission of TDM data.

Each node 104, 108 is allowed to transmit symbols at a slightly different rate than other nodes 104, 108 in the fabric 100, utilizing node-local clock sources. In a preferred embodiment, the client device nodes 104, 108 operate within a predefined tolerance of a specified frequency, allowing the fabric 100 to rely upon a limited range of clock speeds for each node 104, 108; however this does not limit the frequency at which an individual node 104, 108 can operate. This allows the fabric 100 of the present invention to provide a more flexible and accommodating structure for the interconnection of a variety of client device nodes 104, 108 which are typical in an asynchronous network. However, without the improvements of the present invention, this system would not allow the individual nodes 104, 108 to recognize the passage of time by counting packets of data as the data arrive and depart and thus could not process TDM data accordingly. Client data is transmitted in terms of symbols. Figure 3 is an illustration of the symbols and symbol periods used in an isochronous cycle 300 in a preferred embodiment of the present invention. Symbols preferably carry 64 bits of data payload and have a period often nanoseconds, although the actual bit- width and period are not critical to the present invention. There are two main types of symbol periods transmitted throughout the fabric 100: a slot symbol period 312 and a gap symbol period 308. A slot symbol period 312 may be occupied by a mark 304, or any other isochronous symbol 316 or asynchronous symbol 324 except a gap symbol 320; a gap symbol period 308 may only contain a gap symbol 320. As shown in Figure 3, slot symbol periods 312 are increasingly enumerated symbol periods. Gap symbol periods 308 are not enumerated, and are shown in Figure 3 with a "g." An isochronous cycle 300 (iCycle) is preferably a time period defined to endure 125 μsec according to the chosen reference timebase of the fabric 100, starting with the first slot 312 following a mark 304, and ending with the slot 312 occupied by the next mark 304. The iCycle 300 can last for any period of time, however a 125 μsec period is chosen as it is the convention for telephony applications. The next slot 312 subsequent to a mark 304 is designated as slot zero. The gap symbol periods 308 are used to equalize the number of slots 312 flowing through a node 108 to ensure synchronization while allowing each node 108 to process slots 312 at its own rate. Gap symbol periods 308 may be inserted or deleted at will by each node 108, 104.

An isochronous symbol 316 bears client payload through the fabric 100. Each isochronous symbol 316 is pre-arranged to be produced and launched by a particular producer node 104, 108, to propagate through the fabric 100 in accordance with a schedule, and to be received by each consumer node 108 within a particular slot 312. During any particular isochronous cycle 300 every node 104, 108 sends and receives precisely the same number of slots 312 at each operational inbound and outbound lane of each of its fabric ports 112, 120. If a slot 312 contains an isochronous symbol 316, then that symbol 316 is fully identified by the number of the slot 312 that the isochronous symbol 316 occupies and the link 116 from which was transmitted. This allows the slots 312 to be used for communicating synchronous data.

A mark symbol 304 is a symbol which occupies the last slot of an isochronous cycle 300, and whose occurrence identifies the following slot as slot zero of the next isochronous cycle 300. A mark 304 is preferably placed in a slot 312 that is substantially temporally equidistant from the gap symbol period 308 that precedes it and the gap symbol period 308 that follows it. The mark symbol 304 also carries a "next mark" field which identifies the slot number the next mark 304 will arrive within. This prediction is useful for error checking, as described below. Figure 4a represents an isochronous cycle 300 for a slowest node 108 in a network. Figure 4b represents an isochronous cycle 300 for a fastest node 108 in a network. As shown, the total number of slots 312 remains the same [15] for all three nodes 108. However, due to different processing speeds, each node 108 processes a different number of symbol periods 312, 308. Accordingly, in Figure 4a, only one gap 320 is processed by the slowest node 108. However, for the faster node 108, in Figure 4b, 5 gaps are processed in an isochronous cycle 300. Thus, gap symbol periods 308 containing gaps 320 are used to ensure that same number of slots 312 within an isochronous cycle 300 are processed by each node 108.

To determine the number of gaps 308 (Gmin) to be inserted in an iCycle 300, preferably the system determines the minimum number of slots 312 that any operation node 108 will ever send during an isochronous cycle 300 (Pmin). Next, the system determines the maximum number of slots (Pmax) that a cycle master 104 will ever send during an isochronous cycle 300. The number of gaps 320 to be inserted therefore is given by Gmin = Pmax - Pmin. This ensures that there are enough gaps 320 to ensure that the cycle master 104 will process only Pmin slots 312 within the isochronous cycle 308. The number of gaps 320 inserted into the isochronous cycle 300 is preferably minimized to allow the capacity of the fabric 100 to be used for carrying information-bearing isochronous symbols 316 instead of the place holder gap symbols 320.

In accordance with the present invention, and as shown in Figure 1 , one node is selected to be a cycle master node 104. The other nodes 108 are referred to as the destination nodes 108. The cycle master node 104 is the node which is responsible for generating the temporal information which is distributed throughout the fabric 100. The cycle master node 104 originates marks 304 and establishes the number of slots 312 available during each iCycle for sending symbols. Gap and slot symbol periods 308, 312, and mark symbols 304, may all be considered to originate at the cycle master 104, and to be broadcast therefrom to every other point in the fabric 100. The cycle master 104 broadcasts the slots 312 as synchronizing events for the nodes 108 in the fabric 100. The slots 312 are analogous to the ticks of a clock; one can keep track of the elapsed time by counting clock ticks. The cycle master 104 also has the duty to place a mark 304 periodically into a slot 312 as an isochronous cycle 300 framing event. The mark 304 is analogous to an hourly mark; the local time is known by each node 108 by noting the time by the hourly mark, and the local time is tracked thereafter by counting clock ticks or slots 312 since the last mark 304. Because the cycle master 104 issues slots 312 and marks 304, local time with respect to a particular inbound fabric lane is known by counting slots 312 since the last received mark 304, thus providing each node 108 the ability to recognize the passage of time by counting symbols.

To allow the nodes 108 to recognize the passage of time, conventions for the receipt and transmission of marks 304 through the ports 112, 120 of a node 108 are defined. Marks 304 are sent from the cycle master 104 to all destination nodes 108 using a broadcast mechanism whereby each node 108 is responsible for transmitting received marks 304 to its immediate neighbor nodes 108. The neighbor nodes 108 also are responsible for transmitting received marks 304 to their neighbor nodes 108 and so on, and thus a node 108 that is first to send a particular mark 304 to an outbound lane of a fabric link 116 will promptly receive that mark 304 back via the corresponding inbound lane of the same link 116 from the neighbor node 108 to which it originally sent the mark 304. The port at which a node 108 receives a first instance of a particular mark 304 is designated as the root port 112. All other ports of that node 108 are designated as the leaf ports 120. For example, in Figure 1, the cycle master node 104 transmits a mark 304 across a link 116 to destination node 108(1). The destination node 108(1) receives the mark 304 at its south port. As this port is the port which receives a mark 304 first, i.e., prior to the reception of this mark 304 by any other port, this port is designated as a root port 112. The root ports 112 of each node 108 are preferably designated upon transmission of the first mark 304 of a new fabric 100. As this first mark 304 propagates through the fabric 100, the root ports 1 12 of the nodes 108 are determined. If a node receives the first mark 304 at two ports simultaneously, then an arbitrary designation of the root port 112 is made by the node processing unit 500 of the node 108. For example, node 108(8) may receive a mark 304 retransmitted by destination node 108(2) and 108(4) at the same time at its east and north ports. The node 108(8) must choose which is to be the root port 112 in accordance with an arbitrary algorithm programmed into its node processing unit 500.

Figures 2A-E illustrate in more detail the propagation of a mark 304 or other broadcast symbol through an arbitrary topology. In accordance with the present invention, no pre-configuration or address assignment is required, the broadcast of a symbol always terminates regardless of any redundant paths or loops in the topology, and the broadcast symbol propagates throughout the network with minimum delay compared to any and all other fabric communication. The example of Figures 2A-E illustrates the propagation of broadcast symbols in a two-dimensional toroidal fabric. However, the broadcast method applies to meshes, toroids, hypercubes, or any other topologies of any number of dimensions, as long as the inter-nodal links have lanes flowing in both directions. Five stages (times) of broadcast propagation are shown in Figures 2a-2e. In Figure 2a, a mark 304 is broadcast by a node 108(A) to its immediate neighbor nodes 108(B). As shown in Figure 2b, a node 108(B) that receives the first instance of a mark 304 clones and retransmits the mark symbol 304 to all ports 1 12, 120, which results in the marks 304 being transmitted to nodes 108(C) and being returned to node 108(A) along the outbound lane of the link 1 16 from which the mark 304 arrived from node 108(A). Thus, node 108(A) will receive, on all of its inbound lanes 1 12, 120, copies of the mark 304 which it transmitted. However, nodes 108 do not rebroadcast copies of a mark 304 symbol which it has just transmitted. Accordingly, upon receipt of a mark 304, the node processing unit 500 determines whether a received mark 304 is a first instance of the mark 304 or whether the node 108 is receiving a copy of a mark 304 that the node 108 has just sent. This is preferably achieved either by an identifier field within the mark 304 that distinguishes it from the mark 304 of the previous or following cycle 300, or by a timeout which is set as being greater than the worst-case mark return time and less than the isochronous cycle time. Responsive to determining that the received mark 304 is a copy of a mark 304 just sent, the node 108 prohibits retransmission of the mark 304, thus providing the self-terminating ability of the fabric 100.

The propagation methodology is further shown in Figure 2c. Each node 108(C) transmits the mark 304 to all of its neighbor nodes 108(D) and also back to the source node 108(B). At this stage, node 108(A) does not receive or transmit a mark 304 because in the previous time period of Figure 2b, the marks 304 it received were all copies of marks 304 which it had sent out. In Figure 2d, the nodes 108(C) are retransmitting marks 304 to the other nodes 108(C), 108(D), and 108(E). Nodes 108(B) have stopped receiving and transmitting marks 304. In Figure 2e, only node 108(E) is still propagating a mark 304. After this time period, the propagation of the original mark 304 is complete, and has self-terminated. At each node 108, the arrival times at each inbound lane fix the path-dependent variance in actual time of the same logically simultaneous event — the launching of the broadcast symbol at a single instant in time with respect to node 108(A). The cycle master 104 launches a mark from every fabric port; all of its ports 120 are leaves. This node 104 identifies the start of every isochronous cycle 300 by its issuance of marks 304, acting as the isochronous time reference for all other nodes 108 in the fabric. As described above, the immediate neighbor nodes 108 echo the mark back to the cycle master 104 as soon as they receive the marks 304; therefore while slot zero occurs simultaneously on each outbound lane of the cycle master 104, slot zero on each inbound lane occurs later, and is dependent on the distance of the neighbor node 108 and of the speed with which the neighbor node 108 responds. As long as the distance and speed are substantially invariant, the neighbor nodes 108 are able to track the passage of time responsive to the receipt of the mark 304. Although slot zero (or time zero) is occurring for all different nodes 108 at different "times", the ticks of the clock, i.e., the time between receipt of different slots 312 is constant, and provides the necessary temporal reference in order to allow the nodes 108 to communicate synchronously. A plesiosynchronous network, in contrast, denies the ability of each node to know the time by counting its own clock cycles. This ability is restored in accordance with the present invention by restricting the insertion and deletion of symbols to those that are issued with the above transmission method, shown in the example above as originating at node 108(A). Node 108(A) may be called the isochronous cycle master node 104.

Figure 5 is a block diagram illustrating a node 108. A node 108 has a root port 112, and three leaf ports 120. A link 1 16 having both an inbound and outbound lane is coupled to each port 112, 120. As described above, the root port 112 is the port at which a first instance of a mark 304 is received. As discussed above, the root port 1 12 is designated on initialization of the fabric 100, when a first mark 304 is transmitted through the system. In the event that a first instance of a mark 304 is received simultaneously on two ports 112, 120, the node processing unit 500 arbitrarily selects which port 112, 120 will be designated the root port 112, and then uses that designation subsequently. The node processing unit 500 is a microprocessor, microcontroller, or other integrated circuit computing device as is known in the art for performing logical operations in conformance with programming. The node processing unit 500 can either be a specially designed unit to perform the network management functions, or the node processing unit 500 functionality can be performed by an existing computing device already present in the node 108. A node-local clock source 508 is coupled to the node processing unit 500 to provide timing to the processing unit 500. The node-local clock source 508 is also coupled to the logic which transmits symbols on the outbound lane of every link 1 16.

An elasticity buffer 504 is positioned within the node 108 at the inbound lane of each link 116. Each elasticity buffer 504 discards all gaps 320 arriving at its input, but stores all other inbound symbols 304, 316, 324. The node processing unit 500 preferably reads one symbol simultaneously from the output of every elasticity buffer 504. Whenever one or more elasticity buffer 504 does not have a symbol available for reading, the node processing unit 500 pauses, and causes a gap 320 to be transmitted simultaneously at the outbound lane of every link 116. Therefore, by allowing each node 108 to delete all received gaps 320 on inbound lanes, and to insert gaps 320 as needed on outbound lanes, the nodes 108 can process symbols in accordance with their node-local clock sources 508 irrespective of the different clock sources being used throughout the fabric 100. However, due to the insertion and addition of gaps 308, the enumeration of slots 312 is maintained and nodes 108 can mark time and interpret information by counting the number of slots 312 received after a mark 304 has been received to process TDM data.

It is the responsibility of each node 108 to preserve a predictable correspondence between a slot number and its occupying isochronous symbol 316, as the isochronous symbol 316 is forwarded from an inbound lane of the node 108 to an outbound lane of the node 108. The slot number of a particular isochronous symbol 316 may be altered as it passes through a particular node 108, but that alteration must be consistent for all isochronous symbols 316 that are forwarded through a node 108. to allow other nodes 108 to interpret the symbol correctly. Nodes 108 may add or delete gaps 320 which alter the position of a symbol in an iCycle 300 as described above. However, as gaps 308 are not enumerated, their deletion or addition does not alter the logical enumeration of the slots 312 in the iCycle 300.

The adherence of each node 108 to constancy of slot assignment provides a method of mark propagation error detection for the fabric 100, as illustrated in Figure 6. First, the incoming node 108 determines 600 a predicted slot arrival for a next mark 304. Preferably, each mark 304 has a field which enumerates its own sequential number and provides a field to predict the slot 312 at which the next mark 304 will arrive. In this embodiment, decisions regarding the insertion or deletion of gaps 308 must be made in advance to accurately predict the arrival of the next mark 304. Next, the next mark 304 is received 604. The slot containing the mark 304 is compared 608 to the predicted slot number. The node 108 determines 612 whether the slot numbers match. If they match, the mark 304 is processed 614 normally. If they do not match, an error message is transmitted 616 to a fabric manager to indicate that an isochrony error has occurred. The fabric manager, a conventional network traffic manager as is known in the art but as modified in accordance with the teachings of the present invention, can then take steps to determine the nature of the problem. The possibly erroneous mark 304 is not propagated 620 by the node 108 which detected the mark 304, which prevents the error from spreading throughout the fabric 100. Instead, a new mark 304 is created and propagated 624 within the predicted slot 312 of the previous mark 304, thus providing the correct timing for the rest of the fabric 100. Alternatively, if the mark 304 was received after the predicted time, the absence of the mark 304 at the predicted time causes the node 108 to generate and propagate a substitute mark 304 at the predicted time. Upon receipt of the delayed mark 304, the mark 304 is prevented from propagating and an error message is sent. Thus, errors in the fabric 100 are quickly detected and are prevented from propagating throughout the system.

The above description is included to illustrate the operation of the preferred embodiments and is not meant to limit the scope of the invention. The scope of the invention is to be limited only by the following claims. From the above discussion, many variations will be apparent to one skilled in the art that would yet be encompassed by the spirit and scope of the present invention.

Claims

Claims:

1. A method of processing time-domain multiplexed (TDM) data and asynchronous data in a fabric comprising individual nodes, wherein each node is not required to have similar processing capabilities and similar clock speeds, comprising the steps of: receiving a mark symbol; identifying a beginning of an isochronous cycle responsive to receiving the mark symbol; receiving a symbol having information to be processed; identifying the symbol responsive to its position relative to the received mark; and processing the mark responsive to its identity.

2. The method of claim 1, further comprising: receiving a gap symbol; and deleting the gap symbol.

3. The method of claim 1 , wherein a node requires symbols for processing at a frequency responsive to its processing capability and clock speed, the method further comprises the steps of: requiring a symbol for processing; inserting a gap symbol; transmitting the inserted gap symbol.

4. A method of determining local time by a destination node in network of connected nodes, in which slots containing data symbols and mark symbols are transmitted by a central node, and the mark symbols indicate a beginning of an isochronous cycle, said method comprising the steps, performed by the destination node, of: receiving a mark symbol at a root port; responsive to receiving a mark symbol at a root port, designating a next slot received at the root port as an initial slot; and determining local time as a function of a number of slots received after the initial slot.

5. The method of claim 4 wherein the step of determining local time is determined responsive to a time stamp of the received mark symbol.

6. The method of claim 4 wherein the destination node has ports for receiving and transmitting symbols, said method comprising the additional steps of: receiving an initial mark symbol at a first port; and designating the port which received the initial mark symbol as a root port.

7. The method of claim 6 further comprising the step of: retransmitting the received initial mark symbol to at least one node connected to at least one port of the destination node.

8. The method of claim 7 wherein the retransmitting step further comprises: retransmitting the received initial mark symbol to a node that sent the initial mark symbol to the destination node.

9. The method of claim 7 wherein the retransmitting step further comprises: retransmitting the received initial mark symbol to all nodes connected to all ports of the destination node.

10. The method of claim 6 further comprising the steps of: receiving a mark symbol at a port; determining whether the received mark symbol has already been received; and responsive to determining that the received mark symbol has already been received, prohibiting retransmission of the mark symbol.

11. In a network of connected nodes in which at least one node processes symbols at a different rate than at least one other node, in which a cycle master node is connected to at least one destination node, in which a mark symbol is transmitted to indicate a beginning of an isochronous cycle, slots containing data having informational content are transmitted and gap symbols without information content are transmitted, a method of ensuring isochronicity between nodes, said method comprising the steps of: determining processing capabilities of at least one node in the network; 9 identifying a node having a slowest processing capability in the

I o network; and

I I the cycle master node transmits a number of slots which does not

12 exceed the processing capabilities of the node having the

13 slowest processing capability.

1 12. The method of claim 11 wherein the cycle master has a processing capability

2 greater than the processing capability of the node having the slowest processing

3 capability, the cycle master further performing the step of:

4 transmitting a number of gap symbols responsive to a difference

5 between the processing capability of the cycle master node and

6 the node having the slowest processing capability.

1 13. The method of claim 12 wherein the step of transmitting a number of gap

2 symbols further comprises the substep of:

3 determining a number of slots which may be transmitted prior to

4 transmitting a gap; and

5 transmitting a gap responsive to the determined number of slots having

6 been transmitted.

1 14. The method of claim 13 wherein the number of slots which must be transmitted

2 after which a gap is to be transmitted is selected to evenly distribute gaps within an

3 isochronous cycle.

1

15. A system for transmitting information between nodes of a network, comprising: a cycle master node, for generating gap symbols, isochronous symbols, and mark symbols, and for transmitting the symbols through ports of the cycle master node substantially in accordance with a timing signal received from a clock coupled to the cycle master node; and at least one destination node, coupled to receive symbols transmitted by the cycle master node, and coupled to a local clock source, for determining local time responsive to determining a number of symbols that have been received after a last received mark symbol, and for transmitting symbols to any other destination nodes substantially in accordance with the local clock source.

16. The system of claim 15 wherein the at least one destination node comprises: an input buffer, for storing isochronous symbols in a queue prior to transmission; and a node processing unit, coupled to the input buffer, for inserting a gap symbol into the queue responsive to determining that an outbound number of symbols exceeds an inbound number of symbols.

17. The system of claim 16 in which the node processing unit deletes a gap symbol from the queue responsive to determining that an inbound number of symbols exceeds an outbound number of symbols.

18. In a network of connected nodes, in which a cycle master node is connected to at least one destination node, in which a mark symbol is transmitted to indicate a beginning of an isochronous cycle, isochronous symbols containing data having informational content are transmitted, and gap symbols without informational content are transmitted, a method of processing symbols by a destination node comprising the steps of: receiving an isochronous symbol; and responsive to determining that a number of inbound symbols exceeds a number of outbound symbols, inserting a gap symbol.

19. The method of claim 18 further comprising the steps of: receiving a gap symbol; and responsive to determining that an outbound number of symbols exceeds an inbound number of symbols, discarding the received gap symbol.

20. A method of detecting erroneous transmission of data in a network of interconnected nodes, said method comprising the steps, performed by a destination node, of: receiving a mark symbol; predicting a slot number at which a next mark symbol will arrive; receiving a next mark symbol; comparing an actual slot number at which the next mark symbol arrived to the predicted slot number; and responsive to the predicted slot number differing from the actual slot number, transmitting an error message.

21. The method of claim 20 further comprising the step of: prohibiting transmission of the next mark responsive to determining that the predicted slot number differs from the actual slot number.

22. The method of claim 21 further comprising the step of: recreating a mark symbol to replace the next mark; and transmitting the recreated mark symbol at the predicted slot number.