AU703403B2 - Method and apparatus for asynchronous transfer mode (ATM) network - Google Patents

Description

Rcgulation 3.2

AUSTRALIA

Patents Act 1952 COMPLETE SPECIFICATION FOR A STANDARD PATENT

(ORIGINAL)

r Name of Applicant: NETWORK EQUIPMENT TECHNOLOGIES, INC.

Actual Inventors: Address for Service: Invention Title: Peter NEWMAN Gururaj SINGH Willie T. GLOVER Amar GUPTA Clifford James BUCKLEY DAVIES COLLISON CAVE, Patent Attorneys, 1 Little Collins Street, Melbourne, 3000.

Method and Apparatus for Asynchronous Transfer Mode (ATM) Network The following statement is a full description of this invention, including the best method of performing it known to us: -1- F "W la METHOD AND APPARATUS FOR ASYNCHRONOUS TRANSFER MODE(ATM) NETWORK BACKGROUND OF THE INVENTION A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

The present invention relates to communication systems and more particularly to asynchronous transfer mode (ATM) communication networks, A communication network functions to interconnect many network users. Each user is connected to the network at a port.

Typically, the network is formed by a number of interconnected nodes where information (from a user at a source) is input at an input port, is passed by the input port from node to node through the network, and is output at an output port (to a user at a destination).

Information can be communicated through a communication network in various forms. For example, time-sensitive voice, video and circuit emulation information is typically transmitted in constant bit rate (CBR) form while computer and other nontime-sensitive information is typically packetized and communicated in variable bit rate (VBR) form.

In communication networks, each node switches incoming information at input ports to outgoing information at output ports. For time-sensitive information, the communication through the ATM network is in CBR (constant bit rate) form and for packetized information the communication through the ATM network is in VBR (variable bit rate) form.

Fast packet switches typically can transfer hundreds of thousands of packets per second at every switch port in a communication network. Each switch port is typically designed to transfer information at a rate from 50 Mbits/s to 600 Mbit/s 2 for broadband integrated service digital networks (ISDN).

Switch sizes range from a few ports to thousands of ports.

The term "fast packet switch" includes switches capable of z handling both variable-length packets of information and fixedlength packets of information. Fixed-length packets of information are desirable since they simplify the switch design. Fast packet switches using short, fixed-length packets (called "cells") are referred to as asynchronous transfer mode (ATM) switches. ATM switches connected together form an ATM network.

For ATM networks and switches, the term "packet" refers to variable-length information. Typically, a packet is hundreds and sometimes thousands of 8-bit bytes in length. For ATM networks and switches, the term "cell" refers to short (relative to packets of maximum length), fixed-length information.

Typically, a cell is 53 3-bit bytes in length.

Fast packet switches are capable of handling different types of communications services in a single integrated communication network including the time-sensitive voice, video and circuit emulation information in constant bit rate (CBR) form and including computer and other non-time-sensitive information in variable bit rate (VBR) form. Although voice and video services can tolerate only a limited amount of delay and delay variance through a communication network, ATM switches are suitable for such services since they minimize delay and delay variance. ATM networks are suitable for concurrently transmitting both CBR time-sensitive and VBR non-time-sensitive information from sources to destinations through an ATM network having many differnt users.

The ATM standard for broadband ISDN networks defines a cell having a length of 53 bytes with a header of 5 bytes and data of 48 bytes. Broadband ISDN defines two transmission bit rate standards of 155M bit/s and 622M bits/s so that the cell clock periods are 2.7 microseconds and 0.68 microseconds, respective- 1 3 ly, and the cell clock rates are 0.36M cells/sec and 1.47M cells/sec, respectively.

In a typical ATM switch, tl-a cell processing functions are performed within the nodes of a network. Each node is an ATM switch which includes input controllers a switch fabric output controllers (OC's) and a node control The node control is used for functions including connection establishment and release, bandwidth reservation, congestion control, maintenance and network management.

In each switch, the input controllers are typically synchronized so that all cells from input controllers arrive at the switch fabric at the same time and so that cells can be accepted or rejected according to their priority. The traffic through the switch fabric is slotted and the switch fabric delay equals the sum of the timeslot duration, pipeline delay, and the queueing delay.

The node control communicates with the input controllers and the output controllers either by a direct communication path which by-passes the switch fabric or via control cells transmitted through the switch fabric.

External connections to the switch are generally bidirectional. Bidirectional connections are formed by grouping an input controller (IC) and an output controller (OC) together to form a port controller (PC).

The input sequence of cells in a virtual channel is preserved across the switch fabric so that the output sequence of cells on each virtual channel is the same as the input sequence. Cells contain a virtual channel identifier VCI in the cell header which identifies the connection to which the cell belongs. Each incoming VCI identifier in the header of each cell is translated in an input controller to specify the outgoing VCI identifier. This translation is performed in the input controller typically by table look-up using the incoming 4 VCI identifier to address a connection table. This connection table also contains a routing field to specify the output port of the switch fabric to which the connection is routed. Other information may be included in the connection table on a per connection basis such as the priority, class of service, and traffic type of the connection.

In an ATM switch, cell arrivals are not scheduled. In a typical operation, a number of cells may arrive simultaneously at different input ports each requesting the same output port.

Operations in which requests exceed the output capacity of the output port are referred to as output contention (or conflict) Since an output port can only transmit a fixed number (for example, one) cell at a time, only the fixed number of cells can be accepted for transmission so that any other cells routed to that port must either be discarded or must be buffered in a queue.

Different methods are employed for routing cells through a switch module, for example, self-routing and label routing.

A self-routing network operates with an input controller prefixing a routing tag to every cell. Typically, the input controller uses a table lookup from a routing table to obtain the routing tag. The routing tag specifies the output port to e which the cell is to be delivered. Each switching element is able to make a fast routing decision by inspecting the routing tag. The self-routing network ensures that each cell will arrive at the required destination regardless of the switch port at which it enters.

A label routing network operates with a label in each cell referencing translation tables in each switching element. The label is translated in each switching element and hence any arbitrary network of switching elements may be employed.

Switches have two principal designs, time-division and space division. In a time-division switch fabric, all cells flow 5 through a single communication channel shared in common by all input and output ports. In a space division switch, a plurality of paths are provided between the input and output ports. These paths operate concurrently so that many cells may be transmitted across the switch fabric at the same time. The total capacity of the switch fabric is thus the product of the bandwidth of each path and the average number of paths that can transmit a cell concurrently.

When the traffic load exceeds the available system resources in a network, congestion is present and performance degrades.

When the number of cells is within the carrying capacity of the network, all cells can be delivered so that the number of cells delivered equals the number of cells sent without congestion.

However, if cell traffic is increased to the level that nodes cannot handle the traffic, congestion results.

:Congestion can be brought about by several factors. If nodes in a network are too slow to perform the various tasks required of them (queueing buffers, updating tables, etc.), queues build up, even though excess line capacity exists. On the other hand, even if nodes are infinitely fast, queues will build up whenever the input traffic rate exceeds the capacity of the output traffic rate for any particular group of outputs.

If a node has no free buffers for queuing cells, the node must discard newly arriving cells. For packet data traffic, when a cell is discarded, the packet from which the discarded cell came will be retransmitted, perhaps many times, further aggravating the congestion.

A difference exists between congestion control and flow control. Congestion control relates to insuring that each part of the network is able to carry the offered traffic. Congestion control is a network-wide consideration involving the behavior and carrying capacity of each part of the network at the time

I

6 that traffic is offered to each part of the network.

Flow-control, in contrast, relates to the point-to-point traffic between a given source and a given destination. Flow control insures that a source does not continually transmit data faster than the destination can receive it. Flow control generally involves direct feedback from the destination to the source to signal the source as to the availability of the destination to receive transmissions from the source.

Packet Congestion Control Algorithms Congestion control algorithms for packet switches have included preallocating resources, packet discarding, packet o. restricting, flow control, and input choking.

The preallocating of resources avoids a congestion problem for virtual channels. When a virtual channel is set up, a call request travels through the network making table entries at each node that control the route to be followed by subsequent o traffic. With preallocating, each call request reserves one or more data buffers in each node unless all the buffers are already reserved. If all buffers are reserved, another route el is found or a "busy signal" is returned to the source. If o buffers are reserved for each virtual channel in each node, there is always a place to store any incoming packet to that node. The problem with preallocation is that substantial buffer resources are allocated to specific virtual channel connections, whether or not there is any traffic over the virtual channel.

An inefficient use of resources results because resources not being used by the connection to which they are allocated are nevertheless unavailable to other virtual channels.

With packet discarding, if a packet arrives at a node and there is no place to store it, the node discards it. Congestion is resolved by discarding packets at will. A copy of the packet is kept elsewhere and retransmitted later. Discarding packets

I

7 at will, if carried too far, results in inefficient operation since the retransmission tends to increase traffic in an already congested network.

Packet restricting limits the number of packets at any location in a network to avoid congestion since congestion only occurs when there is an excessive number of packets at a particular location in the network. Packet restricting is performed in one method by issuing permits. Whenever a node requests sending a packet, the node must first obtain a permit.

When the destination finally receives the packet, the permit is again made available for reuse. Although the permit method guarantees that the network as a whole will never become acongested, it does not guarantee that a given node will not become swamped with packets. Also, the permit distribution mechanism is difficult to implement without suffering long delays.

Flow control has been used in some networks (for example, SARPANET) in an attempt to eliminate isolated congestion. For example, flow control has been used by the transport layer to keep one host from saturating another host and by one node to prevent that node from saturating its neighbor node. However, flow control has not been effective to solve network-wide congestion.

Input choking has been used to control congestion. Whenever congestion is detected, a choke packet is sent back to the source to choke off input packets. To determine congestion, each newly arriving packet at a node is checked to see if its output line is congested. When congestion is detected at a node, the node sends a choke packet back to the source identifying the destination having congestion. When a source receives a choke packet, the source reduces by some factor the traffic sent to the specified destination. Other packets having the same destination that are already under way generate additional 8 choke packets that are ignored by the source for a first time period. After that first time period has expired, the source looks for more choke packets for a second time period. If during the second time period a choke packet arrives indicating that the line is still congested, the source reduces the flow still more and repeats the process. If no choke packets arrive during a subsequent second time period, the source increases the flow again. The feedback in this protocol tends to prevent congestion but does not throttle flow until congestion is detected.

Several variations on the input choking algorithm have been proposed for packet switches. In one variation, the nodes maintain two congestion thresholds. Above the first threshold but below the second, choke packets are sent back. Above the second threshold, incoming traffic is discarded, it being assumed by the second threshold that the host has received notification of the congestion.

Another input choking variation is to use queue lengths instead of line utilization as the trigger signal for congestion. Another choking variation has nodes propagate congestion information along with routing information, so that the congestion trigger is not based on information from only one node, but on information that somewhere along the path there is congestion. By propagating congestion information in the network, choke packets can be sent at an early time, before too many more packets are under way, thus preventing congestion from building up.

United States Patent No. 5,457,687 discloses a method and apparatus for reactive congestion control in an asynchronous transfer mode (ATM) network where the network is formed by the interconnection of nodes. Each of the nodes includes a forward path for P (41 .P (01. 1 IN', Lt.M, 1 :)9 -9transfer of information from source to destination through the network and a return path for returning explicit congestion control signals. Each source includes a modifiable issue rate unit which issues forward information signals at different rates for virtual channels in response to the presence and absence of congestion signals received on the return path. A variable issue rate unit has an exponential rate for reducing the issue rate in response to receipt of congestion signals. The variable rate source has an exponential increase in the issue rate in the absence of congestion control signals after the issue rate has t0 been reduced.

There is a further need for practical corponents for use in ATM networks that are capable of processing cells for a plurality of virtual channels.

SUMMARY OF THE INVENTION According to the present invention there is provided a communication system having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, I r p (J'Ili R I itll x pj I said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes.

The invention also provides a wide-area communication system ving local reactive congestion control comprising, a plurality of local communication systems, each local communication system including, a plurality of local sources, each local source including a modifiable issue rate unit which transmits forward information signals at different transmission rates, said issue rate unit decreasing the transmission rate in response to the presence of congestion signals and increasing the transmission rate in response to the absence of congestion signals, a plurality of local destinations, one or more local nodes forming an asynchronous transfer mode (ATM) local network connecting the sources to the destinations, each local node including, a forward path for transfer of information at different selectable transmission rates in a forward direction toward a destination through the network, a return path for returning congestion signals in a Sreverse direction through the network toward a source, I I Ldi I 111 i (iWI Si i71 1'%1iM 1. 29 10A congestion signal generating means for generating congestion signals in response to congestion in the local node, said network having a plurality of virtual channels whore, for each virtual channel, one of said plurality of local sources is linked to one of said plurality of local destinations through said local network by linking forward paths from one or more local nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of local destination: is linked to said one of said plurality of local sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said local nodes generates said congestion signals for the virtual channel in response to congestion in said oner o said local nodes, one or more wide-area sources connected to one ot said local networks, one or more wide-area destinations connected to another one of said local networks, a plurality of local network links interconnecting two or more local networks to form a wide-area network, said wide-area network having a plurality of virtual channels where, for each virtual channel, one of said plurality of wide-area sources is linked to one of said plurality of wide-area dstinat ions through a plurality of said local networks over said local networks and said local network links.

The invention also provides a communication system of one or more virtual channels having reactive congestion control comprising, -P a plurality of sources, each source including a modifiable P i i'l',jIA7% W'A1'.1 1 1'JB issue rate unit which tran: .a forward information tiqnals at different transmission rates in response to congestion signals, said issue rate unit transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate in the absence of congestion and at a rate below the maximum-channel-peak-cellrate for each channel experiencing congestion, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a ;source, congestion signal generating means for generating congestion signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality cf destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rat's in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes.

The invention also provides a multiple issue rate source for use in a communication system of one or more virtual channels having reactive control where said system includes, one or more of said multiple issue rate sources a plurality of destinations, I one or more nodes forming an asynchronous transfer mode

I

1 1ilit Ii' S I vI i? M IV,W 1C (ATM) network connecting sources to destination1, oah node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning control signals in a reverse direction through the network toward a source, control signal generating means for generating control signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, said one of said plurality of destinatiuns is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the control signal generating means in one or said nodes generates said control signals for the virtual channel in response to congestion in said one of said nodes, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit forward information signals at different transmission rates in response to control signals, said issue rate unit having an output for transmitting, for each of one or more virtual channels, at a maximum-channel-peak-cell-rate in the absence of a control signal and at a rate below the maximumchannel-peak-cell-rate for each channel providing a control signal, The invention also provides a multiple issue rate source for use in a communication system where the communication system has one or more multiple issue rate sources connected through one or more channels to a plurality of destinations and has control signal generating means for generating control signals for the channels for reactively controlling the transmission rates over the channels, said multiple issue rate source IZ4 comprising, 11 (11-1-4 A 'I 'SI I% Is I j ')9 10D a modifiable issue rate unit having means to transmit information signals at different transmission rates in reponse to control signals, said issue rate unit having an output for transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate or at a rate below the maximum-channel-peak-cell rate as a function of the control signals for each channel.

The invention also provides a communication system having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals and in response to average rate metering, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to 11O RIG J J7 I'i7'i Ib t!-i UE said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion.

The invention also provides a wide-area communication system having local reactive congestion control comprising, a plurality of local communication systems, each local communication system including, a plurality of local sources, each local source including a modifiable issue rate unit which transmits forward information signals at different t.-ansmission rates, said issue rate unit decreasing the transmissi.,n rate in response to the presence of congestion signals and ilicreasing the transmission rate in response to the absence of congestion signals and in response to average rate metering, a plurality of local destinations, one or more local nodes forming an asynchronous transfer mode (ATM) local network connecting the sources to the destinations, each local node including, a forward path for transfer of information at different selectable transmission rates in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the local node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of local sources is linked to one of said plurality of local destinations through said local network by linking forward paths from one or more local nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, S 1 A It f I'>'iOi7 9'6 1 I4 I 2IN 1OF said one of said plurality of local destinations is linked to said one of said plurality of local sources through said network by linking return paths from said one or more nodes, the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion, one or more wide-area sources connected to one of said local networks, one or more wide-area destinations connected to another one of said local networks, a plurality of local network links interconnecting two or more local networks to form a wide-area network, said wide-area network having a plurality of virtual channels where, for each virtual channel, one of said plurality of wide-area sources is linked to one of said plurality of wide-area destinations through a plurality of said local networks over said local networks and said local network links.

The invention also provides a communication system of one or more virtual channels having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals, said issue rate unit transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate in the absence of congestion and at a rate below the maximum-channel-peak-cellrate for each channel experiencing congestion and transmitting under control of average rate metering, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a P' DI T IO(t I' P SOWS r, III %1 1+,1;99 10G forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion.

The invention also provides a communication system having one or more virtual channels each having reactive control and average rate metering where said system includes, one ore more multiple issue rate sources, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning control signals in a reverse direction through the network toward a source, control signal generating means for generating control signals in response to congestion, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of Ssaid plurality of destinations through said network by linking P OIT It( Al 511S IS 96 t T I I V) 10H forward paths from one or more nodes, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the control signal generating means in one of said nodes generates said control signals for the virtual channel in response to congestion, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit forward information signals at different transmission rates in response to control signals and average rate metering, said issue rate unit having an output for transmitting, for each of one or more virtual channels, at a maximum-channel-peak-cellrate in the absence of a control signal and at a rate below the maximum-channel-peak-cell-rate for each channel providing a control signal.

The invention also provides a multiple issue rate source for use in a communication system where the communication system has one or more multiple issue rate sources connected through one or more channels to a plurality of destinations and has control signal generating means for generating control signals for the channels for reactively controlling the transmission rates over the channels and for controlling the transmission rates over the channels with average rate metering, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit information signals at different transmission rates in response to control signals, said issue rate unit having an output for transmitting, for each of one or more channels, at a maximumchannel-peak-cell-rate or at a rate below the maximum-channelpeak-cell-rate as a function of the control signals for each channel.

The invention also provides a communication system having N reactive congestion control comprising, p fli'l~I~ b(piI%?~ ~'I.N1 2 101 a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals and includes average rate metering means for insuring that the transmission rate does not exceed an average rate, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes.

The foregoing features and advantages of the invention will be apparent from the following detailed description in X conjunction with the drawings.

r L I I I 11 BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 depicts a block diagram of communication system including a source communication unit connected by an ATM network to a destination communication unit.

FIG. 2 depicts a block diagram of the FIG. 1 system in which the source includes a segmentation unit and the destination includes a reassembly unit.

FIG. 3 depicts a schematic block diagram of a plurality of source/destination users connected through a multinode ATM network.

o, FIG. 4 depicts a schematic representation of a circuit with one S/D user connected to another S/D user through a sequence of nodes in the network of FIG. 3.

FIG. 5 depicts a schematic representation of the FIG. 4 circuit with a virtual channel connection of the source (S) sending information in a forward direction to a destination and with a reverse direction for transmitting control signals to the source FIG. 6 depicts a schematic representation of a typical one of the nodes in the FIG. 3 network.

FIG. 7 depicts further details of one embodiment of the typical node of FIG. FIG. 8 depicts a schematic block diagram of a plurality of source/destination users connected through a wide-area multinode network formed of a plurality of local networks.

FIG. 9 depicts timing diagrams representative of the operation of a source in response to congestion.

FIG. 10 depicts a schematic block diagram of the source of FIG. 1.

FIG. 11 depicts a schematic block diagram of the destination of FIG. 1.

FIG. 12 depicts a representation of the two-dimensional rate 1 Il 12 queues used in the segmentation unit in the source of-FIG. FIG. 13 depicts a timing diagram representing the pipelining operation of the segmentation unit in the source of FIG. FIG. 14 depicts a timing diagram representing the pipelining operation of the reassembly unit in the destination of FIG. 11.

FIG. 15 depicts timing diagrams representative of the operation of a network with the segmentation unit in the source of FIG. 9 and the reassembly unit in the destination of FIG. 11.

FIG. 16 depicts a timing diagram representing the cell clock for the segmentation unit of FIG. FIG. 17 depicts a timing diagram representing the Data_RequestTransmit (DRT) queue for the segmentation processing unit ~of FIG. FIG. 18 depicts a timing diagram representing the TIQ signal from the TI counter.

FIG. 19 depicts a timing diagram representing the Cell_Ctr signal for the segmentation unit of FIG. FIG. 20 depicts a timing diagram representing the Data- Transmit signal for the segmentation processing unit of FIG.

I

I 13 DESCRIPTION OF THE PREFERRED EMBODIMENTS Source/Destination ATM System FIGS. 1 and 2 In FIG. 1, a source communication unit 4-S transmits cells over an asynchronous transfer mode (ATM) network 1 to a destination communication unit 4-D. The geographical area of ATM network 1 can extend between local hosts (backplane connection), can be a local-area network (LAN), a wide-area network (WAN) and, in general, can extend to any size ATM network. In FIG.

1, the source communication unit 4-S and the destination communication unit 4-D are typical of the many sources and destinations that commiunicate through ATM network 1. In FIG.

1. 1, sources and destinations are typically organized in pairs and S such pairs are designated as sources/destinations 4-1, In FIG. 2, further details of the source 4-S and destination 4-D of FIG. 1 are shown that are typical of the sources/destinations 4-1, of FIG. 1. In FIG. 2, the source and destination communication units each include a host unit (101 and 101'), a cell unit (103 and 103') and common memory (102 and 102'). In the source 4-S, the cell unit (103) is a segmentation processing unit for segmenting packets for transmission on ATM path 108 over an asysnchronous transfer mode (ATM) network 1 to ATM path 109 to destination 4-D. Packets from the host to be transmitted are segmented into a plurality of cells concurrently for a plurality of channels and the cells are transmitted over the asysnchronous transfer mode (ATM) network 1. Cells received from the asysnchronous transfer mode (ATM) network 1 are reassembled concurrently for the plurality of channels in the destination 4-D by the cell unit 103'. In the destination, the cell unit 103' is a reassembly processing unit that reassembles the packets concurrently for the plurality of channels.

The cell units 103 and 103' are pipelined processing units ~L I 14for segmentation and for reassembly, respectively, of-packets.

In FIG. 2, the source 4-S includes a host unit 101, a common memory 102, and a cell unit 103. The common memory 102, in a preferred embodiment, is a dual ported memory having a port for the host unit 101 and having a port for the cell unit 103. The host unit 101 includes a host memory interface 106, for interfacing to one of the ports of memory 102. Similarly cell unit 103 includes a cell unit memory interface 107 for interfacing to the other one of the dual ports of the dual port memory 102. The dual ported memory 102 of the source 4-S of FIG. 1 allows packet information from the host unit 101 to be stored into the common memory 102. The cell unit 103 retrieves the 0 packet information from the common memory 102, segments the packets into cells, and transmits the cells over the ATM path °•eJ So"108 to the ATM network 1 over the ATM path 109 to the destinacoo tion 4-D.

In FIG. 2, the destination 4-D includes a host unit 101', ceao a common memory 102', and a cell unit 103'. In a preferred embodiment, the common memory 102' is a dual ported memory which ~connects through one port to the host unit 101' and through the S other port to the cell unit 103'. The host unit 101' includes a host memory interface 106' and the cell unit 103' includes a "cell unit memory interface 107'. The cell unit 103' receives the cells from the segmented packets on the ATM path 109 and reassembles those cells into packets for a plurality of channels. The cell unit 103' deposits the cells into the common memory 102' where they become available to the host unit 101' as packets for a plurality of channels.

In FIG. 2, the common memory 102 is partitioned into a control memory 81 and a packet memory 83. Similarly. the control memory 102' is partitioned into a control memory 81' and a packet memory 83'. In a preferred embodiement, each of the control memories 81 and 81' is a separate dual ported memory and

I_.

15 each of the packet memories 83 and 83' is also a separate dual ported memory.

The cell unit memory interface 107, therefore, includes a cell unit control memory interface 82 which connects the cell unit to the control memory 81 and includes a cell unit packet memory interface 84 which connects the cell unit 103 to the packet memory 83. The cell unit 103 also includes a cell interface 126 which connects the cell unit 103 to the ATM path 108.

In the destination 4-D, the cell unit 103' includes a cell interface 126' for receiving over the ATM path 109 cell data from the source 4-S. The cell unit 103' includes a cell unit control memory interface 82' which connects cell unit 103' to the control memory 81' in the.common memory 102'. Also, the cell unit 103' includes a cell unit packet memory interface 84' for connecting the cell unit 103' to the packet memory 83' of the common memory 102'.

In FIG. 2, each of the cell processing units 103 and 103' includes four interfaces. Namely, processing unit 103 includes the host interface 127, control memory interface 82, the packet memory interface 84, and the cell interface 126. Similarly, the control unit 103' includes the four interfaces, namely the host interface 127', the control memory interface 82', the packet memory interface 84', and the cell interface 126'.

In accordance with one preferred embodiment of the present invention, each of the cell units 103 and 103' is a pipelined processing unit which operates to present information concurrently at each of the four interfaces (host interfaces 127 and 127', control memory interfaces 82 and 82', packet memory interfaces 84 and 84', cell interfaces 126 and 126').

In addition to the segmentation and reassembly processing of the FIG. 2 cell units, the source 4-S typically includes variable issue rate means which varies the rate at which cells

IL

16 are issued to the ATM network 1.

In one preferred embodiment, the issue rate is controlled within the source 4-S in accordance with reactive congestion signals received from the ATM network I as described in connection with the above-identified application METHOD AND APPARATUS FOR REACTIVE CONGESTION CONTROL IN AN ASYNCHRONOUS TRANSFER MODE (ATM) NETWORK.

Overall Network FIG. 3 In FIG. 3, a plurality of network users are represented as the source/destination 4. Each user typically sends information as a source and receives information as a destination The source of an S/D unit 4 will send information to the destination of some other S/D unit 4.

In order for information to be transferred from a source to a destination, each S/D unit 4 connects through a multinode (N) network 1. The network 1 includes many nodes 5. The nodes are connected from node to node so that, in general, any particular one of the S/D units 4 can connect to any one of the other S/D units 4 by forming a virtual channel connection across a chain of nodes 5 in the network 1. In general, the links 'between the S/D units 4 and a node 5, and the links between nodes 5, are by bi-directional links 8 which enable information to be transferred in both directions.

In FIG. 3, the number of nodes 5 shown is for clarity a relatively small number, but the network may include hundreds or more nodes. Also, the S/D units 4 include the S users 4- 0, 4-1, 4-2, 4-3, 4-4, The value of S can be any integer where S is typically equal to hundreds or higher.

In a typical embodiment, the FIG. 3 communication system is an ATM network in which the unit of transfer of information is a cell. A plurality of cells form packets of information. The 17 network 1 communicates cells and packets so as to support different types of information including images, voice and data.

Virtual Channel FIG. 4 In FIG. 4, the S/D unit 4-x connects through a plurality C of nodes 5-0, 5-1, to the S/D unit 4-y.

In FIG. 4, the S/D unit 4-x is typical of any of the S/D units 4 of FIG. 3. For example, the S/D unit 4-x may represent the S/D unit 4-2 in FIG. 3. Similarly, the S/D unit 4-y in FIG.

4 may represent any of the S/D units 4 in FIG. 3. For example, S/D unit 4-y may represent the S/D unit 4-4 in FIG. 4. In such an example, the nodes 5-0, 5-1, represent the C nodes in the network 1 of FIG. 3 which are used to connect the S/D unit 4-2 to the S/D unit 4-4.

In FIG. 4, the bi-directional links 8-0, 8-1, connect from the S/D unit 4-x through the nodes 5-0, 5-1, to the S/D unit 4-y.

In FIG. 4, information may be transferred from the source in the S/D unit 4-x to the destination in the S/D unit 4-y. Similarly, information from the source in the S/D unit 4-y can be transferred to the destination in the S/D unit i 4-x. While information may be transferred in either direction in FIG. 4, it is convenient, for purposes of explanation to consider transfers between a source and a destination (D) whether that be from the S/D unit 4-x to the S/D unit 4-y or from the S/D unit 4-y to the S/D unit 4-x. Regardless of the direction, each transfer is from a source to a destination S to D Transfers FIG. In FIG. 5, a schematic representation for a source to destination transfer in the virtual channel of FIG. 4 is 18 shown. In FIG. 5, the source unit in the S/D unit 4-x of FIG. 4 connects to the destination unit in the S/D unit 4-y of FIG. 4.

In FIG. 5, each of the links 8-0, 8-1, 8-(C) includes a forward channel for transferring information in the forward direction and a reverse channel for transferring information in the reverse direction. The reverse channel in FIG. 5 is associated with the transfer of information from the source unit to the destination unit The reverse channel in FIG. 5 is for the purpose of sending control information used in connection with congestion control within the network of FIG. 4. The reverse channel is distinguished from the forward channel used for the transfer of information in the forward direction from S/D unit 4-y to S/D unit 4-x, as discussed in connection with FIG. 4. Both the forward (F) and the reverse channels are associated with the source unit transfer to the destination unit Each of the nodes in FIG. 5 includes forward circuitry 6 and reverse (R) circuitry 7. In FIG. 5, the forward channels 8-OF, 8-1F, 8-(C-1)F connect as inputs respectively to the forward circuits 6-1, The forward channel 8-(C)F connects from the forward circuit to the D unit Similarly, the reverse channels 8-OR, 8-1R, 8-(C-1)R connect from the reverse circuits 7-0, 7-1, The reverse channel 8-(C)R connects from the D unit to the reverse circuit In FIG. 5, each of the nodes 5 has a feedback connection 9 connecting from the forward circuit 6 to the reverse (R) circuit 7. Specifically, the feedback channels 9-0, 9-1, connect from the forward circuits 6 to the reverse circuits 7 in the nodes 5-0, 5-1, respectively. In the FIG 5 circuit, a virtual channel connection is made along the forward channel setting up a communication path in the 19 forward direction between the S unit and the D unit 4-(D) Because other virtual channels are also established in the network 1 of FIG. 3, congestion may arise at any node or destination including the nodes of FIG. 5. The feedback channels 9 are vtilized to send back an explicit congestion signal using the reverse channels 8-OR, 8-1R, 8-(C- 1)R, 8-(C)R and reverse circuits 7. When the source unit receives an explicit congestion signal on the reverse channel 8-OR, source unit takes corrective action to reduce the congestion in the network. Specifically, in accordance with the present invention, source unit reduces the rate at which data is inserted over the forward channel through channel 8-OF from the assigned rate (Maximum Channel .:Peak Cell Rate) to some lower rate (Current Channel Peak Cell Rate). After a time period such that congestion is likely to have cleared, the source unit restores the insertion rate of information in the forward channel over the link 8-OF to its original maximum rate.

Since the FIG. 5 virtual channel is typical of all of the virtual channels established in the network of FIG. 3, the combined operation of all the virtual channels in an ATM system is to dynamically control the insertion rate of input informai tion, and specifically to reduce the insertion rate in the presence of congestion in the network and to restore the insertion rate in the absence of congestion in the network.

Each virtual channel, like the FIG. 4 virtual channel, functions with a feedback control for controlling the insertion rate of information into the network. In order for the feedback control in FIG. 5 to have a good response time, the delay in time from the detection of congestion until the source unit takes corrective action should be small. In general, the feedback delay value has two basic components. The first component of delay is the circuit path delay for each node to 20 node transfer required from the point of congestion back to the source. The second component of the delay is the signal processing time required in each node including generation of the congestion signal in each reverse circuit 7 in each node in order to transmit the explicit congestion signal through the node. The total feedback delay is, therefore, the sum of the node-to-node reverse signal transfer times and the sum of the reverse signial processing times. For any given system, the reverse signal transfer time from node to node is not readily changed and is substantially fixed for any particular system.

The present invention causes the reverse signal processing time to be much less than the reverse signal transfer time.

Therefore, substantially all of the feedback delay is a result of the reverse signal transfer time and the feedback processing time is small or negligible.

Single Node Embodiment FIG. 6 In FIG. 6, one typical embodiment of a node having the feedback control reverse signal paths of FIG. 5 is shown. In :FIG. 6, the node 5 includes N links 18-0, 18-1, 18-n, Each of the links 18 of FIG. 6 are analogous to the bi-directional links 8 of FIG. 4.

In FIG. 6, the links 18-0, 18-1, 18-n, 18-(N-1) connect to port controllers 11-0, 11-1, 11-n, The node of FIG. 6 is used in connection with the information transfer of FIG. 4, for example, by having one of the links 18, for example, input link 18-0 in FIG. 6, connect through switch fabric 10 to another one of the links 18, for example, link 18-n. In the example described, the switch fabric functions to connect the link 18-0 to the link 18-n.

In an example where the node of FIG. 6 represents the node 5-1 in FIG. 4, the link 8-1 in FIG. 4 is the link 18-0 in FIG.

6 and the link 8-2 in FIG. 4 is the link 18-n in FIG. 6.

I_

21 With such a connection, the node of FIG. 6, connects information in one direction, for example, from link 18-0 to link 18-n, and connects information in the opposite direction from the link 18-n to the link 18-0. The links 18-0 and 18-n were arbitrarily selected for purposes of explanation. Any of the N links 18 might have been selected in the FIG. 4 circuit for connection to any of the other links 18.

When the node of FIG. 6 is used in the virtual channel connection of FIG. 4 with the source on the left and the destination on the right, then for purposes of explanation it is assumed that the link 18-0 is an input to the node 5 in the forward direction and the link 18-n is output from the node in the forward direction.

In FIG. 6, port controllers (PC) 11-0, 11-1, 11-n, 11-(N-1) have input controllers 14-0, 14-1, 14-n, respectively and have output controllers (OC) 15-1, 15-n, 15-(N-1) respectively. In FIG. 6, forward information cells from the source of FIG. 5 sent to the destination of FIG. 5 connect from the bus 18-01 through the input controller 14-0 to the bus 20-01 through the switch fabric 10 to the bus 20-nI through the output controller 15-n to the bus 18-nO. If congestion is detected at the node then an explicit congestion signal is connected, for example, from the output controller 15-n to the input controller 14-n to the bus 20-nI back through the switch fabric 10 to the bus 20-00 to the output controller 15-0 to the output bus 18-00.

NODE FIG. 7 In FIG. 7, a more detailed embodiment of the node of FIG.

6 is shown. In FIG. 7, the port controller 11-0 is shown typical of all of the port controllers 11 of FIG. 6. Similarly, the port controller 11-n is typical of all of the port controllers 11 of FIG. 6. In FIG. 7, the port controller 11-n is

Y

22 identical to the port controller 11-0 except that the images in port controller 11-n are flipped about 180' in order to provide symmetry in the drawing. As indicated in connection with FIG.

6, in the virtual channel of FIG. 4, the controller 11-0 of FIG.

7 serves as an input for forward going cells while the controller 11-n functions as an output for forward going cells.

In FIG. 7, the port controllers 11-0 and 11-n include input controllers (IC) 14-0 and 14-n each having a selector 50, a translator 51 and a reverse signal queue 52. The port controllers 11-0 and 11-n also include an output controller (OC) 15-0 and 15-n each having a selector (SEL) 53, a queue 54 and a reverse marker (RM) 49.

The operation of the FIG. 7 node in connection with the virtual channel of FIG. 4 is as follows. Forward transmitted cells appear on the bus 18-01 input to the selector 50 of the input controller 14-0. Selector 50 normally is set to select the input bus 18-01 unless the reverse queue 52 has congestion control information which must be returned in connection with the operation of some other virtual channel. If the not empty (N-EMT) signal is asserted indicating that congestion information is stored in the queue 52, selector 50 selects the output from queue 52 but otherwise selects the forward cells on bus 18-01.

The translator 51 for each cell translates the value of the incoming virtual channel identifier (VCI) for the cell to an outgoing VCI for the cell, directs the cell to an outgoing link, and marks the priority of the cell. For example, the translator is implemented as a table storing values where the table is addressed by the incoming VCI.

The cell information together with the translated VCI appear on the bus 20-nO to the selector 53 in the output controller Selector 53 normally selects the bus 20-nQ for connection to the queue 54. If queue 54 is filled beyond a threshold as I I 23 indicated by the Q1 signal selector 53 copies the VCI and control information to the queue 52 through the reverse marker unit 49. Reverse marker unit 49 marks the VCI label with a reverse signal indication so that reverse signals can be distinguished from forward signals.

The threshold signal Ql indicating the fullness of queue 54 is a congestion signal that is transmitted back to the source of FIG. 5. If queue 52 has any information stored, it issues the N-EMT signal indicating a not empty condition to the selector 50 in the input controller 14-n. Selector 50 connects its output to the translator 51. Translator 51 translates the return signal VCI back to the original value (which appeared on the input bus 18-01) so that switch fabric 10 receives the return path congestion signal on the bus 20-00 input to the selector 53. Selector 53, assuming that the queue 54 is not full, connects the return signal to the queue 54 for output on the return bus 18-00. Accordingly, the translator 51 in the input controller 14-n reverses the translation done in the translator 51 in the input controller 14-0.

Wide-Area Network Formed By Plural Local Networks FIG. 8 In FIG. 8, a wide-area communication system employing networks of the FIG. 3 type is shown. The FIG. 8 system is comprised of a group of generally local communication systems including the networks 1-0, 1-1, Each of the local networks of FIG. 7 includes source and destination units of the type previously described in connection with the FIG. 3 network. Specifically, local communication systems are formed by the local network 1-0 together with the local source/destination units 4(0,S 0 the local network 1-1 together with the local the S/D units 4and the local network 1-2 together with the local S/D units 4(2, S 2 and the local network 1-(L-1) 24 together with the local the S/D units 4(L-1, In FIG. 8, the wide-area system includes a wide-area source connected to local network 1-0 and a wide-area destination connected to local network In FIG. 8, the local networks 1 are connected together by links 8 and specifically link 8(0,1) connects local networks and 1-1, link 8(1,2) connects local networks 1-1 and 1-2 and so on such that local network is connected through links 8 and local networks to local network 1-0. In this manner, wide-area source 4-0 is connected to wide-area destination The wide-area S/D units 4-0 and are typical r. of many such units which, for clarity, are not shown in FIG. 8.

In FIG. 8, the links 8(2, 8( ,L-1) between local networks are the same as the links 8 between local S/D units except that they are typically longer in length.

In FIG. 8, a plurality of virtual channels are established between wide-area S/D units and local S/D units. Accordingly, a virtual channel is established, for example, between S/D unit 4-0 and S/D unit The operation of the FIG. 8 network is described in connection with FIG. 9 in which it is assumed that the cell transmission interval (CTI) is short. This short cell transition interval is evident since the transmitted cell at XR 1 3 results in a congestion signal, Cong_Signal, at a time XR,3+, only a short time after the cell is transmitted. Accordingly, the forward transmission of the cell at XR,3 originated (with reference to FIG. 5) at the source and is transmitted into the network along the virtual channel (FIG. 4) possibly as far as the node or the destination When congestion is encountered, a return cell is transmitted along the return path 8-R (FIG. 5) so as to arrive back at the source in a fraction of a Channel Cell Interval (CCI). In FIG.

25 9, for virtual channel 1, the CCI interval is the- interval between each XmitRate, time, for example, between XR 1 0 and XR1.

Accordingly, the reactive control operation in reducing the transmission rate in the FIG. 9 example is very fast.

In FIG. 9, the return transmission interval for a congestion signal returning from a congested node to the source is short and in particular is less than a single CCI as measured from the time that the original cell is transmitted from the source in the forward direction. The return transmission interval, therefore, includes both the forward path delay (from the source to the node having congestion) and the return path delay (from the node having congestion back to the source). Also included within the return transmission interval is the delay resulting from the return cell processing time required for converting forward cells to return cells and processing the return cells.

While the FIG. 9 example has a return transmission interval of less than one CCI, return transmission intervals of less than approximately 10 CCI are still considered short so that the issue rate unit can reactively modify the transmission rate and thereby reduce congestion.

In order to keep the return transmission interval short, the return cell processing time must also be kept short and is desireable much less than the return transmission interval.

In the FIG. 8 system, it is contemplated that the distance between the local network 1-0, for example, and the local network may be very far so that the delay time that may be encountered by a source 4-0 associated with network 1-0 in cell transmissions to a destination, for example, associated with a network may be much longer than a Channel Cell Interval. The longer the transmission time from a source to a destination and the return time from the destination back to the source, the less effective becomes the reactive control mechanism. If all communications in a very long network II I 26 are from the source at one end of the network to a destination at the far end of the network, then the effectiveness of reactive control for very long distance networks is reduced.

However, experience indicates that a large percentage (for example, 80%) of data traffic in a large network tends to be local and not the full long distance of the network. For example, if the traffic for the network 1-0 between the S/D units 4(0,S0-1) is principally local, then the reactive control mechanism is effective for controlling the congestion by controlling the transmission issue rate of the local sources for the network 1-0. Similarly, for each of the other local networks such as networks 1-1 and 1-2 of FIG. 8, the local traffic in each of those networks is controlled by *controlling the issue rate and the sources for local traffic in those networks. Since the entire network of FIG. 8 is composed of groups of local networks and each local network is effective to control congestion by controlling the local traffic issue rates, congestion is effectively controlled for the entire network of FIG. 8. The issue rates in sources are controlled o by variable rate units in each source.

In FIG. 8, each source (like source 4-S in FIG. 2) in the S/D units 4 typically includes a variable rate unit. A typical S variable rate unit (VRU) is described in greater detail in the above-identified application entitled METHOD AND APPARATUS FOR REACTIVE CONGESTION CONTROL IN AN ASYNCHRONOUS TRANSFER MODE (ATM) NETWORK. In the present specification, the variable rate unit is included within the segmentation unit 103 of FIG. 2 which is described hereinafter in further detail in connection with FIG. When congestion is present in the network of FIG. 8, a congestion signal (for example, on reverse path 8-OR of FIG. for a typical virtual channel) is received in a return cell by a reassembly unit (like reassembly unit 103' in FIG. 2 and FIG.

i I 27 11). The return cell has a return indicator for indicating congestion on the virtual channel. The reassembly unit detects the congestion and indicates the presence of that congestion in a return cell. The return cell causes a rate control signal to be output to rate control output 87' in a reassembly unit (see FIG. 11). The rate control output 87' inturn causes a rate control input to rate control input unit 87 of a corresponding segmentation unit (see FIG. 10). The rate control input (unit 87 in FIG. 10) provides the rate control input signal to a control unit in the segmentation unit control unit 116 in FIG The rate control signal includes a pointer for pointing to a virtual channel identifier, Cong_Signal_VCI, which identifies the virtual channel which has experienced congestion.

The variable rate unit as part of the segmentation unit 103 of FIG. 2 includes a packet memory 83 and a direct memory access (DMA) control 84. During normal operation, the DMA control 84 obtains cells from the packets in packet memory 83 and outputs those cells to the cell buffer 86 for transmission in the forward direction on the forward link from buffer 86 (the forward link corresponds to link 8-0F in FIG. The output rate on the forward link is the rate from the from the DMA controller 84 and that rate will normally be at the maximum cell rate for each virtual channel in the absence of a return cell indicating congestion received back on the return link (the return link corresponds to the link 8-0R in FIG. The variable rate unit also utilizes the control memory 81 and a control memory interface 82 of FIG. 2. Control memory 81 stores control information about each virtual channel that is established to send information from packet memory 83 to the output channel (for example, link 8-0F in FIG. The control information stored for each channel includes a congestion level value, Cong Level, indicating the level of the reduced rate of 28 transmission due to congestion on the virtual channel. The control information stored for each channel includes a recovery rate value, RCMode, indicating the rate of recovery of transmission rate after the rate has been reduced due to congestion on the virtual channel. The control information stored for each channel includes a counter field, Recovery_- Count, for counting cells during congestion periods and a control field, RC_Inhibit, for controlling changes in rate during congestion periods.

Upon receipt of a congestion signal, the control rate for the virtual channel is modified along with the transmission rate used for that particular channel. After the congestion signals no longer exist for a particular virtual channel, the rate of transmitting the information for the virtual channel which was previously experiencing congestion is modified.

The apparatus for and the methods of operation of source units having packet memories and DMA controllers are described in greater detail in the above-identified application entitled METHOD AND APPARATUS FOR REACTIVE CONGESTION CONTROL IN AN ASYNCHRONOUS TRANSFER MODE (ATM) NETWORK. The operation of such sources is explained with reference to a number of terms.

Terms For Automatic Rate Control Algorithm "Virt Chan" identifies one of a number of virtual channels, where each virtual channel transmits ATM cells at a selected one of a number of Xmit Rate rates unless, in response to an explicit congestion signal for that virtual channel, the transmission rate is modified by a congestion rate modification value, CongLevel.

"RateQueue" is a list of virtual channel identifiers that identify virtual channels that are to be transmitted at a particular rate.

"Xmit Rate" defines a number of transmission rates where 29 each Xmit_Rate is the rate to transmit cells used by all of the virtual channels in a Rate_Queue.

"Xcount" defines a number of 4-bit counts, one for each Rate_Queue, where each Xcount for the corresponding Rate_Queue is incremented to count the number of cell transmission opportunities at the full Xmit_Rate rate for that corresponding Rate_Queue.

"Xmit Time" defines a number of state bits, one for each RateQueue, where each XmitTime state bit indicates when it is time to service the corresponding Rate_Queue. Each Xmit Time state bit is set to the active state each time the Xcount is incremented for the corresponding Rate_Queue.

"Cong_Level" defines a number of 3-bit congestion level values, one for each virtual channel, where each value for the corresponding virtual channel is used to reduce the transmission rate of that corresponding virtual channel below the normal non-reduced Rate Queue rate.

"RC Mode" defines a number of 3-bit exponents, one for each virtual channel, where each exponent for the corresponding virtual channel is used to control the rate of restoring the transmission rate of that corresponding virtual channel back to the normal Rate Queue rate after the transmission rate of that virtual channel has been reduced because of congestion. The RC Mode rate of restoring is determined by a Recovery_Value 2M "RC Inhibit" defines a number of state bits, one for each virtual channel, where each RC_Inhibit state bit for the corresponding virtual channel is used to inhibit changes in Cong_Level during a period existing after a change in Cong_Level for that corresponding virtual channel and before a cell has been transmitted over that corresponding virtual channel.

"Recovery_Count" defines a number of 4-bit count values, one for each virtual channel, where each Recovery_Count value for the corresponding virtual channel is used for counting trans- 30 mitted cells for that corresponding virtual channel as measured from the time that the last Cong_Signal was detected for that virtual channel. Recovery_Count is compared with Recovery_Value for the corresponding virtual channel to control restoring the transmission rate of cells for that corresponding virtual channel. When Recovery_Count equals Recovery_Value for the corresponding virtual channel, Cong_Level is decremented to thereby increase the transmission rate for the corresponding virtual channel.

"Cong_Signal" indicates the presence of one or more Cong Signal_VCI identifiers stored in a queue for processing where the presence of Cong_Signal is used to trigger the decrementing of the transmission rate of cells for the virtual channels identified by the Cong Signal VCI identifiers in the queue.

"Cong_Signal_VCI" are virtual channel identifiers, one for each virtual channel on which a cell has been returned to indicate congestion.

"PktLoss Signal" indicates the presence of one or more Pkt_LossVCI identifiers stored in a queue for processing where the presence of Pkt_Loss_Signal is used to trigger the dec- Srementing of the transmisson rate of cells for the virtual channels identified by the PktLoss_VCI identifiers in the queue.

"Pkt Loss VCI" are virtual channel identifiers, one for each virtual channel on which the loss of cells has occurred and has been interpreted as an indication of congestion.

The variable rate unit within each source like source 4-S of FIG. 2 (explained in greater detail hereinafter in connection with FIG. 10) uses a counter which is clocked to provide timing. The clock signals establish different transmission rates, XmitRate, one for each Rate_Queue, for transmitting 31 cells over the virtual channels in that Rate_Queue. -A number of different rates can be established and in one embodiment, Q rates Xmit_Rate 0 Xmit_Rate,, Xmit_Rate(, are established from counter 88. Each one of the Q XmitRates is associated with a 4-bit Xcount which is incremented as a counter by the corresponding Xmit_Rate. Accordingly, the Xcounto, Xcount,, Xcount., counts are incremented by Xmit_Rateo, Xmit_Ratel, Xmit_Rateo.- rates, respectively.

Each of the Q Xmit_Rates is associated with a corresponding queue, Rate_Queue, for storing virtual channel identifiers that *identify virtual channels that have cells to be transmitted at the Xmit_Rate of the corresponding Rate_Queue. Accordingly, for uncontested normal operation, the virtual channels having identifiers stored in the RateQueueo, Rate_Queue, Rate_- Queue., queues are serviced for transmitting cells over virtual channels at the Xmit_Rateo, Xmit_Rate,, Xmit_Rate, 1 rates, respectively.

Each of the Q Xmit_Rates and the corresponding Q Rate_Queues is associated with a state bit, XmitTime, for indicating when it is time to service the Rate Queue. Accordingly, for uncontested normal operation, the Rate_Queueo, Rate_Queue,, Rate_Queue.

i queues are serviced whenever the corresponding Xmit_Time, XmitTimel, Xmit_Time i state bits are set, respectively. Each Xmit_Time state bit is set to the active state each time the Xcount is incremented for the corresponding RateQueue.

The variable rate unit is constructed with conventional logic circuits which implement an automatic rate control algorithm. Although the logical functions can also be performed by programed processors, such programmed processors are generally too slow for high speed operations required for ATM systems. The logic for the variable rate unit is included within the following TABLE 1-1 and TABLE 1-2.

32 TABLE 1-1 defines the parameter values for the automatic rate control algorithm. TABLE 1-2 defines the automatic rate control algorithm using pseudo code.

The operation of the variable rate unit is further described in connection with FIG. 9. The times when XmitTime, are active for Xmit_Rate, are shown in FIG. 9 as times XR,0, XR,1, XR 1 2, XR,3, and so on. For purposes of explanation, it was assumed that after transmitting cells by virtual channel 1 with Xmit_Celli at XR 1 O, XR 1 1, XR 1 2, and XR 1 3, an explicit congestion signal Cong_Signal, was returned at XR 1 3+ so that no cell was transmitted at XR 1 4 due to a decrease in the transmission rate.

The operation continues thereafter in FIG. 9 both to further decrease the transmission rate and to thereafter recover the rate when congestion stops after XR,9+.

FIG. 9 also depicts another virtual channel transmitting at another rate, Xmit_Rate 2 which is at one half the rate of Xmit_Ratei. One congestion signal, Cong_Signal 2 is received at XR,1+ causing the cell transmission of Xmit_Cell 2 to be at one half the Xmit_Rate 2 rate. For clarity, recovery has been assumed to be disabled (RC_Mode 101 Disable_Recovery) for this virtual channel.

In FIG. 9, the ATMOut line represents the sum of the i Xmit_Cell, and the Xmit_Cell 2 lines and is the output from the variable rate generator.

It will be apparent from FIG. 9 that the variable rate unit decreases the issue rate (for example, Xmit_Cell 1 rate) exponentially in response to a congestion signal. The PROCEDURE Rate Adjust is a means for decreasing the issue rate exponentially.

Similarly, the Xmit Rate Recovery Routine is a means for increasing the issue rate exponentially in response to the absence of a congestion signal. The rate of increasing the transmission rate, after it has been decreased in response to congestion, is less than the original rate used to decrease the 33 transmission rate in response to congestion. Making the increasing and decreasing rates different tends to avoid oscillation in the transmission rate and improve the reactive control.

The overall operation of an ATM network, using variable rate sources of the type described, is characterized using the following terms.

"Bit" is a binary unit of information.

"Byte" is a fixed number of bits, typically 8 bits.

"Packet" is a communication message formed of bytes, typically of variable length from 40 to 10,000 bytes.

."Cell" is a fixed length communication message formed of bytes, typically 53 bytes (5 byte header/48 byte data).

*99.

"Bit Rate" (BiR) is rate at which a communication system transmits bits.

"Byte Rate" (ByR) is rate at which a communication system 9999 transmits bytes.

"Cell rate" (CR) is rate at which a communication system transmits cells.

*999 "Cell Interval" (CI) is period from one cell to the next 9*99 cell.

"Channel Cell Interval" (CCI) is the cell interval for a virtual channel transmitting at the Channel Peak Cell Rate.

"Maximum Channel Peak Cell Rate" (MCPCR) is the cell rate assigned to a particular channel in the absence of congestion.

"Current Channel Peak Cell Rate" (CCPCR) is the current cell rate at a particular time for a virtual channel and may equal the Maximum Channel Peak Cell Rate in the absence of congestion and may be less than the Maximum Channel Peak Cell Rate when congestion is present.

"Communication Interval" (CI) is a time period in a network, typically measured as a number of cell intervals.

"Average Cell Rate" (ACR) is the average number of cells

I

34 transmitted over a communication interval in a virtual channel.

"Cell Transmission Interval" (CTI) is the average period of time for. a cell to travel from one location in a network to another location in the network.

"Modification Interval" (MI) is the period of time required to modify a communication parameter.

Segmentation Unit FIG, In FIG. 10, the segmentation unit 103, the memory unit 102 and the host 101 are typical for a source such as source 4-S in .o the communication system of FIG. 1 and FIG. 2.

In FIG. 10, the common memory unit 102 includes a control memory 81 and a packet memory 83. The segmentation unit 103 includes a control memory interface (CM IF) 82, a packet memory interface (PM IF) 84, a host interface 127, and a cell interface 126. The cell interface includes an output cell buffer 86 and .a rate control input 87. The segmentation unit 103 includes a core processor 128 which includes a logic control unit 117, a 6 register array 114, and a counter 88. The counter 83 is stepped .by an external clock 89. The control memory interface 82 connects the core processor 128 with the control memory 81. The control memory interface includes an address generator 113, an address register 110, a control unit 111 and a data register 112. The address generator 113 provides addresses to the address register 110 under control of information from the register array 114. The address from address register 110 addresses the control memory 81 to provide data into the data register 112 from memory 81 or to receive data from the data register 112 for storage in control memory 81. The control 111 connects with the logic control unit 117 for providing control signals to and from the control memory 81. The control unit Ill provides conventional read and write signals to the control memory 81 under control of the logic control unit 117.

35 In FIG. 10, the packet memory interface includes an address generator 123 connected to receive address datz from the register array 114 for providing addresses into the address register 120. Address register 120 provides addresses for addressing the packet memory 83 for reading or writing information into or from the packet memory 83 under control of the control unit 121. The control unit 121 provides conventional read and write signals to the packet memory 83 under control of the logic control unit 117. The data register 122 provides data to the input cell buffer 86 for being processed by the Scyclically redundancy code unit (CRC) 115. The data for the output cell is stored in the data register 122 from the packet memory 83 or from the constant bit rate (CBR) input 134.

The operation of the FIG. 10 source is to provide packetized r o cells or CBR information to the ATM link as an output to the output cell buffer 86. The header information for each cell in buffer 86 is received from the data register 112. The logic control unit 117 controls the logical operations necessary to segment the packets in the packet memory 83 into cells for .2 output to buffer 86 or alternatively to output information from the CBR input. The details of the logic control unit 117 in the core processor 128 are shown in the following tables TABLE 1-1 and TABLE 1-2. TABLE 1-2 is a representation of pseudo code that defines, in a conventional manner, circuits in semiconductor chip technology. The pseudo code can be directly transformed into VHDL circuit specifications standardized by the

IEEE.

I

36 TABLE 1-1 Segmentation Processor Terms COPYRIGHT 1992 Adapative Corporation Preset operation mode (for entire network) Congestion_Indicator 0 ReturnCell Congestion indicated by a return cell 1 Packet_Loss Congestion indicated by packet loss Preset static control variable (for each virtual channel) RCMode 000 Recovery_Value 20 1 001 Recovery_Value 21 2 010 Recovery_Value 22 4 011 Recovery_Value 2 3 8 100 Recovery_Value 24 16 101 DisableRecovery 110 Disable RateControl 111 VirtualChannel Shutdown TIQ 6-bit value Time Interval Quota; CQ 6-bit value Cell Quota; State variables (in the chip) Current_Descriptor 13-bit value The descriptor number that acts as a pointer to the descriptor table.

Previous Descriptor 13-bit value Descriptor previous to the Current Descriptor in the rate queue.

This has no meaning if Current_Descriptor StartDescNum.

State variables (for each rate queue) Start Desc Num 13-bit value Starting descriptor number of the rate queue.

0 no descriptors are queued on the rate queue.

37 TABLE 1-1 Segmentation Processor Terms COPYRIGHT 1992 Adapative Corporation State variables (for each virtual channel) Cong_Level: 000 001 010 011 100 101 normal half quarter eighth sixteenth zero Full Rate_Queue Xmit_Rate 1/2 Rate_Queue Xmit_Rate 1/4 Rate_Queue Xmit_Rate 1/8 Rate_Queue Xmit_Rate 1/16 Rate_Queue X :t_Rate no cell transmis..on

S.

ease 00 so S *5

S

RC Inhibit: 0 Inhibit Inactive 1 Inhibit Active Recovery_Count: 4-bit count TI Ctr 6-bit count Time Interval Counter; CellCtr 11-bit count Cell Counter; Last Desc Num 13-bit value Last descriptor in the VirtChan chain.

0 no descriptors are queued on this Virt Chan.

State variables (for each Descriptor) VirtChan 16-bit value NextDescin_RQ 13-bit value Next Desc in VC 13-bit value Virtual Channel the descriptor is associated with.

Next descriptor in the Rate Queue chain.

0 This is the last descriptor on the rate queue.

Next descriptor in the Virt Chan chain.

38 TABLE 1-1 Segmentation Processor Terms COPYRIGHT 1992 Adapative Corporation 0 This is the last descriptor on the VirtChan chain.

PacketAddress 24-bit value Address pointer to the Buffer from which the packet is being segmented.

e ooeo oO.

ooeo Packet_Length 16-bit value Length of the packet in number of cells ResidualCRC 32-bit value Contains the running CRC of the packet Incoming Constant Bit Rate (CBR) traffic interface signal CBRXmit Signal: 0 No_Cell_Waiting 1 CBR_cell_ready for transmission State variables (for CBR traffic) CBRChan 16-bit value Virtual Channel the CBR cell is associated with.

Incoming congestion signals Cong_Signal: 0 No_Cell_Waiting 1 Cell_Waiting No return cell waiting Return cell waiting for processing Cong_SignalVCI: identifier Virtual channel identifier of return cell, typically 10 to 16 bits Pkt_Loss_Signal: 0 No_Signal_Waiting 1 Signal_Waiting No Pkt_Loss_Signal waiting Packet loss signal waiting for processing Pkt Loss VCI: 39 TABLE 1-1 Segmentation Processor Terms COPYRIGHT 1992 Adapative Corporation identifier Virtual channel identifier of Pkt LossSignal, 10 to 16 bits typical*/ Signals from timers (for each Rate_Queue) Xmit Time: i Active 0 Inactive X count: 4-bit count Time to service this Rate_Queue Not time to service this Rate_Queue Count from X-counter 000 0* 0 00 Communication Queues (In Memory) PacketReady_Queue: Circular Queue in memory (of descriptor numbers) loaded by the host to enqueue descriptors for segmentation.

Transfer Comp_Queue: Circular Queue in memory (of descriptor numbers) loaded to enqueue descriptors after the packets have been segmented.

0000

I

40 TABLE 1-2 Segmentation Processor Logic COPYRIGHT 1992 Adapative Corporation Scheduler

REPEAT

FOREACH Rate_Queue IF (Xmit Time [Rate Queue] Active) THEN ServiceRate Queue (Rate_Queue)

ENDIF;

IF (CBR Xmit_Signal Active) CBR Transmission*/ THEN Xmit Cell (CBR Chan)

ENDIF;

Check_Cong_Signal; Link_New Descriptor;

-ENDFOREACH;

-ENDREPEAT;

Check for congestion signal waiting for processing 7/ -PROCEDURE Check_Cong_Signal; -IF (CongestionIndicator Return_Cell)

-THEN

IF (Cong_Signal Cell Waiting) THEN RateAdjust (CongSignal_VCI);

ENDIF;

-ELSE

IF (PktLoss_Signal Signal Waiting) THEN RateAdjust (PktLossVCI);

ENDIF;

ENDIF;

L-.ENDPROC;

Modify identified virtual channel rate upon receiving cong signal -PROCEDURE RateAdjust (Virt_Chan); READ for Virt Chan (RC_Mode, RC_Inhibit, Recovery_Count, Cong_Level); IF (RCMode !=Disable_RateControl) AND (RCInhibit !=Inhibit Active)

-THEN

IF (Cong_Level zero) THEN increment Cong_Level;

ENDIF;

RC Inhibit InhibitActive; Recovery Count 0;

-ENDIF;

WRITE for Virt Chan (RCInhibit, Recovery Count, Cong_L vel);

ENDPROC;

II i, 41 TABLE 1-2 Segmentation Processor Logic COPYRIGHT 1992 Adapative Cor.poration Service each virtual channel in rate queue PROCEDURE Service_RateQueue (RateQueue); Current_Descriptor Start_DescNum Previous_Descriptor 0 -WHILE (Current Desc Num 0 READ for Current_Descriptor (Virt_Chan, Step RCD Next_Descin RQ Next_Desc_in_VC, Packet Length, PacketAddress, Residual_CRC); READ for Virt_Chan (RC_Mode, RC_Inhibit,/* Step RVc Recovery Count, CongLevel, TIQ, CQ, TI_Ctr, Cell_Ctr, Last DescNum); Average Metering Routine -IF (TI_Ctr 0)

THEN

TICtr TIQ

ELSE

TICtr max( (TI_Ctr 0); END IF; IF (TICtr 0)

THEN

Cell Ctr max (Cell_Ctr CQ); END IF; /*Step Calc Xmit Cell Routine s -IF (RCMode VC Shutdown)

-THEN

IF (Cong_Level normal) OR (Cong_Level half AND Xcount[Rate_Queue] OR (Cong Level quarter AND Xcount[Rate_Queue] OR (Cong Level eighth AND Xcount[Rate_Queue] OR (Cong Level 1/16 AND Xcount[Rate_Queue] OR (CongLevel zero AND Xcount[Rate_Queue]

-THEN

IF ((Cong_Level zero) AND (CellCtr 0))

-THEN

Cell Ctr max ((Cell Ctr XmitCell (VirtChan); X Packet_Length Packet_Length 1; St W-tITE for CurrentDescriptor (Packet_Leng ??ll) ?111) 1111) 1111) mit next cell ep WCD th; 1 42 TABLE 1-2 Segmentation Processor Logic COPYRIGHT 1992 Adapative Corporation PacketAddress, Residual_CRC); IF (PacketLength 0)

THEN

Delink Descriptor (Virt_Chan, Current_Descriptor, NextDescin_RQ, NextDesc inVC, Last_DescNum, Previous Descriptor);

ENDIF;

ENDIF;

RC_Inhibit InhibitInactive; Xmit Rate Recovery Routine IF (RC_Mode Disable_RateControl) AND (RC_Mode Disable_Recovery)

-THEN

IF (Recovery_Count Recovery_Value)

THEN

IF (Cong_Level normal) THEN decrement Cong_Level;

ENDIF;

Recovery Count 0;

ELSE

Increment Recovery_Count;

ENDIF;

-ENDIF;

Step WVc WRITE for Virt_Chan (RCInhibit, Recovery_Count, Cong_Level, TI Ctr, CellCtr);

-ENDIF;

ENDIF;

Previous_Descriptor=Current_Descriptor; CurrentDescriptor NextDesc inRQ IF (CBR Xmitc_Signal Active) CBR Transmission THEN Xmit Cell (CBR Chan)

ENDIF;

Check Cong_Signal; Link_New_Descriptor;

ENDWHILE;

ENDPROC;

I I II I 43 TABLE 1-2 Segmentation Processor Logic O COPYRIGHT 1992 Adapative Corporation Transmission of a cell PROCEDURE Xmit Cell WHILE (XmicCellHeader NotDone) Wait END WHILE; Read for Virt_Chan (Cell_Header); Load Buffer from Packet Memory; WHILE ((Load Buffer from Packet_Memory (Read for Virt_Chan Not Done)) Wait END WHILE; CellAvailablefor_Transmission;

ENDPROC;

Step XCH Step RVH Step LBPMem Not Done) or

S

Link new descriptor for transmission PROCEDURE Link_NewDescriptor; IF (Packet_Ready Queue Not_Empty)

-THEN

Read for Packet_Ready_Queue (Current_Descriptor) Read for Current_Descriptor (Virt_Chan); Read for VirtChan(Last Desc Num); -IF (Last Desc Num 0) Link at head of Vertical chain

-THEN

Next Desc_in RQ Start Desc Num Next Desc-in-VC 0; Start Desc Num Current Descriptor; Last_Desc_Num Current_Descriptor; Write for VirtChan (Last Desc_Num); Write for Current_Descriptor(Next_Descin_RQ,Nex _Desc in VC); -ELSE Link at tail of the Horizontal chain Next_Desc in VC CurrentDescriptor Write for LastDescNum (Next Desc in VC); Next_DescinVC 0; Last_Desc_Num Current_Descriptor; Write for Virt_Chan (Last Desc_Num); Write for Current_Descriptor (Next Desc in VC);

-ENDIF;

ENDIF;

-ENDPROC;

I I I

Y

44 TABLE 1-2 Segmentation Processor Logic COPYRIGHT 1992 Adapative Corporation De-Linking of a descriptor after transmission has been completed PROCEDURE Delink_Descriptor (Virt_Chan, Current_Descriptor, Next_Desc_in_RQ, NextDescinVC, LastDesc_Num, Previous Descriptor); -IF (Current_Descriptor StartDescNum) Descriptor is at the head of the Rate Chain

THEN

IF (Next_Desc in VC 0) *Last descriptor horizontal chain

THEN

Write for Transfer_Comp_Queue (Current_Descriptor); Start Desc Num NextDesc_in_RQ; Last_Desc_Num 0; Write for Virt_Chan (Last_Desc_Num); ELSE Link next descriptor in horizontal chain Write for Transfer_Comp_Queue (Current_Descriptor); Start_Desc Num Next_Desc_in_VC; Write for Current_Descriptor (Next_DescinRQ);

ENDIF;

ELSE

IF (Next_Desc in_ 0) Last descriptor in horizontal chain

-THEN

Write for Transfer_Comp_Queue (Current_Descriptor); Last_Desc_Num 0; Write for Virt_Chan (Last_Desc_Num); Write for Previous_Descriptor (Next_Descin RQ); -ELSE Link next descriptor in horizontal chain Write for TransferComp_Quueu (Current_Descriptor); Current_Descriptor PreviousDescriptor; Write for Current_Descriptor (Next_Desc in VC);

-ENDIF

LENDIF;

-ENDPROC;

Emma 45 Reassembly Unit FIG. 11 In FIG. 11, the reassembly unit 1 the memory unit 102' and the host 101' are typical for a destination such as destination 4-D in the communication system of FIG. 2.

In FIG. 11, the common memory unit 102' includes a control memory 81' and a packet memory 83'. The reassembly unit 103' includes a control memory interface (CM IF) 82', a packet memory interface (PM IF) 84', a host interface 127', and a cell interface 126'. The cell interface includes an input cell buffer 86' and a rate control output 87'. The reassembly unit 103' includes a core processor 128' which includes a logic control unit 117, a register array 114', and a counter 88'. The counter 88' is stepped by an external clock 89'. The control memory interface 82' connects the core processor 128' with the S control memory 81'. The control memory interface includes an address generator 113', an address register 110', a control unit 111' and a data register 112'. The address generator 113' provides addresses to the address register 110' under control of information from the register array 114'. The address from address register 110' addresses the control memory 81' to provide data into the data register 112' from memory 81' or to receive data from the data register 112' for storage in control memory 81'. The control 111' connects with the logic control unit 117 for providing control signals to and from the control memory 81'. The control unit 111' provides conventional read and write signals to the control memory 81' under control of the logic control unit 117.

In FIG, 11, the packet memory interface includes an address generator 123' connected to receive address data from the register array 114' for providing addresses into the address register 120'. Address register 120' provides addresses for addressing the packet memory 83' for reading or writing information into or from the packet memory 83' under control of c 46 the control unit 121'. The control unit 121' provides conventional read and write signals to the packet memory 83' under control of the logic control unit 117. The data register 122' receives data from the input cell buffer 86' after being processed by the cyclically redundancy code unit (CRC) 118.

The data from the input cell is stored in the data register 122' for input to the packet memory 83' or for output to the constant bit rate (CBR) output 134'.

The operation of the FIG. 11 destination is to receive packetized cells or CBR information from the ATM link as an input to the input cell buffer 86'. The header information from each cell in buffer 86' is input to the logic control 117. The logic control unit 117 controls the logical operations necessary to reassemble the cells from buffer 86' into packets in the packet memory 83' or alternatively to output the cell informa- S tion on the CBR output. The details of the logic control unit 117 in the core processor 128' are shown in the following tables TABLE 2-1 and TABLE 2-2. TABLE 2-2 is a representation of pseudo code that defines, in a conventional manner, circuits in semiconductor chip technology. The pseudo code can be directly transformed into VHDL circuit specifications standardized by the IEEE.

47 TABLE 2-1 Reassembly Processor Terms Reassembly algorithm for pipeline architecture Preset Control variables Cell_Type One of the four types of the ATM cell.

BOM Beginning Of Message COM Continuation Of Message EOM End Of Message SSM Single Segment Message Initial CRC Value 32-bit value Value the CRC checker is initialized to (All VirtChan Virtual Circuit Index extracted from the cell header State variables (in the chip) VC Sel Virtual Channel range select.

Packet_Aging_Timer Timer used to determine the time interval for checking the state of packet in reassembly.

Age_VCI Index of the Virtual Circuit that is checked for aging packets.

State variables (for each cell) •HeaderVCI Virtual Circuit Index contained in the Header of the ATM cell.

BASize Buffer Allocation size: This is carried in the BOM/SSM S"cells can be used to obtain the correct size buffer.

State variables (for each Virt_Chan) Status Inactive no packet is being reassembled on this VirtChan Active packet is being reassembled on this Virt Chan BufferDescriptor 13-bit value The descriptor number associated with the buffer that the packet is being reassembled in.

PacketAge_Limit 48 TABLE 2-1 Reassembly Processor Terms The initialization value for PacketTimeout counter that is used to age packets.

State variables (for each Buffer_Descriptor) Residual CRC 32-bit value Contains the running CRC of the packet Packet Address 24-bit value Address pointer to the Buffer in which the packet is being reassembled in.

Packet_Length 16-bit value Length of the packet received in bytes.

Buffer Capacity 16-bit value: Number of free locations left in the buffer in bytes.

,Packet Timeout 8-bit value Indicates the time interval that the packet has been active (during reassembly).

Signals from the Input FIFO Cells In FIFO 1 Available At least one cell is available in the FIFO 0 Empty The FIFO is empty or has less than one cell Outgoing congestion signal Cong Signal VCI: identifier Virtual channel identifier of return cell *o *k 49 TABLE 2-2 Reassembly Processor Logic COPYRIGHT 1992 Adapative Corporation Scheduler

REPEAT

-IF (CellsInFIFO Available)

THEN

Receive Cell ELSEIF (Packet_Aging Timer Overflow)

THEN

Age_Packet;

ENDIF;

-ENDREPEAT

t.

p.

.r 66 6 .16 P -PROCEDURE Receive_Cell ReadCellHeader(n) LWHILE (ReadCellHeader(n) NotDone) Wait END WHILE; I F (Cell_Header=Return_Cell) THEN Cong_Signal_VCI HeaderVCI Exit

ENDIF

Mapping function for Step RCH the VC -IF (VC_Sel 000) THEN Virt Chan ELSEIF (VC_Sel THEN Virt Chan ELSEIF (VC_Sel THEN Virt Chan ELSEIF (VCSel THEN Virt Chan ELSEIF (VC Sel THEN Virt Chan ELSEIF (VC Sel THEN Virt Chan ELSEIF (VCSel THEN Virt Chan

ELSE

THEN Virt Chan

-ENDIF;

Header VCI 001) Header VCI 010) Header VCI 011) Header VCI 100) Header VCI 101) Header VCI 110) mod 2**16 mod 2**15 mod 2**14 mod 2*13 mod 2**12 mod 2**11 '96.

6o op 6 6 Header VCI mod 2**10 Header 'CI mod 2**9 Read for Virt_Chan (Status, Buffer_Descriptor, Packet_Age_Limit); WHILE (Read for VirtChan NotDone) Wait END WHILE; F IF ((Status Active) AND ((Cell_Type COM) OR (Cell_Type EOM))) Step RVc 50 TABLE 2-2 Reassembly Processor Logic COPYRIGHT 1992 Adapative Corporation

THEN

Read for Buffer Descriptor (Residual_CRC); RDcrc ELSEIF ((Status Active) AND ((Cell_Type BOM)OR(Cell_Type SSM))) Write back the descriptor and get new descriptor Residual_CRC Initial_CRC_Value; Write for Packet_Complete_Queue (Buffer_Descriptor) -IF (BASize Sml_Buf_Siz)

-THEN

Read for Small Free_Descriptor Queue (BufferDescriptor)

ELSE

Read for Large Free Descriptor_Queue (Buffer Descriptor)

ENDIF;

-ELSEIF ((Status Inactive) AND S((Cell_Type BOM) OR (Cell_Type SSM))) Get new descriptor ResidualCRC Initial CRCValue; -IF (BASize Sml BufSiz)

-THEN

Read for Small Free Descriptor Queue (BufferDescriptor)

-ELSE

Read for Large Free_Descriptor_Queue (Buffer Descriptor)

ENDIF;

-ELSE

Exit; Error cell E ENDIF; Wt WHILE (Read for Buffer Descriptor Not_Done) Wait END WHILE; Read_Cell_Payload(n) Step RCPyld Read for Buffer Descriptor (Packet_Address, Packet_Length, •Buffer_Capacity, PacketTimeout); RDesc SWHILE ((ReadCell_Payload(n) Not Done) or (Read for Buffer Descriptor NotDone)) Wait END WHILE; WHILE (Payload_Transfer(n-l) Not_Done) Wait

ENDWHILE;

Transfer_Cell_Payload(n); Transfer Payload to PM IF (Cell_Type BOM)

THEN

PacketTimeout Packet_Age_Limit 1

L,

51 TABLE 2-2 Reassembly Processor Logic COPYRIGHT 1992 Adapative Corporation L ENDIF; Write for Buffer_Descriptor (Packet_Address, Packet_Length, Buffer_Capacity, ResidualCRC, Packet_Timeout); WDesc r-IF ((Cell Type EOM) OR (Cell_Type SSM))

THEN

Write for Packet Complete_Queue (Buffer Descriptor) Status Inactive;

ELSE

Status Active;

ENDIF;

Write for Virt_Chan (Status, Buffer_Descriptor); WVc

-ENDPROC;

52 TABLE 2-2 Reassembly Processor Logic O COPYRIGHT 1992 Adapative Corporation PROCEDURE Age_Packet AgeVCI Age_VCI 1; Mapping function for the VC IF (VC_Sel 000) THEN Virt_Chan Age_VCI mod 2**16 ELSEIF (VC Sel 001) THEN Virt_Chan Age_VCI mod 2*15 ELSEIF (VC Sel 010) THEN Virt_Chan Age_VCI mod 2**14 ELSEIF (VCSel 011) THEN Virt_Chan Age_VCI mod 2**13 ELSEIF (VCSel 100) THEN Virt_Chan Age_VCI mod 2**12 ELSEIF (VC Sel 101) STHEN Virt_Chan Age_VCI mod 2**11 ELSEIF (VCSel 110) THEN VirtChan Age_VCI mod 2**10

ELSE

THEN Virt_Chan AgeVCI mod 2**9

L-ENDIF;

Read for Virt_Chan (Status, Buffer_Descriptor); WHILE (Read for Virt Chan Not Done) Wait END WHILE; -IF (Status Active) -THEN Read for Buffer Descriptor (PacketTimeout); V" rWHILE (Read for Buffer Descriptor Not Done) Wait END WHILE; PacketTimeout Packet_Timeout 1; Write for Buffer Descriptor (PacketTimeout); IF (Packet Timeout Overflow)

THEN

Status Inactive; Write for VirtChan (Status, BufferDescriptor); Write for Packet Complete_Queue(BufferDescriptor)

-ENDIF;

-ELSE

Exit;

-ENDIF;

-ENDPROC;

53 Operation Of Core Processing Unit The core processor 128 of FIG. 10 as defined by TABLE 1-2 is a processing unit for controllirng the processing of cells for each of the one or more virtual channels.

The core processing unit 128 is defined by TABLE 1-2. TABLE 1-2 includes a variable issue rate unit that is described in detail in the above-identified application entitled METHOD AND APPARATUS FOR REACTIVE CONGESTION CONTROL IN AN ASYNCHRONOUS TRANSFER MODE (ATM) NETWORK. In addition to variable issue rate unit, additional functions and units are added in the present specification which relate to two-dementional queueing, pipelining, packet aging and constant bit rate transmission.

Overall, the core processing unit 128 is under control of the scheduler circuits defined by lines 1 through 12 in TABLE 1-2. The basic timing of the core processing unit 128, as it appears in the segmentation unit of FIG. 10 is controlled by the Xmit_Time[Rate_Queue] of line 3 of TABLE 1-2. If the transmit time is active for the particular Rate_Queue (for example, the rate queue of FIG. 12), then the Service_Rate_Queue procedure S" at line 46 of TABLE 1-2 is entered. The service rate queue procedure services each virtual channel on the rate queue and carries out a number of routines including the average metering routine in lines 61 through 71, the transmit (Xmit) cell routine in lines 74 through 100 and lines 124 through 126, the transmit rate recovery routine in lines 102 through 120. After the S' procedure ends, a return is made to the scheduler where in line 6 a check is made to see whether or not a CBR transmission signal is active and if so, then a CBR cell is transmitted on a CBR channel.

The procedure for transmitting the cell appears in lines 134 through 145. If thereafter a return is made where in line 9 of the scheduler a check is made for a congestion signal which is the procedure from lines 16 through 27. Thereafter, a return L L 54 is made to line 10 of the scheduler where a Link_New_Descriptor procedure is called which appears at lines 148 through 171.

The core processor 128 of TABLE 1-2 is a processing unit for controlling the processing of cells for each of the one or more virtual channels. The core processor includes control logic for controlling the processing of control information and data logic for controlling the processing of data. The control logic and the data logic operate to concurrently process data and control information for sustained transmission of cells in the ATM network up to the cell clock rate for each of the one or more virtual channels.

Referring to TABLE 1-2, the data logic includes part of the Xmit_Cell procedure starting at line 134. The data logic makes the cell available at the cell interface for transmission on the ATM link at line 144. The cell trasnmitted includes the cell header read at line 138 using the control logic and the control o memory interface. The cell trasnmitted includes the cell payload read at line 139 using the data logic and the packet memory interface.

Two Dimentional Oueues FIG. 12 In FIG. 12, an example of the rate queue and the channel queue structures that are created in the control memory of FIG.

10 are shown. The rate queue which appears in the vertical direction in FIG. 12 includes a descriptor for each packet that is to be transfered over the ATM network with the same rate.

The channel queue which appears in the horizontal direction in FIG. 12 includes a descriptor for each packet for the same virtual channel that is to be transfered over the ATM network with the same rate.

Referring to FIG. 12-1 each of the rectangles represensts a descriptor, X, and includes two pointers, a vertical pointer, v, and a horizontal pointer, h. Additionally, each descriptor

.W

55 includes other control information asscociated with a packet including for example, a length field, 1, indicating the number of bits (or cells) in the packet. In the packet descriptor, X indicates the identity of the particular packet being processed by the queue of FIG. 10. For each descriptcr, X, an associated rate queue pointer, v, identifies the next-in-order descriptor in the rate queue (vertical chain) that forms a linked list of packets having the same transmission rate. In FIG. 12-1, Descriptor6 (X 6) is at the tail of the rate queue since the next-in-order vertical packet is DescriptorO, that is v=0 for Descriptor6. Descriptorl is at the head of the linked list.

Descriptorl has a vertical pointer 3 pointing to the next-inorder packet 3. Dascriptor3 has a pointer 4 (v=4j which points to Descriptor4. Descripror4 has a pointer 6 which points to the tail of the queue Descriptor6. The rate queue in the vertical direction includes Descriptori, Descriptor3, Descriptor4 and Descriptor6.

In addition to the rate queue, one or more channel queues are shown in the horizontal direction. Descriptorl has a horizontal pointer 2 which points to Descriptor2. Descriptor2 has a horizontal pointer, h=0, and a vertical pointer that is blank. Accordingly, Descriptor2 is at the tail of the horizontal channel queue which includes Descriptorl and Descriptor2 and which represent one virtual channel. In a similar manner, is at the tail of the channel queue that includes Descripcor4 and Descriptor5. Similarly, Descr:iptor7 is at the tail of the channel queue that includes Descriptor6 and Descriptor7.

In FIG. 12-2, the queue structure of FIG. 12-1 is modified by the addition of an additional descriptor, Dercripzor6, co tnr channel queue including he Des ripror4 and Descriptcr5. lote that the addition of Descr±ptor8 causes the h pointer in to be changed fromi a 0 as it appeared in FIG. 12-1 I -1 56 to an 8 as it appears in FIG. 12-2. The 8 is a pointer to the Descriptor8 added to the channel queue of Descriptor4 and In FIG. 12-3, Descriptor3 is removed from the rate queue.

Removal of Descriptor3 is accompanied by changing the v pointer in Descriptorl from a 3 as it appeared in FIG. 12-2 to a 4 as it appears in FIG. 12-3 thereby pointing to the next-in-order Descriptor4.

In FIG. 12-4, the Descriptor4 of FIG. 12-3 is removed. This removal is accompanied by changing the pointer in Descriptorl as it appeared in FIG. 12-3 from a 4 to a 5 as it appears in FIG. 12-4. Similarly, Descriptor5 has its vertical pointer changed from a blank to a 6 to thereby point to Descriptor6 which is at the tail of the rate (vertical) queue.

FIG. 12 is an example of one of possibly many rate queues concurrently processed by core processing unit 128 of FIG. The core processing unit 128 of FIG. 10 adds and removes packet descriptors from the queues. In operation, the core processing unit 128 is controlled by the scheduler circuits defined by lines 1 through 12 in TABLE 1-2.

Referring to TABLE 1-2, the linking of a new descriptor for either the vertical rate queue or the horizontal channel queue is controlled by the Link_New_Descriptor procedure starting at line 148 in TABLE 1-2. The insertion at the head of the S vertical rate queue commences at line 154 and the insertion at the tail of the horizontal channel queue commences at line 162.

In TABLE 1-2, the delinking of a descriptor from either the vertical rate queue or the horizontal channel queue is controlled by the Delink_Descriptor procedure starting at line 174 in TABLE 1-2. The removal to collaspe the vertical rate queue commences at line 182 or line 195 and the removal to shrink the horizontal channel queue commences at line 187 or line 200.

57 Segmentation Unit Pipelined Operation FIG. 13 Referring to FIG. 13, an example of the pipelined operation of the segmentation core processor 128 of FIG. 10 is represented. The processing for the cell interface, the control memory interface, the host interface and the packet memory interface, together with internal calculations for the processor are shown for a current cell N, for a prior cell N-i and for a next cell N+1.

In FIG. 13, the straight lines opposite the cell interface, the control memory interface, and the packet memory interface represent processing that occurs by those interfaces, respectively. In FIG. 13, the curved lines between straight lines of the cell interface, the control memory interface, and the packet memory interface represent timing dependences between those interfaces. A proceess ending at the beginning of a curved arrow along one interface must complete before the process beginning at the end of the curved arrow at another interface.

It is apparent from FIG. 13 that the cell processing for cell N-i overlaps the cell processing for cell N and similarly that the processing for cell N overlaps the processing for cell By this pipelined overlapping of the processing for different cells, the cell rate of transmission is maximized.

In FIG. 13, the prefix W generally refers to a write operation, the prefix R generally refers to a read operation and the prefix X generally refers to a transmit operation. The legends in FIG. 13 are correlated to the comment fields in TABLE 1-2 to show the correlation between the TABLE 1-2 circuit specification logic and the pipelined operation thereof as indicated in FIG. 13. Through pipelining the processing of information or cells for operations at each of the cell interfaces, the core processing unit 128 in the source of FIG.

is able to to concurrently process data and control information for sustained transmission of cells in the ATM netwurk up ml- 58 to the cell clock rate for each of the one or more -virtual channels.

Referring to FIG. 13 'ach cell transmitted has a header portion and a payload portion. For the cell N, the header portion is determined at the control memory interface by the read virtual header (RVH) operation of cell N. Concurrently with the reading of the header for cell N by the control memory interface, the payload information for cell N is read by the packet memory interface during the load LBPmem operation for cell N. While the header and payload for cell N are being read by the control memory interface and the packet memory interface, during the RVH and LBPmen operations for cell N, respectively, the data transmission for the N-i cell payload is being concurrently processed by the cell interface during the XCPyld operation for cell N-l.

After these operations are complete for cell N, the header for cell N is transmitted during the XCH operation which is then followed by the payload transmitted in the XCPyld operation for cell N. At the same time that the header and payload are transmitted by the cell interface, the control memory interface is performing concurrently the WCD, WVc, RCD, and RVC operations. In this manner it can be seen that the pipelined Soperations over the various interfaces including the cell interface the control memory interface and the packet memory interface are concurrently operating.

Reassembly Unit Pipelined Operation FIG. 14 Referring to FIG. 14, the pipelined processing of the core processor 128' in the reassembly unit 103' of FIG. 11 is shown.

The processing in the the core processor 128' is pipelined in that the processing of cell N-i is concurrent with the processing for cell N and the processing for cell N is concurrent with the processing for cell N+1. The legends in FIG. 14 are

I

I 59 correlated to the comment fields in TABLE 2-2 to show the correlation between the TABLE 2-2 circuit specification logic and the pipelined operation thereof as indicated in FIG. 14.

Through pipelining the prc;essing of information for cells for operations at each of the cell interfaces, the core processing unit 128' in the destination of FIG. 11 is able to to concurrently process data and control information for sustained reception of cells in the ATM network up to the cell clock rate for each of the one or more virtual channels.

Combined Segmentation And Reassembly Operation FIG. Referring to FIG. 15, a schematic representation of the segmentation operation is a source and the corresponding operation in a destination is shown for a typical packet. Each packet in FIG 15 like the Descriptorl packet 141 includes a plurality of cells shown as In a S similar manner, the additional Descriptor2 and Descriptor3 packets 142 and 143 in FIG. 15 have Descriptor2 cells and Descriptor3 cells. The Descriptor2 cells are C(2,2), Similarly, the Descriptor3 cells are C(3,1), The initial cells on the ATM link occur, for example, as as and so forth up to These initial cells are then subsequently followed by cells and so forth up to In this way, outgoing cells from different packets are interleaved on the ATM link. The cells from each packet are received at the reassembly processor in the destination in the same order that they are sent by the segmentation processor in the source. The interleaved cells in the reassembly operation are reassembled into the packets so that the cells in the packet 140' at the destination are in the same order after the reassembly namely C(1,n).

-u 60 Average Rate Metering FIGs. 16-20 Referring to FIG. 16, the cell clock has cell periods from 0 through 31 which establish the cell clock rate.

In FIG. 17, a request on a particular channel to send a cell occurs as an Xmit_Cell request in TABLE 1-2. At time 0 of the cell clock in FIG. 16, two requests to transmit cells were pending. With the average rate metering routine in TABLE 1-2, on average only one cell can be transmitted for each timeout of the TI counter. Referring to FIG. 18, the TI counter counts down from the maximum count (TIQ) to the minimum count (0) during the 0 to 4 clocks and resets on clock 5 to the maximum count (4 in the particular example shown). The TI counting operation is shown in the average metering routine on lines 61- 71 of TABLE 1-2.

In FIG. 19, the cell counter is an accumulator that accumulates the number of opportunities that are available to transmit cells. The cell counter is incremented each time the TI counter is counted down to 0. The cell counter is decremented each time a cell transmission occurs. A data transmission is shown in FIG. 20. Referring to FIG. 19, at time 4, the cell counter is incremented at the time that the TI counter is decremented to 0. Cell counter time 4 indicates that one cell can be transmitted in accordance with the average metering routine. Accordingly, because a data request is pending as shown by FIG. 17 data is actually transmitted at time 5 in FIG.

thereby causing the cell counter to be decremented at T5 and reducing the data request to transmit to one as shown in FIG.

17. Similarly, the cell counter of FIG. 19 is incremented again at time 9 when the TI counter times out at time 9 as shown in FIG. 18. Since a request is pending as indicated in FIG. 17 a data cell is transmitted at time 10 as shown in FIG. 20 and the cell counter is decremented back to 0. At that time as shown in FIG. 17, no more data requests to transmit are pending and I I I 61 at time 14 when the TI counter times out, the cell counter is again incremented. Again at time 19 when the TI counter decrements to 0, again no data request to transmit is pending and hence the cell counter is incremented to two at time 19.

Similarly at time 24 when the TI counter times out the cell counter is incremented to a count of three. Thereafter when a data request to transmit occurs for a request to transmit 4 cells, cells are transmitted at times 26, 27 and 28 for a total of 3 cells. Similarly, the cell counter is decremented from its value of 3 to 0 corresponding to the transmission of the 3 cells at times 26, 27, and 28 in FIG. 20. At time 28, the cell counter is at 0 meaning that the fourth cell to be transmitted by the data request is still pending but cannot be sent at time 29. However, when at time 29 when the TI counter again is decremented to 0, the cell counter of FIG. 19 is incremented to one thereby allowing a further data transmission at time 30 to occur so that the data request queue is reset to 0 after time 31 with no further requests pending, Referring to TABLE 1-2, the operation of the TI counter is as follows. If as in line 62 the TICtr counter is equal to 0 then in line 64 the TI counter is reset to the maximum value of TIQ. If it is greater then 0, then the TI counter is decremented by 1. If the TI counter is equal to 0 then the cell counter is incremented by 1 up to a maximum value of CQ.

The operation continues in line 84 of TABLE 1-2 where in line 84 if the cell counter is greater than 0, then the cell counter is decremented by 1 and in line 87, a transmission of a cell occurs on the virtual channel.

In TABLE 1-2, issue rate unit includes in lines 61 through 71 means for establishing an average rate, in lines 102 to 115 for the Xmit rate recovery routine means to restore the transmission rate in response to the absence of congestion signals, and in line 84 means to prevent the average transmis- 62 sion rate from exceeding the average rate.

Packet Aging The operation of the reassembly processor of TABLE 2-2 is controlled by the scheduler and lines 1 through 9. Whenever input data occurs in the buffer from the ATM link, then the Receive_Cell procedure is called in line 4 where the procedure appears in lines 11 through 104. When not busy carrying out the receive cell operation, and sufficient time has elapsed so that the packet aging timer of line 5 has overflowed, then the agepacket routine of lines 106 through 148 is called. The time packet aging timer is a common timer used for all packets.

Although the timer is common, each packet can have a different aging time. Referring to the lines 89 through 92, each cell that is received is examined to determine if it is a beginning of message (BOM) cell and if it is then the packet time out for that packet is set to the packet age limit. The packet age limit is the starting count which is then incremented each time the age packet routine is executed. The packet timeout value set in line 91 for a particular cell is written in line 93 into the control memory (WDesc operation in FIG. In the age packet procedure, the package timeout is incremented by one in line 136 and the incremented packet timeout is written into the buffer descriptor. If the packet timeout has been incremented Sto the overflow condition then the packet is discarded by setting the status in line 140 to inactive and writing the inactive status for the virtual channel in line 141 into the status and buffer descriptor fields. Finally in line 142 the Write for Packet_Complete_Queue makes the packet available to the host unit.

Further and Other Embodiments While the invention has been particularly shown and I 63 described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention.

Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

I

Claims (7)

1. A communication system having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or 65 more nodes, the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes.

2. The communication system of Claim 1 wherein said issue rate unit increases the transmission rate in response to the absence of congestion signals.

3. The communication system of Claim 1 wherein the transmission interval for a congestion signal returning from a congested node to the source is short thereby enabling the issue rate unit to reactively modify the transmission rate and thereby reduce congestion.

4. The communication system of Claim 1 wherein the transmission interval for a congestion signal returning from a congested node to the source is less than the interval between information signals transmitted over the virtual channel whereby the issue rate unit reactively responds to modify the transmission rate to reduce congestion. The communication system of Claim 1 wherein the transmission interval for congestion signal returning from a congested node to the source is less than ten intervals between information signals transmitted over the virtual channel whereby the issue rate unit reactively responds to modify the transmission rate to reduce congestion.

6. The communication system of Claim 1 wherein the processing time required to generate congestion signal at each node is much less than said transmission interval for congestion signal. I

66- 7. The communication system of Claim 1 wherein said issue rate unit has means for decreasing said transmission rate exponen- tially in response to a congestion signal. 8. The communication system of Claim 7 wherein said issue rate unit, after decreasing said transmission rate, has means for increasing said transmission rate exponentially in response to the absence of a congestion signal. 9. The communication system of Claim 8 wherein the rate of increasing said transmission rate is less than the rate of 99 decreasing said transmission rate. "1 0. The communication system of Claim 1 wherein said forward information is fragmented into cells, each cell including a virtual channel identifier, and wherein each node includes a switching element, each switching element including, a switch fabric, Sa plurality of port controllers interconnected by said switch fabric, each of said port controllers including, an input controller, an output controller, congestion detection means for detecting congestion in 9. 9 the node, reversing means for copying a cell, having a virtual channel identifier, from the forward path to the return path to thereby generate a congestion signal on said return path. 11. The communication system of Claim 10 wherein said reversing means includes reverse marker means for marking copied cells with a return path indicator whereby return cells are distin- guished from forward cells. a I I 67 12. The communication system of Claim 10 wherein each of said port controllers includes selector means for selecting cells from one virtual channel for the forward path or from another virtual channel for the return path. 13. The communication system of Claim 12 wherein said selector means grants priority to cells from the return path whereby congestion signals are given priority in the network. 14. The communication system of Claim 10 wherein said forward paths include queues for storing cells in the forward direction, each of said queues including means for providing a queue-level signal, as a function of the fullness of the queue, to form the .6 congestion signal. 00. The communication system of Claim 1 wherein said forward 0 information is fragmented into cells, each cell including a 6000 virtual channel identifier, and wherein each node includes a switching element, each switching element including, a switch fabric, 0000 .000 a plurality of port controllers interconnected by said switch fabric, each port controller connected to a communication link and each port controller including, °e an input controller for said link including a transla- tor for receiving the virtual channel identifier of incoming cells on the link and responsively translating the input virtual channel identifier of incoming cells to an output virtual channel identifier of outgoing cells for another link for another port controller connected to said switch fabric, an output controller for said link, congestion detection means for detecting congestion ii, 68 the node, reversing means for copying a cell, having a virtual channel identifier, from the forward path to the return path to thereby generate a congestion signal on said return path, said reversing means connecting said output virtual channel identifier to be reverse translated to said input virtual channel identifier whereby the virtual channel identifiers for cells in the forward path and in the return path over the link are the same. 16. The communication system of Claim 15 wherein said port controller includes one or more queues connected in parallel for queueing cells for the forward path and the return path of a link, said queues providing queue-level signals indicating the fullness of the queues and including selection means for selecting cells from said queues. 17. The communication system of Claim 16 wherein said selector means grants priority to cells from the return path whereby congestion signals are given priority in the network. 18. The communication system of Claim 15 wherein said reversing means includes reverse marker means for marking copied cells with a return path indicator whereby return cells are distin- guished from forward cells. 19. The communication system of Claim 15 wherein each of said port controllers includes selector means for selecting cells from one virtual channel for the forward path or from another virtual channel for the return path. The communication system of Claim 15 wherein said selector means grants priority to cells from the return path whereby 69 congestion signals are given priority in the network. 21. The communication system of Claim 15 wherein said forward paths include queues for storing cells in the forward direction, each of said queues including means for providing a queue-level signal, as a function of the fullness of the queue, to form the congestion signal. 22. A wide-area communication system having local reactive congestion control comprising, a plurality of local communication systems, each local communication system including, a plurality of local sources, each local source including a modifiable issue rate unit which transmits forward information signals at different transmission rates, said issue rate unit decreasing the transmission rate in response to the presence of congestion signals and increas- ing the transmission rate in response to the absence of .congestion signals, a plurality of local destinations, one or more local nodes forming an asynchronous transfer mode (ATM) local network connecting the sources to the destinations, each local node including, a forward path for transfer of information at different selectable transmission rates in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating 70 congestion signals in response to congestion in the local node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of local sources is linked to one of said plurality of local destina- tions through said local network by linking forward paths from one or more local nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of local destina- tions is linked to said one of said plurality of local sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said local nodes generates said congestion signals for the virtual channel in response to congestion in said one of said local nodes, one or more wide-area sources connected to one of said local networks, one or more wide-area destinations connected to another one of said local networks, a plurality of local network links interconnecting two or more local networks to form a wide-area network, said wide-area network having a plurality of virtual channels where, for each virtual channel, one of said plurality of wide-area sources is linked to one of said plurality of wide-area destinations through a plurality of said local networks over said local networks and said local network links. 23. A communication system of one or more virtual channels 71 having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals, said issue rate unit transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate in the absence of congestion and at a rate below the maximum-channel-peak-cell- rate for each channel experiencing congestion, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network 9 by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, 72 the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes. 24. A multiple issue rate source for use in a communication system of one or more virtual channels having reactive control where said system includes, one or more of said multiple issue rate sources, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning control signals in a reverse direction through the network toward a source, control signal generating means for generating control signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the control signal generating means in one of said nodes generates said control signals for the virtual channel in response to congestion in said one of said nodes, said multiple issue rate source comprising, i 73 a modifiable issue rate unit having means to transmit forward information signals at different transmission rates in response to control signals, said issue rate unit having an output for transmitting, for each of one or more virtual channels, at a maximum- channel-peak-cell-rate in the absence of a control signal and at a rate below the maximum-channel-peak- cell-rate for each channel providing a control signal. A multiple issue rate source for use in a communication system where the communication system has one or more multiple issue rate sources connected through one or more channels to plurality of destinations and has control signal generating means for generating control signals for the channels for reactively controlling the transmission rates over the channels, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit information signals at different transmission rates in response to control signals, said issue rate unit having an output for transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate or at a rate below the maximum-channel-peak-cell-rate as a function of the control signals for each channel. P \OPERkCP50575 96 CLM 3/2/99 -74- 26. A communication system having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals and in response to average rate metering, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion. 27. The communication system of claim 26 wherein said issue C- L- I' ,01R\G(rRSO.1S 196C(LN 31199 75 rate unit includes means for establishing an average rate, means to restore the transmission rate in response to the absence of congestion signals, and means to prevent the average transmission rate from exceeding the average rate. 28. The communication system of claim 26 wherein the transmission interval for a congestion signal returning from a congested node to the source is short thereby enabling the issue rate unit to reactively modify the transmission rate and thereby reduce congestion. 29. The communication system of claim 26 wherein the transmission interval for a congestion signal returning from a congested node to the source is less than the interval between information signals transmitted over the virtual channel whereby the issue rate unit reactively responds to modify the transmission rate to reduce congestion. The communication system of claim 26 wherein the transmission interval for congestion signal returning from a congested node to the source is less than ten intervals between information signals transmitted over the virtual channel whereby the issue rate unit reactively responds to modify the transmission rate to reduce congestion. 31. The communication system of claim 26 wherein the processing time required to generate congestion signal at each node is much less than said transmission interval for congestion signal. 32. The communication system of claim 26 wherein said issue rate unit has means for decreasing said transmission rate exponentially in response to congestion signals. 33. The communication system of claim 32, wherein said issue I 1.1 C P 101IBR\OG(K05 5'O 96 CIAM ICON -76- rate unit includes means for establishing an average rate, means for decreasing said transmission rate due to congestion signals, and means for increasing said transmission rate exponentially in response to the absence of congestion signals, and means to prevent the average transmission rate from exceeding the average rate. 34. The communication system of claim 33 including means for increasing said transmission rate at a rate less than the rate of decreasing said transmission rate. The communication system of claim 26 wherein said forward information is segmented into cells, each cell including a virtual channel identifier, and wherein each node includes a switching element, each switching element including, a switch fabric, a plurality of port controllers interconnected by said switch fabric, each of said port controllers including, an input controller, an output controller, congestion detection means for detecting congestion in the node, reversing means for copying a cell, having a virtual channel identifier, from the forward path to the return path to thereby generate a congestion signal on said return path. 36. The communication system of claim 35 wherein said reversing means includes reverse marker means for marking copied cells with a return path indicator whereby return cells are distinguished from forward cells. 37. The communication system of claim 35 wherein each of said port controllers includes selector means for selecting cells Sfrom one virtual channel for the forward path or from another I I e I' \()PEl\('lCIP50575 96 CLM 3/2/99 -77- virtual channel for the return path. 38. The communication system of claim 37 wherein said selector means grants priority to cells from thz return path whereby congestion signals are given priority in the network. 39. The communication system of claim 35 wherein said forward paths include queues for storing cells in the forward direction, each of said queues including means for providing a queue-level signal, as a function of the fullness of the queue, to form the congestion signal. The communication system of claim 39 wherein said forward information is segmented into cells, each cell including a virtual channel identifier, and wherein each node includes a switching element, each switching element including, a switch fabric, a plurality of port controllers interconnected by said switch fabric, each port controller connected to a communication link and each port controller including, an input controller for said link including a translator for receiving the virtual channel identifier of incoming cells on the link and responsively translating the input virtual channel identifier -f inccning cells to an output virtual channel identifier of _it,oing cells for another link for another port controller connected to said switch fabric, an output controller ror said link, congestion detection meanb for detecting congestion in the node, reversing means for copying a cell, having a virtual channel identifier, from the forward path to the return path to thereby generate a congestion signal on said return path, said reversing means connecting :a±id output virtual channel z identifier to be reverse translated to said input virtual I P \(OPEIR\C(P'\50OS5 96 (I.Ml 32/99 -78- channel identifier whereby the virtual channel identifiers for cells in the forward path and in the return path over the link are the same. 41. The communication system of claim 35 wherein said port controller includes one or more queues connected in parallel for queueing cells for the forward path and the return path of a link, said queues providing queue-level signals indicating the fullness of the queues and including selection means for selecting cells from said queues. 42. The communication system of claim 41 wherein said selector means grants priority to cells from the return path whereby congestion signals are given priority in the network. 43. The communication system of claim 40 wherein said reversing means includes reverse marker means for marking copied cells with a return path indicator whereby return cells are distinguished from forward cells. 44. The communication system of claim 40 wherein each of said port controllers includes selector means for selecting cells from one virtual channel for the forward path or from another virtual channel for the return path. The communication system of claim 40 wherein said selector means grants priority to cells from the return path whereby congestion signals are given priority in the network. 46. The communication system of claim 40 wherein said forward paths include queues for storing cells in the forward direction, each of said queues including means for providing a queue-level signal, as a function of the fullness of the queue, to form the congestion signal. -I P \01TIMNIN/S0575 96 CLM M1299 -79- 47. A wide-area communication system having local reactive congestion control comprising, a plurality of local communication systems, each local communication system including, a plurality of local sources, each local source including a modifiable issue rate unit which transmits forward information signals at different transmission rates, said issue rate unit decreasing the transmission rate in response to the presence of congestion signals and increasing the transmission rate in response to the absence of congestion signals and in response to average rate metering, a plurality of local destinations, one or more local nodes forming an asynchronous transfer mode (ATM) local network connecting the sources to the destinations, each local node including, a forward path for transfer of information at different selectable transmission rates in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the local node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of local sources is linked to one of said plurality of local destinations through said local network by linking forward paths from one or more local nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of local destinations is linked to said one of said plurality of local sources through said network by linking return paths from said one or more nodes, P /OPER\G(P505O75 96 V-1/2/99 the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion, one or more wide-area sources connected to one of said local networks, one or more wide-area destinations connected to another one of said local networks, a plurality of local network links interconnecting two or more local networks to form a wide-area network, said wide-area network having a plurality of virtual channels where, for each virtual channel, one of said plurality of wide-area sources is linked to one of said plurality of wide-area destinations through a plurality of said local networks over said local networks and said local network links. 48. A communication system of one or more virtual channels having reactive congestion control comprising, a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals, said issue rate unit transmitting, for each of one or more channels, at a maximum-channel-peak-cell-rate in the absence of congestion and at a rate below the maximum-channel-peak-cell- rate for each channel experiencing congestion and transmitting under control of average rate metering, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, Scongestion signal generating means for generating 0 IOIFR CI505o75%9C(L..N V22 91 81 congestion signals in response to congestion in the node, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means generates said congestion signals for the virtual channel in response to congestion. 49. A communication system having one or more virtual channels each having reactive control and average rate metering where said system includes, one ore more multiple issue rate sources, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting sources to destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning control signals in a reverse direction through the network toward a source, control signal generating means for generating control signals in response to congestion, for each of one or more virtual channels in the network, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, said one of said plurality of destinations is linked I V~f~7i 6 1. .M 12/99

82- to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the control signal generating means in one of said nodes generates said control signals for the virtual channel in response to congestion, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit forward information signals at different transmission rates in response to control signals and average rate metering, said issue rate unit having an output for transmitting, for each of one or more virtual channels, at a maximum-channel-peak-cell- rate in the absence of a control signal and at a rate below the maximum-channel-peak-cell-rate for each channel providing a control signal. A multiple issue rate source for use in a communication system where the communication system has one or more multiple issue rate sources connected through one or more channels to a plurality of destinations and has control signal generating means for generating control signals for the channels for reactively controlling the transmission rates over the channels and for controlling the transmission rates over the channels with average rate metering, said multiple issue rate source comprising, a modifiable issue rate unit having means to transmit information signals at different transmission rates in response to control signals, said issue rate unit having an output for transmitting, for each of one or more channels, at a maximum- channel-peak-cell-rate or at a rate below the maximum-channel- peak-cell-rate as a function of the control signals for each channel. 51. A communication system having reactive congestion control IN comprising, P JPI R )6 I .1st 112('I) -83- a plurality of sources, each source including a modifiable issue rate unit which transmits forward information signals at different transmission rates in response to congestion signals and includes average rate metering means for insuring that the transmission rate does not exceed an average rate, said issue rate unit decreasing the transmission rate in response to congestion signals, a plurality of destinations, one or more nodes forming an asynchronous transfer mode (ATM) network connecting the sources to the destinations, each node including, a forward path for transfer of information in a forward direction toward a destination through the network, a return path for returning congestion signals in a reverse direction through the network toward a source, congestion signal generating means for generating congestion signals in response to congestion in the node, said network having a plurality of virtual channels where, for each virtual channel, one of said plurality of sources is linked to one of said plurality of destinations through said network by linking forward paths from one or more nodes, and wherein said issue rate unit transmits forward information signals at different transmission rates in response to congestion signals over the virtual channel, said one of said plurality of destinations is linked to said one of said plurality of sources through said network by linking return paths from said one or more nodes, the congestion signal generating means in one of said nodes generates said congestion signals for the virtual channel in response to congestion in said one of said nodes. 52. A communication system substantially hereinbefore Sdescribed with reference to the accompanying drawings. 11-- e I P li|R\ C'P.i075 6 CI.M 3 W99 -84- 53. A multiple issue rate source for use in a communication system substantially as hereinbefore described with reference to the accompanying drawings. DATED this 3rd day of February 1999 NETWORK EQUIPMENT TECHNOLOGIES, INC. By DAVIES COLLISON CAVE Patent Attorneys for the Applicant %I ABSTRACT Reactive congestion control in an asynchronous transfer mode (ATM) network where the network is formed by the inter-connection of nodes each including a forward path for transfer of information from source to destination through the network and a return path for returning congestion control signals. Each source includes a modifiable issue rate unit which 15 issues forward information signals at different rates for virtual channels in response to the :°eooe presence and absence of congestion signals received on the return path. A segmentation V communication unit segments packets into a plurality of cells concurrently for a plurality of channels for transmission over the ATM network. A reassembly communication unit i reassembles cells received from the ATM network concurrently for the plurality of channels. Pipelined processing units are employed for segmentation and for reassembly. i