WO2001026305A1 - Fair flow controlling method in packet networks - Google Patents

Abstract

There is provided a fair flow controlling method in a packet switching network. The packet switching network has a plurality of nodes and each node is connected to a plurality of sources which transmit/receive data and a data queue related with a current queue length and a target queue length for storing data received from the sources. In the fair flow controlling method, each node estimates the number of locally bottlenecked virtual circuits (VCs) using the explicit rate (ER) and the minimum cell rate (MCR) of a corresponding source. The node allocates an ER to each source based on the difference between the current queue length and the target queue length of the node, the derivative of the current queue length, and the estimated number of the locally bottlenecked VCs, then conveys the ER to each source through a feedback signal.

Description

FAIR FLOW CONTROLLING METHOD IN PACKET NETWORKS

BACKGROUND OF THE INVENTION

1 Field nf the Invention

The present invention relates generally to a packet switching network, and in particular, to a fair flow controlling method.

2. Description nf the Related Art

Packet switching networks include ATM (Asynchronous Transfer Mode) networks and the Internet. A fair flow control has significant effects in the information transmission over a packet switching network. In particular, a fair flow control of the ATM networks is related to the ABR (Available Bit Rate) service.

The ATM layer provides the following four services: CBR (Constant Bit Rate), VBR (Variable Bit Rate), UBR (Unspecified Bit Rate), and ABR. In the ABR service, the source dynamically changes its bit rate within the available bandwidth from the network to adapt the changing network conditions when transmitting the data. The ABR service was introduced to the ATM network in order to support data applications that were not efficiently supported by a guaranteed bandwidth service, such as the VBR service. For details, see S. Sathaye, ATM forum Traffic Management Specification, Version 4.0, Feb. 1996, F. Bonomi and K. W. Fendick "The Rate-Based Flow Control Framework for the Available Bit Rate ATM Service", IEEE Network, vol. 9, no. 2, pp.

25-39, 1995, and R. Jain "Congestion Control and Traffic Management in ATM Networks: Recent Advances and Survey", Computer Network and ISDN Systems, vol. 28, no. 13, pp. 1723-1738, 1996.

Most data applications are highly bursty and thus hard to predict the data traffic.

To make successful data transmission, the network has to work comply with a certain condition satisfying the predefined cell loss requirements, time-varying tolerance requirements, and cell delays. Due to these characteristics, the network has to modify their data transfer rates according to the network loading conditions. Thus, the notion of elastic traffic services was introduced in which the data transfer rates are adjusted based on the bandwidth available from the network. A representative example of the elastic traffic services is the ABR service of ATM networks. The ABR service, m principle, does not require a complex traffic characterization and call admission control, respectively, at the source and at the switches Due to this simplicity, it has been expected that the implementation and the deployment of the ABR service would be much easier than those of the bandwidth-guaranteed services, i e CBR or VBR service In reality, however, the implementation of ABR-capable switches appears to be much more difficult than originally expected The difficulty mainly lies in the designing of a simple, scalable, and stable ABR flow algorithm, more specifically an ER (Explicit Rate) allocation algorithm in an asynchronous and distributed network environment

The ATM forum has selected a closed-loop rate-based approach for the ABR flow control The closed-loop rate-based flow control approach, as its name implies, uses feedback information from the network to control the data rate at which each source can transmit a number of cells into the network The feedback information is conveyed to the source through a specific control cell known as a resource management (RM) cell There are three mechanisms for a switch to write its congestion status onto a RM cell' Explicit Forward Congestion Indication (EFCI) marking, Relative Rate (RR) marking, and ER marking, at least one of which has to be implemented on a switch for the rate-based flow control

Meanwhile, the long and diverse round trip delays (RTDs) involved in the closed loop and the distributed bottle neck locations of the ABR VCs (Virtual Circuits) make it difficult to design a high performance ER allocation algorithm ABR queues in the network can hardly be stabilized when the transmission rates of ABR sources are determined based on the network state information at different times In particular, if only a binary feed-back mechanism (either EFCI or RR marking, or both) is employed, ABR queues in a steady state inevitably exhibit persistent oscillation with its magnitude and period being an increasing function of the delay-bandwidth product For details, see E Hernandez- Valencia, et al , "Rate Control Algorithms for the ATM ABR Service",

European Transactions on Telecommunications, vol 8, no 1, pp 7-20, 1997, F Bonomi, D Mitra, and J B Serry "Adaptive Algorithms for Feedback-Based Flow Control in High-Speed, Wide-Area ATM Networks", LEEE J Select Areas on Communications, vol 13, no 7, pp 1267-1283, 1995, and K K Ratmarkrishnan and Jain "A Binary Feedback Scheme for Congestion Avoidance in Computer Networks with a Connectionless

Network Layer", Proc ACM SIGCOMM'88, pp 303-313, 1988 Such an oscillatory behavior of the ABR queues will increase the likelihood of the cell loss and the link under-utilization due to the periodic buffer overflow and underflow An ABR flow control scheme using the ER marking has been introduced to realize an asymptotic stability of the ABR queues, hence overcome the drawback of the binary feedback mechanisms Still, designing an asymptotically stable ER allocation algorithm, particularly in a simple form, is a difficult task This problem naturally falls into the feedback control problem having a delay

L Benmohanmed and S M Meerkov formulated the rate-based flow control problem as a discrete-time feedback control problem with a delay, and derived an ER allocation algorithm that achieves asymptotic stability and allows arbitrary control of the closed-loop performance This information is disclosed in "Feedback Control of Congestion in Packet Switching Networks The Case of Single Congested Node", IEEE/ACM Trans On Networking, vol 1, no 6, pp 693-708, 1993, and "Feedback Control of Congestion in Packet Switching Networks The Case of Multiple Congested

Nodes", International Journal of Communication Systems, vol 10, no 5, pp 227-246, 1997 Their ER allocation algorithm is formed as

' ^rmax r[k + ϊ] = r[k] - ∑ a, (q[k - ι]- q_τ) - ∑ β [k - j] . (1) = 0

where r[k] is the ER calculated by the switch at the discrete time k, q[k] is per-class ABR queue length at the time k, qT is a target queue length, αi and βj are controller gains, τmax is the largest RTD of an ABR VC, and I is an arbitrary constant greater than 0

Despite its solid theoretical foundation, the practical use of the algorithm (1) is limited by its high degree of the implementation complexity The shortcomings and the limitations of the algorithm are described in A Kolarov and G Ramamurthy, "A Control Theoretic Approach to the Design of Close Loop Rate Based Flow Control for High Speed ATM Networks", Proc , IEEE LNFOCOM'97, vol 1, pp 293-301, 1997 In the ER allocation algorithm, the ER terms should be maintained at present and in the past, up to time lags τmax, and a number of floating point multiplications should be performed in every discrete time slot

At the same time, S Chong has proposed a simpler control-theoretic ER allocation algorithm ("Second-Order RateOBased Flow Control with Dynamic Queue

Threshold for High-Speed Wide- Area ATM Networks", preprint 1997) A Elwalid ("Analysis of Adaptive RateOBased Congestion Control for High-Speed Wide-Area Networks", Proc IEEE ICC'95, pp 1948-1953, 1995) has proposed a continuous-time ER allocation algorithm, given by

f(t) = - Ar(t) - B(q(t) - q_T), A,B > 0 (2),

and obtained the necessary and sufficient condition for the closed-loop system to be asymptotically stable when the RTDs of all VCs are identical Chong has extended the stability analysis of this algorithm to the general case having arbitrary RTDs

Meanwhile, S Chong, R Nagarajan, and Y T Wang showed that a more simplified ER allocation algorithm ("Designing Stable ABR Flow Control with Rate Feedback and Open-Loop Control' First-Order Control Case", Performance Evaluation, vol 34, no 4) can readily achieve an asymptotically stable system, expressed by

r(t) = [-K(q(t)-qT)]⁺, K > 0 (3),

where [X]^* = max[x, 0] indicates that the greater of x and 0 should be selected

The ER allocation algorithm (3) is disclosed as U S Patent No 5,864,538 entitled "First-Order Rate-Based Flow Control with Adaptive Queue Threshold for ATM Networks", January 26, 1999

In the algorithm (2), two other stabilization conditions have been induced One of them is a sufficient condition for the general case with heterogeneous RTDs and the other is a necessary and sufficient condition for a particular case with homogeneous RTDs

A common drawback of the algorithms (2) and (3), as compared to the algorithm (1), is that unless the controller gains and the queue length threshold are properly chosen according to the instantaneous knowledge on the bandwidth available to ABR traffic and the fraction of the available bandwidth utilized by remotely bottlenecked VCs, the ABR queue length can converge to zero which is undesirable because, at this equilibrium point, the link cannot be fully utilized

The remotely bottlenecked VCs cannot fairly share the link since the phenomenon of bottle neck occurs at another link if the transmission rates of the VCs are not limited by their PCRs (Peak Cell Rates). In contrast, if the algorithm (1) is applied, there exists no such undesirable equilibrium point.

SUMMARY OF THE INVENTION

It is, therefore, an object of the present invention to provide a method of guaranteeing the maximal link utilization and minimal cell loss regardless of RTDs in an ABR loop

It is another object of the present invention to provide a method of minimizing

ABR queue size requirements by ensuring asymptotical stabilization of the ABR queues

It is a further object of the present invention to provide a method of guaranteeing MAX-MIN fairness based on the ATM forum standards by ensuring a fair share of an available bandwidth to each ABR service user

It is still another object of the present invention to provide a method of increasing responsiveness and transient control performance to a communication network environmental change, such as changes in the number of ABR users and an ABR bandwidth

It is yet another object of the present invention to provide a method of providing all functions including EFCI, RR, and ER markings as specified in the ATM forum traffic management specification

It is another object of the present invention to provide a method of achieving high utilization, low cell loss, and MAX-MIN fair rate allocation through the existence of an asymptotically stable operating point

It is further object of the present invention to provide a method of increasing the responsiveness to the network loading changes at multiple time scales, that is, at the cell level rate changes of the VBR and ABR VCs and at the cell level arrivals and departures ofthe VBR and ABR VCs

It is yet another object of the present invention to provide a method of minimizing the number of operations required to compute an ABR service algorithm and to achieve low and scalable degrees of the implementation complexity, by virtually removing per-VC operations including per-VC queuing, per-VC accounting, and per-VC table access

The above object of the present invention can be achieved by providing a fair flow controlling method in a packet switching network The packet switching network includes a plurality of nodes and each node is connected to a plurality of sources which transmits/receives data, and includes a data queue related with a current queue length and a target queue length for storing data received from the sources In the fair flow controlling method, each node estimates the number of locally bottlenecked VCs using the ER and the MCR to be guaranteed of a corresponding source The node allocates an

ER to each source based on the difference between the current queue length and the target queue length of the node, the derivative of the current queue length, and the estimated number of the locally bottlenecked VCs, then conveys the ER to each source through a feedback signal

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features, and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which

FIG 1 illustrates a network configuration to provide the ABR service, FIG 2 illustrates fields in an RM cell,

FIG 3 schematically illustrates a switch fabric for the ABR service having I/O ports connected to I/O cards, FIG 4 is a detailed block diagram of an I/O card shown in FIG 3,

FIG 5 is a block diagram of an ABR service engine according to the preferred embodiment of the present invention,

FIG 6 illustrates a network model with a node of interest, FIG 7 is a graph showing a stable region with respect to U and V according to the preferred embodiment of the present invention,

FIG 8 illustrates an asymptotic decay rate as a function of U and V, FIG 9 illustrates queue length thresholds for ER marking and RR marking, FIG 10 illustrates a peer to peer configuration,

FIGs 11A to 11D illustrate simulation results in the peer to peer configuration with ER marking only and with no VBR background traffic, wherein FIG 11A shows a source transmission rate ai(t) of VCs with PCR = 150Mbps, FIG 1 IB shows a source transmission rate ai(t) of VCs with PCR = 25Mbps, FIG 1 IC shows a queue length at a switch SW1, and FIG 11D shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW1,

FIGs 12A to 12D illustrate simulation results in the peer to peer configuration with ER marking and VBR background traffic, wherein FIG 12A shows a source transmission rate ai(t) of VCs with PCR = 150Mbps, FIG 12B shows a source transmission rate ai(t) of VCs with PCR = 25Mbps, FIG 12C shows a queue length at the switch SW1, and FIG 12D shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW1,

FIG 13 illustrates a parking lot configuration, FIGs 14A to 14F illustrate simulation results in the parking lot configuration with ER marking only and no VBR background traffic, wherein FIG 14A shows a source transmission rate ai(t) of VCs with PCR = 150Mbps, FIG 14B shows a source transmission rate ai(t) of VCs with PCR = 25Mbps, FIG 14C shows a queue length at a switch SW3, FIG 14D shows a queue length at a switch SW4, FIG 14E shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW3, and FIG 14F shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW4,

FIGs 15A to 15F illustrate simulation results in the parking lot configuration with the ER marking and VBR background traffic, wherein FIG 15A shows a source transmission rate ai(t) of VCs with PCR = 150Mbps, FIG 15B shows a source transmission rate ai(t) of VCs with PCR = 25Mbps, FIG 15C shows a queue length at the switch SW3, FIG 14D shows a queue length at the switch SW4, FIG 15E shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW3, and FIG 15F shows the estimated number |Q|avg(t) of locally bottlenecked VCs at the switch SW4, and FIG 16 is a flowchart illustrating the ER allocation control operation in the

ABR service engine according to the preferred embodiment of the present invention

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A preferred embodiment of the present invention will be described hereinbelow with reference to the accompanying drawings For the purpose of clarity, well-known functions or constructions are not described in detail as they would obscure the invention in unnecessary detail

FIG 1 illustrates a network configuration for implementing the ABR service

With reference to FIG 1 , a network 2 for providing the ABR service includes a plurality of nodes 4, 6, 8, and 10 The nodes conceptually represent switches and will be referred to as switches hereinafter Each switch is connected to a plurality of sources In the drawing, the switch 4 is connected to sources A and C as indicated by reference numerals 12 and 14 The switch 8 is connected to sources B and D as indicated by reference numerals 16 and 18 Each source transmits/receives data through the switch 4 or 8 connected to the source Data transmitted from the source reaches a destination through a so-called VC path having a plurality of nodes For example, data from the source A 12 reaches the destination, the source B 16, through a VC path having the switches 4, 6, and

In the ABR service, the bandwidth availability of the network is conveyed to the source through special cells called RM (Resource Management) cells In FIG 1, RMc denotes an RM cell While RM cells are generated in the sources 12, 14, 16, and 18 or the switches 4, 6, 8, and 10, the following description is confined to a source-generated RM cell A source-generated RM cell is transmitted to a destination through the switches

4, 6, and 8 through a VC path The transmission direction of the cell is forward Upon receiving the forward RM cell, the destination returns the cell (backward) back to the source In FIG 1, reference numerals 12 and 14 denote sources, and reference numerals 16 and 18 denote destinations With respect to the sources and destinations, the forward and backward directions are defined as shown The switches 8, 6, and 4 write their allowed bandwidth information in the backward RM cell The source 12 or 14 adapts its rate according to the changing network conditions based on the received bandwidth information

An RM cell contains information that includes fields of CCR (Current Cell Rate),

MCR (Minimum Cell Rate), ER, NI (No Increase), and CI (Congestion Indication), as shown in FIG 2 In FIG 2, an S-R (Source-Receive) field provides information related to a source and a destination information The CCR field is set by the source to its current ACR (Allowed Cell Rate) when the source generates the RM cell The MCR field indicates the minimum bandwidth allocated to each VC when the source generates the RM cell ER is an available bandwidth written in a backward source-generated RM cell by the ABR service engine of a switch as the RM cell passes through the switch Only if a calculated available bandwidth of the ABR service engine is less than the existing available bandwidth, the former value is written in the ER field Thus, the smallest available bandwidth in the VC path is conveyed to the source To this end, the source writes its PCR in the ER field when it generates the RN cell CI and NI are fields used to control RR CI notifies the source that the network is much congested and thus the bandwidth of the source is to be reduced, and NI is used to prevent a source from increasing its ACR

A switch calculates a bandwidth available for the ABR service and writes the available bandwidth information in a backward RM cell, using a switch algorithm What is intended from the switch algorithm is to obtain a bandwidth allowable to a corresponding VC by the switch This available bandwidth is conveyed to the source so that the source adapts its rate to provide the ABR service reliably

As shown in FIG 3, a switch fabric 20 has I/O ports each connected to an I O port card 22 in each switch shown in FIG 1 Each I/O port card includes an I/O buffer management unit 30, an ABR service engine 32, and an output interface 34, as shown in FIG 4 The I O buffer management unit 30 is connected to the switch fabric 20 and responsible for I/O queuing The I/O buffer management unit 30 includes ABR queues 36 The ABR service engine 32 implements an ABR algorithm and its related operations for the ABR service while an RM cell is passing through the ABR engine 32 according to the embodiment of the present invention The output interface 34 acts as a user network interface at the ATM layer

FIG 5 is a block diagram of the ABR service engine 32 shown in FIG 4 With reference to FIG 5, the ABR service engine 32 is comprised of an EFCI marker 40, a |Q| estimation unit 42, an ER engine 44, an average queue calculator 46, and a backward RM cell writer 48

At EFCI congestion, the EFCI marker 40 marks an EFCI bit in an input forward data cell to indicate the EFCI congestion A queue read signal is generated when a cell is removed from a queue and a queue write signal is generated when a new cell enters the queue Upon receipt of the queue write signal, the average queue calculator 46 increases a queue instantaneous variable by one, and upon receipt of the queue read signal, it decreases the queue instantaneous variable by one, to calculate an average queue length within a predetermined period interval for the ER computation The average queue calculator 46 feeds the average queue length q[k] to the ER engine 44 The average queue calculator 46 decides as to whether the EFCI congestion has occurred using the queue read signal and the queue write signal and in case of the EFCI congestion, it outputs an EFCI congestion signal EFCI CG to the EFCI marker 40 According to the embodiment of the present invention, the average queue calculator 46 also outputs a signal CG representative of "congested" condition and a signal VCG representative of "very congested" condition to the backward RM cell writer 48 The congested state and the very congested state are determined based on a predetermined lowest queue length threshold qLT and a predetermined highest queue length threshold qHT for an ABR queue The signals CG and VCG are used to control RR according to the embodiment of the present invention

The |Q| estimation unit 42 estimates the number of locally bottlenecked VCs by reading CCR and MCR from an RM cell and comparing CCR-MCR with a periodically calculated ER r(t) Interval, W, at which the estimated value |Q is calculated will be described later in detail in "(5) Discrete Time ER Algorithm and |Q| Estimation" The Q| is provided to the ER engine 44

The ER engine 44 updates the ER through a periodical ER calculation and outputs the latest ER r(t) to the backward RM cell writer 48 upon arrival of a backward RM cell

The backward RM cell writer 48 writes r(t) in the ER field of the backward RM cell More specifically, the backward RM cell writer 48 compares the ER of the received RM cell with r(t) + the MCR of the RM cell (r(t)+mι) and writes r(t)+mι in the RM cell only when r(t)+mι is less than the ER of the RM cell The backward RM cell writer 48 writes a binary logic state bit in the NI and CI fields of the RM cell according to the signals CG and VCG received from the average queue calculator 46, for control of RR

FIG 16 is a flowchart illustrating an ER allocation control operation in the ABR service engine

With reference to FIGs 5 and 16, the |Q| estimation unit 42 estimates Q in step 100 and the ER engine 44 updates ER through a periodical calculation in step 102 Upon arrival of an RM cell in the backward RM cell writer 48 in step 104, the ABR service engine 32 proceeds to step 106, otherwise, it returns to step 100 In step 106, the backward RM cell writer 48 reads the latest ER r(t) from the ER engine 44 and the MCR mi from the received RM cell The backward RM cell writer 48 calculates an ER allocation value π(t) for VCi (an ι^th VC) by r(t)+mι in step 108 and writes π(t) in the ER field of the backward RM cell in step 110

The purpose of the switch algorithm suggested in the present invention is to converge the AVR queue 36 to a target queue length qT and maintain a queue length at a steady state at the target queue length qT, which implies that the input traffic is equal to the output traffic of the ABR queue 36 A transient period is generated in the process of achieving the target queue length qT In case the input traffic is momentarily greater than the output traffic, the ABR queue 36 is overloaded As a result, unexpected cell loss might occur Accordingly, a switch algorithm is implemented considering the buffer capacity and queue length variation of the ABR queue 36 in the embodiment of the present invention

In order to keep a controller as simple as possible and at the same time remove undesirable non-equilibrium, a continuous-time ER allocation algorithm is proposed in accordance with the embodiment of the present invention,

r( = - *( - tø( - ?_r), A, B > 0 (4)

where r(t) is an ER calculated by the ER engine, r(t) is the derivative of r(t), Q is the set of locally bottlenecked VCs, |Q| is the cardinality of Q, q(t) is the length of the ABR queue 36 at a time t, and A and B are controller gains that are varied depending on the length of the ABR queue 36 for asymptotic stability in the embodiment of the present invention

It is to be noted that the ER allocation algorithm (4) uses q(t) instead of r(t) in the damping term (the first term in the right-hand side) as compared to the algorithm (2) q(t) is the derivative of q(t), which will be described in relation with Eq (21) and Eq (23)

This change indeed removes the aforementioned undesirable non-equilibrium in a closed-loop system, consequently, the ABR queue 36 always converges to the target queue length qT irrespective of the available bandwidth and the fraction of the available bandwidth used by remotely bottlenecked VCs, which implies that full utilization of the available bandwidth is always guaranteed in the steady state

Another notable feature of the proposed algorithm is the normalization of the controller gains A and B by the number of locally bottlenecked VCs, |Q| This normalization makes the asymptotic decay rate of the closed-loop system to be independent of IQI Estimation of |Q| has been an important research subject in the rate-based ABR flow control research (for reference, see M K. Wong and F Bonomi, "A Novel Explicit Rate Congestion Control Algorithm", it Proc LEEE GLOBECOM'98, vol 4, pp 2432- 2439, 1998, L Kalampoukas, A Varma and K K Ramarkrishnan, "An Efficient Rate Allocation Algorithm for ATM Networks Providing MAX-MTN Fairness", Technical

Report UCSC-CRL-95-29, Computer Engineering Dept , University of California, Santa Cruz, June 1995, A Charny, K K Ramakrishnam and A Lauck, "Time Scale Analysis and Scalability Issue for Explicit Rate Allocation in ATM Networks", LEEE/ ACM Trans On Networking, vol 4, pp 569-581, 1996, and R Jain et al , "ERICA Switch Algorithm A Complete Description", ATM Forum/96- 1172, 1996)

The difficulty involved in that |Q| estimation lies in that the dynamics of any |Q| estimation process are coupled with those of the ER allocation process, i e , ER updates affect IQI updates and vice versa until the closed-loop system reaches the steady state so that improper design of an |Q| estimation algorithm can make the closed-loop system unstable In view of the foregoing, a stable yet scalable |Q| estimation algorithm is proposed in relation with the ER allocation algorithm according to the embodiment of the present invention

The control-theoretic ER allocation algorithm (4) of the present invention achieves MAX-MIN fair rate allocation according to the principle as described below An identical ER (common ER) is allocated to all VCs, based on Eq (4), sharing the same link Thus, the minimum allocated ER in the path is conveyed to a source through an RM cell and the source transmits data at the minimum ER If other VCs in a different location are bottlenecked and the source transmits data at a rate less than the switch-allocated common ER, the ABR queue 36 will experience time delay and the delay will be decayed In case the ABR queue 36 gets below the target queue length qT, the switch increases the common ER until the ABR queue 36 reaches the target queue length qT in the algorithm (4) Therefore, each locally bottlenecked VC can get a fair share of the bandwidth that is not used by remotely bottlenecked VCs

The algorithm (4) can be approximated to the following discrete time algorithm when it applies in reality

r[k + \] = r[k] - ^(q[k] - q[k - \]) - ^(q[k] - q_r\ A, B > 0 (5) where T is the duration of a discrete time slot The algorithm (5) is a special case of the

A BT algorithm (1) That is, with 1=1, αl= - Γ , α0+αl=-j— r, and βj=0, Vj, the algorithm (1)

is reduced to the algorithm (5) In contrast to the algorithm (1), the reduced algorithm no longer allows for arbitrary control of the closed-loop dynamics However, the present inventors argue that having the capability of arbitrary control is too expensive in terms of implementation cost and may not be necessary for the ABR flow control design To support this argument, it will be shown that the reduced algorithm (5) in fact allows for an acceptable level of control for the closed-loop performance

In the control-theoretic ER allocation algorithms (4) and (5), the queue length control is a primary concern, and the fair rate allocation is a by-product of the queue length control There is another class of ER allocation algorithm (14) to (18) where the fair rate allocation is a primary concern, and the queue length control, if any, is supplementary In this type of algorithms, it is necessary for each switch to track the available bandwidth, the fair share of remotely bottlenecked VCs at their own bottleneck link, and the number of locally bottlenecked VCs Based on this information, the switch has to update per-VC ER allocation in such a way that the MAX-MIN fair rate allocation and target link utilization are asymptotically achieved

( 1 ) ER Rate Flow Control

The ER allocation algorithm of the present invention will be modeled mathematically and it will be shown that the ER allocation algorithm guarantees an MCR required by each source and at the same time satisfies the MAX-MIN fairness

FIG 6 illustrates a network model with a node of interest In FIG 6, a plurality of VCs are matched with a plurality of corresponding sources, VCi 54 to source l 50 and VCj 56 to source j 52 Reference numeral 36 denotes the ABR queue, qT is the target queue length, and μ denotes the available bandwidth of a link shared by the plurality of VCs τι^f and τι^b are forward and backward path delays, respectively, in VCi 54, and the sum of τι^f and τι^b is the RTD τi of VCi 54 τj^f and τj^b are forward and backward path delays, respectively, in VCi 56, and the sum of τj^f and τj^b is the RTD τj of VCi 56

The network model is analyzed based on the following assumptions

A 1 Traffic is viewed as a deterministic fluid flow, and the network queuing process and the flow control mechanism are continuous in time This assumption enables the closed-loop system to be modeled by a differential equation

A 2 The RTD τi of VCi 54 is the sum of the forward path delay τi^f and the backward path delay τi^b, which includes propagation, queuing, transmission, and processing time Here, the RTD is assumed to be constant

A 3 The sources are persistent until the system reaches a steady state The term "persistent" means that the sources always have enough data to transmit at the allocated rate

A 4 There are no arrivals and departures of VCs until the system reaches the steady state

A 5 The available bandwidth μ at the link is constant until the system reaches the steady state Also, the buffer size at the link is assumed infinite

Let αi(t) and ri(t) respectively denote the rate at which source i transmits data at source time t and the ER of VCi computed by the node of interest at node time t Also, let βi(t) and pi(t) respectively denote the latest minimum of ERs allocated to VCi by the nodes along the VCi's path, except the one allocated by the node of interest and the PCR constraint of VCi

Neglecting the linear increase and exponential decrease operation based on binary feedbacks, the source behavior can be modeled as

αi(t) = min[ri(t-τi^b), bi(t), pi], Vi e N (6),

where N is the set of all the VCs whose route includes the node of interest This model implies that source i 54 transmits data at the smallest value among an ER allocated by the node of interest before the backward path delay τi^b, ri(t— τi ), a minimum ER on the route except the node of interest, bi(t), and the PCR of the VC, pi

The dynamics of the per-class ABR queue 36 of interest are given by

The ER allocation algorithm according to the embodiment of the present invention is a distributed algorithm which runs at each switch based on the current network state including the queue length q(t), the derivative of the queue length, q(t) , and the estimate of the number of locally bottlenecked VCs, Q , given by

ri(t) = r(t) + mi, Vi e N (8),

and

where A, B > 0 and mi denotes the MCR which the node is required to guarantee during the entire life of VCi It is assumed that mi < pi, Vi e N, and there exists a call admission control which guarantees the following condition

Σ^OT, < μ (10), ι eW

where r(t) is the common part Vi of per-VC ER allocations, ri(t) and requires most of the computation of the algorithm The only per-VC computation required is the addition of mi to the common ER, r(t) according to the embodiment of the present invention This is why this algorithm is scalable in terms of computation complexity with an increasing number of VCs Each RM cell of VCi 54 carries the value mi in the round trip

The embodiment of the present invention is characterized in that the node keeps updating the common ER in the background "Background" calculation refers to the calculation of the common ER periodically regardless of the arrival of RM cells The benefit of the background calculation is that the latest common ER, r(t) that is preliminarily updated is directly provided upon arrival of an RM cell in the corresponding node The node, that has updated the common ER, r(t) according to Eq (9) by background calculation, reads mi from the MCR field of a passing-by RM cell in VCi 54, calculates an ER ri(t) to be allocated to VCi 54 by adding mi to the latest common ER, r(t) in Eq (8), and writes ri(t) in the ER field of the RM cell

Another notable feature of the ER allocation algorithm according to the embodiment of the present invention is the normalization of the controller gains, A and B according to the estimate of the number of locally bottlenecked VCs, |Q| The normalization is optional, that is, it is not absolutely necessary but it is recommended since, as it will be described in "(4) Principal Root and Asymptotic Decay Rate", it makes the asymptotic decay rate of the closed-loop system to be independent of the number of locally bottlenecked VCs

The terms, remotely bottlenecked VC and locally bottlenecked VC, are defined in the steady state for a given network loading Remotely bottlenecked VCs in a given link are defined as those VCs which cannot achieve their fair share at the given link because either their transfer rate is limited by their PCR or they represent bottlenecked at some other link in the path In the same manner, locally bottlenecked VCs at a link are defined as those VCs which achieve their fair share at the given link

Let ais = limαJJ) , πs = limr (t) , and bis = lim^ (t) in order to define the locally bottlenecked VCs and the remotely bottlenecked VCs in mathematical terms Here, ais, ris, and bis represent, respectively, ai(t), ri(t), and bi(t) in the steady state Then, the set of all the locally bottlenecked VCs, Q is given by

Q = {i|i E N and ais = ris} (11),

and the set of all the remotely bottlenecked VCs, N-Q, is given by

N-Q = {i|i e N and ais = min[bιs, pi]} (12)

In "(5) Discrete-Time ER Algorithm and |Q| Estimation", a |Q| estimation algorithm is presented which is not only effective in that |Q| rapidly converges to |Q|, but also robust in that Q tends to be greater than |Q| during the transient period Such a robustness property is necessary for the stability of the closed-loop control (2) Steady State and Fairness

A description will be made of the steady state characteristics of the closed-loop dynamics when the ER allocation algorithm according to the embodiment of the present invention is applied. That is, there will be given analysis result according to the embodiment of the present invention. It is assumed that the closed-loop dynamics have an equilibrium point at which the derivatives of the system variables are zero, that is, ϊιmq(t) = 0 and limr(t) = 0. Then, from Eq. (6), Eq. (8), and Eq. (9), t→∞ r->∞

ais - min[ris, bis, pi], Vie N, (13),

ris = rs + mi, Vie N, (14),

and

qs = qT (15),

where qs =\im q(t) and the other notations were defined previously. Since qs=qT > 0, ϊ-^ oo

Eq. (17) implies that

∑«„ = μ (16). ιeΛ'

By combining Eq. (13), Eq. (14), and Eq. (16) and the definitions of Eq. (11) and Eq. (12), the following Eq. (17) is obtained:

∑ . + ∑w, + Σ ^min[ _s> ^>,] = (¹⁷)> ιeQ ιεg teN-Q

which implies that

As described above, the embodiment of the present invention has the following characteristics; and a result, Eq. (19) is obtained. For ∑_{; eΛf} w, < μ , there exists a unique steady state solution (equilibrium point) at which (i) the queue length is equal to the target queue length (qs=qT), (ii) the available bandwidth at the link is fully utilized (∑ _Na_u = μ ), and (iii) individual MCRs are guaranteed at the link and the bandwidth subtracted by the sum of MCRs, - ∑ _x , is fairly shared in the MAX-MIN fair sense. That is, ^ - ∑ _I i _FaN_-Q₀ min[ό„,_jp_I ] - ∑ _ιe0w₁ a„ - |ρj + ^x m'"." ' 6^c β^ (19). min[δ„,/?, ], i s N - 0

The above characteristics imply that when the ER allocation algorithm according to the embodiment of the present invention is applied, the ABR closed-loop system has a unique oscillation-free operating point at which the MAX-MIN fairness with NCR guarantee is achieved, and the queue length is equal to the target value qT no matter what the network loading is. That is, an equilibrium point is independent of the available bandwidth for the ABR traffic and the fraction of the available bandwidth utilized by remotely bottlenecked VCs including PCR-constrained VCs.

A notable point about the embodiment of the present invention is that the ER allocation algorithm results in the unique equilibrium point but the computation is very simple. The ER algorithm requires q(t) and q(t) only, not any other measurement or monitoring result. The estimate Q| , used to normalize the gains A and B, is introduced to accelerate the convergence of the queue length to the target queue length and has little to do with the equilibrium point. The Benmohamed's algorithm in Eq. (1) also results in a unique system equilibrium point with the same properties as the ER allocation algorithm of the present invention but at the cost of high-degree implementation complexity. In contrast, if the algorithms in Eq. (2) and Eq. (3) are applied, the closed-loop system possesses an equilibrium point that varies depending on the available bandwidth and the fraction of the available bandwidth utilized by the remotely bottlenecked VCs and can converge to zero at worst scenario. For details, refer to S. Chong, Nagarijan, and Y. T. Wang, "Design Stable ABR Flow Control with Rate Feedback and Open-Loop Control: First-Order Control Case", Performance Evaluation, vol. 34, no. 4, pp. 189-206, 1998, S.

Chong, "Second-Order Rate-Based Flow Control with Dynamic Queue Threshold for High-Speed Wide- Area ATM Networks", preprint 1997, and A. Elwalid, "Analysis of Adaptive Rate-Based Congestion Control for High-Speed Wide-Area Networks", Proc. IEEE ICC'95, pp. 1948-1953, 1995.

(3) Asymptotic Stability

In this part, there will be given a description of the stability of the ER allocation algorithm according to the embodiment of the present invention. In general, the stability properties of the equilibrium point given in Eq. (19) could be investigated in the case of multiple nodes. However, due to the complex nature of the coupled dynamics between nodes, the analysis could be so complicated that there is no attempt to solve the global stability problem in the coupled multi-node setting in the embodiment of the present invention. This will be only investigated by simulations in "(6) Simulation Results". On the other hand, Benmohamed and Meerkov showed that under a special service discipline there exists a neighborhood of the equilibrium point in which node-to-node dynamics are decoupled ("Feedback Control of Congestion in Packet Switching Networks: The Case of Multiple Congested Nodes", International Journal of Communication System, vol. 10, no. 5, pp. 227-246, 1997). Moreover, they showed through simulations that the local stability condition derived in the neighborhood works well for the FCFS service discipline as well. By appealing to this result, it is assumed that a neighborhood R with such properties exists in the embodiment of the present invention. A subset of the neighborhood R is considered, which satisfies: a) bi(t) = bis, VI e N, i.e., the dynamics of the other nodes are in a steady state; b) {i|i e N and ai(t) = ri(t-ri^b)} = Q and {i|i e N and ai(t) = min[bis, pi]} = N - Q, i.e., the locally bottlenecked VCs transmit data at ri(t-ri ) and the remotely bottlenecked VCs transmit data at minfbis, pi]; c) the saturation nonlinearities in Eq. (7) and Eq. (8) are not activated, i.e., both q(t) and r(t) are positive-valued; and d) the IQI estimation process is in the steady state, i.e., |Q| is constant.

In this neighborhood of the equilibrium point, the dynamic equations (6), (7), and (8) can be simplified to:

g( = ∑ a, (t - τ,^f ) - μ (21), ι eΛ^τ and

A B

^( = - r4( - rjtø( - ?_r) (22)

By combining Eq (20) and Eq (21),

q(t) = Σ mιnβ_,_jp_I ] + ∑ /; (/ - r_I ) - /t (23), ιεW-g ιeβ

Let an error function be defined as e(t) = q(t)-qT BY combining Eq (22), the differentiation of Eq (23), and the differentiation of Eq (8), the following closed-loop equation is obtained

which is a second-order retarded differential equation The characteristic equation of the closed-loop equation is given by

D(s) = s² + e "^• = 0 (25),

which has an infinite number of roots For this asymptotic stability of the closed-loop equation (24), all the roots of the characteristic equation (25) must have negative real parts (see R Bellman and K L Cooke, "Differential-Difference Equations", Academic Press, New York, 1963 and G Stepan, "Retarded Dynamical Systems Stability and Characteristic Functions", Longman Scientific & Technical, 1998)

To find the necessary and sufficient condition for D(s) = 0, one can appeal to

Pontryagin's criterion assuming the discrete delays of rational ratios (see R Bellman and K L Cooke, "Differential-Difference Equations", Academic Press, New York, 1963 and S J Bhatt and C S Hsu, "Stability Criteria for Second-Order Dynamical Systems with Time Lag", Journal of Applied Mechanics, pp 113-118, 1996) For general cases with continuous delays or discrete delays of irrational ratios, Stepan' s criterion provides a way to construct the necessary and sufficient condition (see "Retarded Dynamical Systems Stability and Characteristic Functions", Longman Scientific & Technical, 1989) However, constructing such a condition in an explicit form is extremely complicated particularly for the case with a large number of heterogeneous RTDs

Instead, the necessary and sufficient condition for the asymptotic stability is derived in the case that all the RTDs are identical Let τi = τ, Vi Then, the closed-loop equation 924) becomes

e(t) + Ae(t - r) + Be(t - r) = 0 (26)

This new equation may be normalized as the time lag τ becomes unity Let t τξ In terms of the new variable ξ, Eq (26) becomes

e(ξ) + Ue(ς - 1) + Ve(ξ - 1) = 0 (27),

where U = A τ and V = BT ² The characteristic equation of Eq (27) is

To find the necessary and sufficient condition that all the roots of H(z) = 0 have negative real parts, one can appeal to the aforementioned Pontrygin's criterion

The above theorem yields the following result Let U = and V

Then, the closed-loop equation (26) is asymptotically stable only if

π

0 < U <

where ωl is the unique solution of U = ωsinω in the interval (0, — ) From Eq (29), the controller gains A and B can be set FIG 7 illustrates a stable region with respect to U and V according to the embodiment of the present invention

(4) Principal Root and Asymptotic Decay Rate

In this part, there will be described determination of the rate at which the stable closed-loop system approaches a steady state Any solution to the normalized closed- loop equation (27) can be represented by series

e(ξ) = ∑ PΛξY^n£ (30) fi=l

where pn(ξ) is a suitable polynomial and zn, Vn are the roots of the corresponding characteristic equation (28) Consider the principal root, denoted by z , which is the root having the largest real part Let z^* = -α + jβ, α > 0, β > 0 It follows from Eq (30) that

e(ξ) « C ' (31)

where C is a constant depending on the initial conditions of Eq (27) and x(ξ)«y(ξ) χ(ξ) indicates that — — - asymptotically approaches 1. Also from Eq (30),

£) < ce^~ for large ξ (32)

where ||-|| denotes the Euclidean norm and c is a constant depending on the initial conditions of Eq (27) In terms of the original variable t(=rξ), Eq (32) can be rewritten as,

e(t) < ce^{~ r} for large t (33)

Note that — is the asymptotic decay rate at which the original system tends to τ a the equilibrium point Hence the inverse of it, — , is the time constant of the original τ closed-loop system, i e , the time it takes for a small perturbation around the equilibrium point to decrease by a factor of e ^l Similarly, α and of are the asymptotic decay rate and the time constant of the normalized system

FIG 8 illustrates the asymptotic decay rate α as a function of U and V With reference to FIG 8, the asymptotic decay rate α is a concave function with respect to U and V with its maximum being approximately 0 3 at (U, V) = (0, 6, 0, 1) The contour line at α corresponds to the boundary of the stable region shown in FIG 7 Once α is determined for a given (U, V) pair, β can readily be determined by substituting z = -α + jβ in the characteristic equation (28) and equating the real and imaginary parts

(5) Discrete- Time ER Algorithm and |Q| Estimation

A continuous-time model of the MAX-MIN flow control problem has been dealt with so far In reality, however, feedback information is relayed in RM cells, and thus available not in continuous time, but rather in a sampled form Moreover, the feedback information is not periodic because the RM cells themselves have to complete the link bandwidth along the round trip path so that the inter-arrival time of the feedback RM cells are time-varying

The ER allocation algorithm expressed in Eq (8) and Eq (9) can be implemented at a switch in discrete time as follows Update the common ER periodically with a period T by

r[k + l] = [r[k] - ^(q[k]- q[k - \]) - ^(q[k] - qy] A,B > 0 (34),

where q[k] denotes the low-pass filtered queue length, I e , an average queue length

Particularly, a periodic averaging filter is used in order to obtain q[k] in the embodiment

~ 1 rkT of the present invention such that q[k] = — J q( )d

It is to be noted that the ER update equation (34) corresponds to Eq (9) as T -> oo In contrast to the periodic computation r[k+l] of the common part of ER, r(t), per-

VC ER allocation is performed periodically upon arrival of the corresponding RM cells in either a forward direction or a backward direction That is, upon arrival of a VCi's RM cell at time t, the switch computes ri(t) = r(t) + mi (35),

and writes this result on that RM cell In Eq (35), r(t) denotes the latest value, r[k+l], of the common ER, r[k], which is being updated periodically in the background according to the embodiment of the present invention The value of mi is available from the RM cell that has arrived or the per-VC MCR table being maintained in the switch and updated upon arrival or departure of an RM cell If it is determined that the value of mi is to be taken from the RM cell that has arrived, the per-VC MCR table and the access to it are no longer necessary Therefore, the only per-VC operation required in the ER allocation algorithm of the present invention is the addition in Eq (35)

Meanwhile, many schemes have been proposed for estimating or tracking the number of locally bottlenecked VCs Some of those schemes can be encountered in M K Wong and F Bonomi, "A Novel Explicit Rate Congestion Control Algorithm", it Proc IEEE GLOBECOM'98, vol 4, pp 2432-2439, 1998, L Kalampoukas, A Varma and K

K Ramarkrishnan, "An Efficient Rate Allocation Algorithm for ATM Networks Providing MAX-MIN Fairness", Technical Report UCSC-CRL-95-29, Computer Engineering Dept , University of California, Santa Cruz, June 1995, A Charny, K K Ramakrishnam and A Lauck, "Time Scale Analysis and Scalability Issue for Explicit Rate Allocation in ATM Networks", IEEE/ACM Trans On Networking, vol 4, pp 569-581,

1996, and R Jain et al , "ERICA Switch Algorithm A Complete Description", ATM Forum/96- 1172, 1996 Each of the above schemes varies in the degree of implementation complexity The basic idea of Su et al 's algorithm (in C F Su, G de Veciana, and J Warlrand, "Explicit Rate Flow Control for ABR Services in ATM Networks", preprint, 1997), which estimates the number of "on" sources sharing a link, is attractive since it does not require per-VC accounting

The above algorithm is modified to estimate the number of locally bottlenecked VCs without doing per-VC accounting in the embodiment of the present invention

Suppose that a j^th RM cell arrives at a switch at a switch time t¹ According to the ABR specification, if the j^th RM cell happens to be an RM cell of VCi, it carries the value ai(t^J-τi^f) in the CCR field and the value mi in the MCR field The switch monitors the RM cell arrivals in a synchronous fashion over fixed length intervals of W seconds For an 1^th interval, the number of locally bottlenecked VCs can be approximated by bL . . . (36),

where 1{ } is the indicator function, CCR(t^J) and MCR(t^J) respectively denote the value in the CCR field and the value of the MCR field of the j^th RM cell, and r(t") is the latest value of the common ER at time t^J Upon arrival of the j' RM cell, if the CCR subtracted by the MCR is greater than or equal to the latest value of the common ER at the switch, the VC to which the j^Λ RM cell belongs is counted as a locally bottlenecked VC Otherwise, it is treated as a remotely bottlenecked VC δ is the margin to avoid the underestimation of the number of locally bottlenecked VCs particularly near the steady state As the system approaches the steady state, the CCR of a locally bottlenecked VC stays around the sum of the MCR and the common ER Thus, without the margin δ the

VC could be counted wrongly as a remotely bottlenecked VC even for a small perturbation in the CCR By having this margin, however, this type of underestimation can effectively be avoided Through simulations, it is found out that δ = 0 9 is the recommended choice Also, it is to be noted that the value of the indicator function is normalized by the expected number of RM cell arrivals of the VC within W seconds,

W CCR , so that the summation of these values over a W-second interval gives a

NRM+ V correct estimate of the number of locally bottlenecked VCs Based on this estimate for each interval, the recursive estimate is computed at the end of every interval by

\θ\ lαvg ((I+ \)W)

(37) and

\)W) = int[ <2 l avg ((/ + 1)0 ] (38),

where λ is an averaging factor, int[a] denotes the smallest positive integer greater than or equal to a, and the saturation function sata[b] is defined as

la, b > a s&t_a[b] = \ _L , . (39) ^{a l J} [b, otherwise ^v

Note that 1 <

< \N\ for all t The actual number of locally bottlenecked VCs can be zero for some network loading but the value of its estimate is lower-bound intentionally to avoid the division by near-zero value Now the question is how to choose the interval W and the averaging factor λ As the number of VCs sharing a link increases or the available bandwidth decreased for a given W, the inter-arrival time of RM cells of a VC increases so that the switch begins to fluctuate largely To solve this problem, W may be adjusted according to the changes of the number of VCs sharing the link and the available bandwidth but this is not easy to implement Instead, a large λ close to 1 can be selected in the hope that the periodic averaging operation in Eq (37) will effectively filter out the variability of the |Q|I Through simulations, it is found that λ = 0 98 yields stable and effective estimation of |Q| for a wide range of number of VCs sharing a link and the available bandwidth, irrespective of the choice of W

We further improve the |Q| estimation algorithm for the stability purpose Suppose that the controller gains A and B are selected so that

U V A = —, B = — (40) τ τ

and the (U, V) pair resides inside the stable region depicted in FIG 7 This selection is made in such a way that the |Q| estimation is accurate enough that

« \Q\ for all times

However, since

≠ \Q\ in general, the actual or effective values of U and V, say, U' and V, governing the normalized closed-loop equation (27) are given by

by substituting Eq (40) in Eq (26) If V) resides

somewhere on the straight line connecting the point (U, V) and the origin (0, 0) and thus

O remains inside the stable region In contrast, as —TΓ increases beyond 1, the point (U\

V) moves upward along the straight line including the origin (0, 0) and the point (U, V) and eventually gets out of the stable region In short, overestimation of |Q| is tolerable since it does not affect the stability of the system but it only changes the asymptotic decay rate whereas underestimation of |Q| should be avoided since it can make the system unstable This is why the margin δ is necessary in Eq (36) However, δ does not resolve the whole situations, particularly the situation that a new VC joins with a small initial cell rate (ICR) If a new VC joins with a small ICR, the VC is very likely to be counted as a remotely bottlenecked VC during the transient period since CCR(t^J)-MCR(t^J) is very likely to be less than δ-r(t') in Eq 36 since the margin δ is small enough In this state, if the new VC is actually a remotely bottlenecked VC, it is not a problem but if the new VC is actually a locally bottlenecked VC, it results in an initial underestimation of |Q|, which could make the system diverge To resolve this problem, the |Q| estimation algorithm is improved as follows in the embodiment of the present invention

Upon arrival of a new VC at time t, the estimate is updated on purpose by

|Q|avg(t) = sat|N| [|Q|avg(t)+l] (42)

and

as if the VC is a locally bottlenecked VC By doing so, the initial underestimation of |Q| is avoided so that

converges to |Q| from Eq (42) and Eq (43), which is a highly desirable property for the system stability as stated before The performance of the |Q| estimation algorithm proposed in conjunction with the ER allocation algorithm according to the embodiment of the present invention will be verified through simulations in "(6) Simulation Results"

Next, it will be considered the case where RR marking is applied in conjunction with ER marking The ER marking plays the major role to achieve the MAX-MIN fairness with MCR guarantee in an asymptotically stable manner whereas the RR marking plays a supplementary role to limit the transient overshooting of the queue length and hence minimize the transient cell loss

FIG 9 illustrates the suggested design of the queue length thresholds for the ER plus RR marking In FIG 9, qT is the target queue length, qLT is the lowest queue length threshold for the ABR queue, and qHT is the highest queue length threshold for the ABR queue qT is related with ER marking and qLT and qHT are related with RR marking qLT is a threshold from which the NI bit of a NI field is set and qHT is one from which the CI bit of a CI field If the queue length is greater than qHT, the switch sets the CI bit of the backward RM cell to 1 to indicate its congestion If the queue length is between qHT and qLT, the switch sets the NI bit of the backward RM cell to 1 to prevent the source from increasing its bandwidth By placing qT far below qLT, however, the RR marking is hardly activated since the queue length would stay in the neighborhood of qT by the control of the ER marking unless queue surge due to abrupt changes in the network loading occur Once the queue length overshoots qHT by some loading changes, the linear increase and exponential decrease mode is triggered and remains active until the ER marking re-takes the control of the queue length

In "(6) Simulation Results", it will be shown through simulations that the RR marking effectively limits the worst-case queue length in the transient period and the ER marking indeed takes the control back so that it recovers the queue length from the oscillatory mode to the asymptotically stable mode

(6) Simulation Results

In this part, simulation results will be presented to verify the above-described analysis and demonstrate the excellent performance of the ER allocation algorithm according to the embodiment of the present invention The simulation model used is developed on the NIST ATM simulator platform

Two different network topologies were considered the peer-to-peer configuration and the parking lot configuration, which are fairly standard The recommended values for the design parameters in the switch algorithm suggested in the present invention are summarized in Table 1 to be used in the following simulation studies

(Table 1)

(τmax = max{τι, I e N}, Δ = one cell transmission time)

First the peer-to-peer configuration will be considered, shown in FIG 10 where 20 ABR VCs having an identical path are contained and the capacity of the all links is set equally at 600Mbps To represent a WAN environment, the distance between each of the sources from si to s20 and a first switch SW1 in the path is set at 1,000km If it is assumed that a signal propagation speed is 2 0 x 10⁵Km/sec and that the queuing time are negligible, τmax is roughly 1 Omsecs VC models used in this simulation configuration are listed in Table 2 and all the sources are assumed to be persistent Note that PCR, MCR, ICR, arrival time, and departure time are measured in units of Mbps

(Table 2)

It is to be noted that the PCR value, the MCR value, the ICR value, and the arrival and departure times of the VCs are varied in order to investigate the impacts of the PCR-constrained sources, the difference in MCR and ICR, and the call activities on the network performance For the purpose of comparison, the theoretical fair rates satisfying the MAX-MIN fairness with MCR guarantee are calculated and the results are listed in Table 2 Referring to Table 2 and FIG 10, it is observed that the fair rate of each VC varies in time according to the arrivals and departures of the other VCs and that the sources sl l to slO are bottlenecked at the switch SW1, and the sources sl l to s20 are bottlenecked at the access by its PCR constraint For example, the ABR sources si to s4 should transmit data at 36 1Mbps in 0-1 sec, 30Mbps in 2-3 sec, and 32 5Mbps in 3~oo sec to be Max-MTN fair In accordance with the present invention, 32 data cells are generated between two adjacent forward RM cells, l e , NRM = 32

FIGs 11A to 11D illustrate simulation results in the peer-to-peer configuration with ER marking only and without no VBR background traffic FIGs 12A to 12D illustrate simulation results in the peer-to-peer configuration with ER marking and VBR background traffic

FIGs 11 A and 11B illustrate the source transmission rates aι(t) of VCs, respectively with PCR=150Mbps and PCR=25Mbps From FIGs 11 A and 11B, it is observed that the actual source transmission rates perfectly agree with the theoretical fair rates given in Table 2 The transmission rates of the sources si to s4 and si 0 are equal to the common ER, r(t) computed by the switch SW1 since their MCR is 0Mbps, the transmission rates of the sources s5 to s9 are greater than the common ER, r(t) since their MCR is 10Mbps, and the sources sl l to s20 are PCR-constrained irrespective of their MCR values The initial transient behavior is due to the initial condition that r(0) = 0 at both switches SW1 and SW2 That is, it takes a time for the common ER value to ramp up to the operating point, which is, however, a phenomenon that hardly occurs during the ordinary operation FIG 1 IC illustrates a queue length at the bottleneck node SW1 The joins of the source s20 at lsec and the source slO at 2sec result in the surge of the queue length and the leave of the source s20 at 3 sec results in the sudden drop of the queue length The ER allocation algorithm suggested in the present invention, however, rapidly re-stabilizes the queue length at qT = 250 cells FIG 11D illustrates the estimate of the number of locally bottlenecked VCs, |Q|avg(t), at the switch SW1 By casting the estimate to the smallest positive integer greater than or equal to it by Eq (38), the integer estimate of |Q| can finally be obtained in the way described below Here, symbol "[x" indicates "x<" and symbol "y]" indicates "y>" The integer estimate is 18 in [0, l]sec, 9

[1, 2] sec, and 18 at [2, oo]sec, as shown in FIG 11D It is to be noted that except the initial time interval [0, l]sec this integer estimate perfectly agrees with the true value of IQI which is shown to be 9 in [0, 2]sec and 10 in [2, oo]sec in Table 2 The difference in the interval [0, l]sec is due to the intentional increase of the estimate |Q|avg(t) by 1 upon every arrival of a new VC as in Eq (42) In the given simulation scenario, 18 VCs arrive at Osec, which is why the estimate |Q|avg(t) has the value around 18 initially

With VBR background traffic the results are virtually unchanged as shown in FIGs 12A to 12D That is, the macroscopic (time-averaged) behavior is identical to that of the case with no VBR traffic disturbance The VBR background traffic was generated by a deterministic on/off source with the peak rate and the lengths of on and off periods are, respectively, 10Mbps, 20τmaxsecs, and 20τmaxsecs It is noted from FIGs 12A and 12B that the transmission rate of the sources (except the PCR-constrained sources) almost perfectly follows the on/off pattern of the VBR background traffic The on/off behavior of the VBR background traffic causes the repeated surges and drops of the ABR queue length The ER allocation algorithm according to the embodiment of the present invention, however, rapidly recovers the queue length to the target value as shown in FIG 12C

Next the parking lot configuration shown in FIG 13 will be considered to study the case of multiple bottleneck nodes and VCs with different RTDs 16 ABR VCs with different source locations are contained and the capacity of the links is set equally at 600Mbps except that the link between the switch SW3 and the switch SW4 and the link between the switch SW4 and the switch SW5 are 300Mbps The VC models used in this simulation configuration are summarized in Table 3 and all the sources are assumed to be persistent

(Table 3)

For comparison purposes, the theoretical fair rates satisfying the MAX-MIN fairness with MCR guarantee was computed for the given simulation scenario and the results are included in Table 3 To further clarify the scenario, the theoretical bottleneck location of each VC, which is the location at which each individual fair share is determined, is included in the table FIGs 14A to 14F illustrate the simulation results with ER marking only and no VBR background traffic FIGs 14A and 14B illustrate the source transmission rates ai(t) of VCs, respectively with PCR=150Mbps and PCR=25Mbps FIG 14C illustrates a queue length at the switch SW3, FIG 14D a queue length at the switch SW4, FIG 14E illustrates the estimate of the number of locally bottlenecked VCs, |Q|avg(t), at the switch SW3, and FIG 14F illustrates the estimate of the number of locally bottlenecked VCs, |Q|avg(t), at the switch SW4 It is observed from FIGs 14A and 14B that the actual source transmission rates in the steady state perfectly agree with the theoretical fair rates given in Table 3, irrespective of their RTDs and the bottleneck locations The initial transient behavior is due to the initial condition that r(0) = 0 at all the switches, which is again a phenomenon that hardly occurs during the ordinary operation In the given simulation scenario, there are two congested nodes, SW3 and SW4 As expected, the queue length at the congested nodes converges to the target value, 250 cells, which is shown in FIGs 14C and 14D It is observed that in the steady state the estimates stay around 9 and 2 at the switches Sw3 and SW4, respectively, which agrees with the data in Table 3

Finally, application of RR marking in conjunction with ER marking will be studied The queue length thresholds for the RR marking are set at qLT=750 cells and qHT = 1,000 cells on conjunction with qT = 250 cells This choice of the queue length thresholds was made in the expectation that in the normal operation condition the ER marking plays the major role to guarantee the asymptotically stable MAX-MIN flow control whereas the RR marking plays a supplementary role to limit the transient overshooting of the queue length if any The same parking lot configuration scenario with VBR background traffic is used in the embodiment of the present invention Again the VBR background traffic was generated by a deterministic on/off source with the peak rate and the lengths of on and off periods are, respectively, 10Mbps, 20τmaxsecs, and

20τmaxsecs The results are shown in FIGs 15A to 15F FIGs 15A to 15F illustrate simulation results in the parking lot configuration with ER marking and VCR background traffic FIGs 15A and 15B illustrate the source transmission rates ai(t) of VCs, respectively with PCR=150Mbps and PCR=25Mbps FIG FIG 15C illustrate a queue length at the switch SW3 and FIG 15D illustrates a queue length at the switch SW4

FIG 15D illustrates the estimate of the number of locally bottlenecked VCs, |Q|avg(t), at the switch SW3 and FIG 15E illustrates the estimate of the number of locally bottlenecked VCs, |Q|avg(t), at the switch SW4 Compared to the previous case shown in FIGs 14A to 14F, the transient overshooting of the ABR queue length of the switch SW3 was substantially reduced with its maximum decreasing from 6,000 cells to 3,200 cells Obviously, this gain comes at the cost of temporary oscillation of the queue length and the source transmission rates as observed in FIGs 15 A, 15B, and 15C It is also observed that the ER allocation algorithm according to the embodiment of the present invention effectively takes the control back and hence recovers the queue length behavior from the oscillatory mode to the asymptotic mode

As described above, the ER allocation algorithm according to the preferred embodiment of the present invention is advantageous in that (1) maximal link utilization and minimal cell loss are guaranteed regardless of RTDs in an ABR closed loop; (2) ABR queue size requirements are minimized by ensuring asymptotical stabilization of ABR queues (3) the MAX-MIN fairness based on the ATM forum standards is guaranteed by ensuring a fair share of an available bandwidth to each user, (4) communication network environmental change is fast reacted to such as changes in the number of ABR users and the ABR bandwidth, (5) all functions including EFCI, RR, and ER markings are provided as specified in the ATM forum traffic management specification, (6) high utilization, low cell loss, and the MAX-MIN fair rate allocation are achieved through existence of an asymptotically stable operating point, (7) high responsiveness and transient control performance to network loading changes is achieved at multiple time scales, I e , at the cell level rate changes of VBR and ABR VCs and at the cell level arrivals and departures of VBR and ABR VCs, (8) the number of operations required to compute the algorithm is minimized, and (9) low and scalable degree of implementation complexity is achieved by virtually removing per-VC operations including per-VC queuing, per-VC accounting, and per-VC table access

While the invention has been shown and described with reference to a certain preferred embodiment thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and the scope of the invention as defined by the appended claims

Claims

WHAT IS CLAIMED IS:

1 A fair flow controlling method in a packet switching network of the type having a plurality of nodes, each node being connected to a plurality of sources which transmit/receive data, an explicit rate (ER) engine, and a data queue with a current queue length and a target queue length for storing the data received from the sources, the method comprising the steps of generating a first signal and a second signal, periodically updating an explicit rate (ER) by the ER engine in response to the first signal at one of the sources, upon arrival of a forward cell at one of the sources, estimating a number of locally bottlenecked virtual circuits (|Q|) based on the updated explicit rate (ER) from the ER engine, in response to the second signal, determining a new ER based on the difference between a current queue length and a target queue length of the node, the estimated number of the locally bottlenecked virtual circuits (|Q|), and a derivative of the current queue length, and upon detecting an arrival of a backward cell at one of the sources, writing the new ER in an ER field of the detected backward cell

2 The method of claim 1, further comprising the step of retrieving a minimum cell rate (MCR) from the backward cell upon detecting the arrival of the backward cell at one of the sources

3 The method of claim 1, wherein the step of writing the new ER in the

ER field of the backward cell comprises the steps of comparing the ER of the backward cell with a sum of the newly determined ER and the MCR of the backward cell, and writing the new ER in the ER field of the backward cell if the sum less than the ER retrieved from the backward cell

4 The method of claim 1, wherein the number of the locally bottled necked VCs is estimated by comparing a difference between a Current Cell Rate (CCR) and a minimum cell rate (MCR) of the forward cell and the updated ER received from the ER engine

5 The method of claim 4, wherein the number of the locally bottlenecked VCs is estimated if a difference between the CCR and the MCR is greater than or equal to the updated ER

6 The method of claim 1, wherein the forward and backward cells comprised of a resource management cell

7 The method of claim 1, wherein the updated ER by the ER engine is determined at a predetermined time interval

8 The method of claim 1, further comprising the steps of determining whether congestion occurs in one of the sources upon detecting a data cell, and transmitting a first congestion signal or a second congestion signal if the congestion is detected, wherein the second congestion signal indicates substantially more congested state

9 The method of claim 8, further comprising the step of controlling a relative rate (RR) of the backward cell, wherein the RR is defined by the first signal and the second signal

10 The method of claim 8, wherein the first signal is transmitted if the difference between the current queue length and the target queue length of the node exceeds a low predetermined threshold value, and wherein the second signal is transmitted if the difference between the current queue length and the target queue length of the node exceeds a high predetermined threshold value

11 The method of claim 1 wherein the number of the bottlenecked virtual circuits (IQI) is determined if a difference between a Current Cell Rate (CCR) and a Minimum Cell Rate (MCR) retrieved from the forward cell is less than the updated ER transmitted from the ER engine

12 The method of claim 3, wherein the new ER is calculated by subtracting (((average queue length - previous average queue length) x first gain) - estimated |Q|) + ((average queue length - target queue length) x ((second gain x second signal period) - calculated |Q|)) from a previous ER in response to the second signal