ZA200401870B - Distributed transmission of information in a connectionless packet-oriented communication network - Google Patents

Description

LL + @ ‘e.

Description

Transmisssion of information in a packet -oriented communication network

The subjexct of the application is for example also applicable to the field of standardized networks for the reliable transport of digitally coded information for data, voice, audio/video and other services and applications in compliance with corresporiding service- specific or application-specific quality-of-service requirements and exterads to interactive real-time communication. Patent applications DE 10146349.9, DE 10161508.6, DE 1016154&.9, DE 10161547. 7 are also applicable to this same field. Their disclosure is made bsy reference to the content of the present desscription.

In the pa st two main types of communication networks Have evolved for trans mitting information embedded in traffic streams: packet- oriented data networks and line-based voice networks. Their different quality-of-service (QoS) requirements are ome aspect in which they differ from each other. "Quality ef Service” is defined differently depending on context and is therefore evaluated using different metrics. Kn own examples of metrics for measuring quality of service are the ma ximum number of information elements that can be transmitted (bandw idth), the number of information elements transmitted, the number of information elements not transmitted (loss rate), the — possibly mean - time delay during transmission ((transmission) delay), the - possibly mean - deviation from the otherwise standard —interval between twso information transmissions (delay jitter, imterarrival jitter) or the number of information elements g . . 2 not permitted to be transmitted (blocking rate).

In multimedia networks services are alsso known as multimedia applications. A multimedia network is wmsed here to describe a network in which a plurality of different services is provided. In a narrower sense it refers in particula.r to a broadband, service- integrated network (B-ISDN = Broadband Integrated Services Digital

Network) in which the traffic streams resulting from use of the services can be transmitted by means of a standard, preferably packet-oriented transport mechanism. Th.e term multimedia application thereby covers both service s and normal telephony (also referred to as Voice over IP (VoIP) in packet-oriented IP networks, as well as services such as fax, teleph.one conference, video conference, Video on Demand (VoD) and s o on.

Line-based (voice) networks are designe d to transmit traffic streams in which continuously streaming (voice) information is «nbedded. In specialist circles these axe also referred to as calls or sessions. Information is generally transmitted here with a high —uality of service and security. For example for voice a minimum - e.g. < 200 ms - delay is important without delay jitter, as voice

FTequires a continuous information flow for playback in the weceiving device. Information loss can &therefore not be compensated for by retransmission of information nok transmitted and generally results in the receiving device in an acoustically perceptible clicking. In specialist circles voice tmansmission is also generally referred to as a realtime (transmission) service.

A low blocking rate is achieved for example by appropriate

Aimensioning and planning of the voice retworks. A small and

@® WO 03/026341 PCT/DE02/03584 largely constant delay or delay jitter is generally also achieved in the case of joint transmission of a plurality of traffic streams via a shared channel by using a static time division multiplex also referred to as TDM. Here the traffic streams are segmented in the transmitter into homogenous units of fi xed length - also referred to as time slots - and transmitted tempoorally interleaved in each other. Assignment of the time slots to the respective traffic streams is indicated by their position within the channel. After joint transmission the time slots can koe assigned to their associated traffic streams in the recei ver and where necessary can also be reassembled into the original t raffic streams. As a result the transmission capacity of the traffi c streams is essentially not subject to any fluctuations during line -based transmission but is fixed at a predefined value (e.g. 64 kb ps in modern ISDN telephone networks) .

Packet-oriented (data) networks are des igned to transmit traffic streams configured as packet streams, also referred to in specialist circles as data packet strearns. It is generally not necessary to guarantee a high quality of service here. For example in the case of an email it is not necessary to have a minimum delay without delay jitter, as an email does rot have to be played back in realtime at the receiver. More import-ant here is that the email should be transmitted without error. Information loss is therefore generally compensated for by retransmission of information that was not transmitted or was transmitted incorrectly. The delay of an email therefore varies as a function of the frequency of retransmission. Delay jitter therefore amlso tends to be high. In specialist circles the transmission of @ata is therefore also referred to as a non-realtime service.

N

There is essentially no blocking rate in packet-oriented data networks. In principle all pack ets in all traffic streams are always transmitted. The traffic streams are however transmitted even when there is only moderat e€ loading of a data network with significantly fluctuating time -delays as the individual packets are generally transmitted in the se<quence of their network access, i.e. the time delays increase, the mere packets have to be transmitted by a data network. Joint transm ission of a plurality of traffic streams via a shared channel is generally achieved by using a statistical (time division) mul-tiplex. Here the packets in the traffic streams in the transmitter are transmitted interleaved in time according to statistical rwmles. The rules could for example specify that the packets are to be transmitted in the sequence of their arrival (best effort). If a plurality of packets arrives at the same time, one is transmitted while the remainder are temporarily buffered, resulting in an increase in delay jitter. If more packets than can be buffered arrive at the same time, the surplus packets are discarded. Assignment of the packets to the respective traffic streams is irdicated by assignment information in the packet overhead (comprising a header and/or trailer). After joint transmission therefore the packets can be assigned to their associated traffic streams in tlie receiver. The transmission capacity of the traffic streams is essentially not subject to limitations during packet-orient-ed transmission but can in principle (in the context of the capacity of the shared channel) have a different value at any time.

In the course of the convergence= of line-based voice and packet- oriented data networks, voice tr-ansmission services and in future also faster broadband services such as for example the transmission of moving image information (VoD», video conference) will be provided in service-integrated packet-oriented (multimedia)

ES networks - also referred to as voice/data networks, i.e. realtime services until now generally transmitted #in a line-based manner are now transmitted in packet streams in a corvergent voice/data neetwork. These are also referred to as realtime packet streams.

Thxeis gives rise to the problem that a high quality of service and sescurity are necessary for the packet-oriented provision of a re-altime service to ensure that this is comparable in quality to a li ne-based transmission, while modern (packet-oriented) data ne tworks and in particular the internet have no adequate mechanisms to guarantee a high quality of service.

Qu ality of service requirements in servicer-integrated, packet- or iented networks generally apply to all rmetwork types. They are in dependent of the specific configuration of the packet or ientation. The packets can therefore be configured as internet,

X. 25 or frame relay packets and also as ATM cells. They are sownetimes also referred to as messages, pa.rticularly when a message is transmitted in a packet. Data packet st reams and realtime packet streams are hereby exemplary embodiments of traffic streams transmitted in communication networks. Tra ffic streams are also referred to as connections, even in packet -oriented networks, in whDdch connectionless transmission technolo gy is deployed. For example information is transmitted with TC P/IP using what are known as flows, by means of which, despite the c onnectionless nature of

IP_, transmitter and receiver (e.g. web ser—~ver and browser) are cornected at a logically abstract level, i.e. in a logically abstract way flows also represent connecti.ons. It is only essential forr a connection that a connection setup takes place before transmission, during which process a context is created which coritinues to exist at least during transmisssion. An explicit clesardown of the connection can take place after transmission.

Implicit mechanisms such as for example tirmeout of the connection after a specified transmission-free period are however also possible.

The best known data network at present is the internet. The internet is conceived as ar open (long-range) data network with open interfaces to connect (usually local and regional) data : networks of different manufacturers. The main focus to date has therefore been on the provision of a manufacturer-independent transport platform. Adequate mechanisms for guaranteeing quality of service play a secondary role and therefore barely exist.

The convergence of telecommwunication (also known as voice networks) and the conventional data wrorld (also known as data networks) into

IP (internet protocol) baseed networks and services is a difficult task in respect of IP technology, as this is designed as a packet- oriented data network primarily for “best effort” transmission and at best provides for compli ance with rather vaguely formulated service level agreements (SLA), while in the case of telecommunication very strimgent requirements relating to QoS, reliability, availability amd security of network and services play a major role. The internet world responds to this task with a plurality of increasingly complex and expensive solutions but has not as yet found a total solution that is also manageable and workable from an economic point of view.

The QoS requirements of a service or an application in respect of a network can be defined using different criteria, of which some examples are given below: - the throughput characteristics of the digitally coded information, i.e. the rmecessary bandwidth or bandwidth characteristics (fixed bandwidth, variable bandwidth (e.g. with mean value, peak walue, ‘burstiness’ factor or other characterizing parameters]) and susceptibility to information losses, - the delay characteristics, i.e. the effects of an absolute delay (transit time from information source to information sink) and susceptibility to runtime fluctuations or delay jitter (of course delay jitter can be converted to absolute delay by buffering but this is usually very complex), - the necessary or unnece ssary temporal consistency or time invariance of the transmitted information, i.e. whether the information units have to be delivered in exactly the same sequence in which they arrived or not (in some cases the compatibility or incompatibility of higher service and application layers must also be taken into account).

The consequences of different QoS requirements can be clarified using two examples:

I. Unidirectional audio/video applications (e.g. streaming video) require realtime presentation at the receiver but ir most cases it is imma terial whether the absolute delay is 1/100, 1 or 5 seconds, as long as there is continuity aftew the start of playback . Such delay tolerance could for example be used to compensate for information losses using repeats, thereby improving the quality. Alternatively transmission could also take place with redundancy {higher bandwidth) to compensate for possible data losses.

II. Interactive, i.e. bidirectional realtime communication (voice, video, etc.) loetween people must take into account the response capability and typical communication and dialog behaviors of people. Here the absolute delay (and therefore of course also tlie delay jitter) must be limited to a few hundred milliseconds (e.g. 200 ms). On the other hand in some instances someawhat higher loss rates can be tolerated, as t=he capacity of tke human brain to “smooth out irregularities” in speech and visual perception is very well-developed and alertness to rninor defects is somewhat reduced in dialog.

Realtime dialogs between machines are more complex, however.

In this case At may be that attention must be focused on the completeness of the information and on short delays close to the physical Mimit due to geographical distance (transit time approx. 5 ms per 1000 km distance).

If the QoS specific ations are defined and if a network still has reserves in one of these areas, it can deploy these to compensat e for deficits in ano ther area. Such compensation can be clarified using two examples:

I. If an appli cation tolerates relatively high information losses, the delay jitter can be reduced by discarding information units which have been subjected to a high le vel of delay. Conversely larger delay jitter can of course a lso be deployed to achieve lower losses, which however resul ts in large bu ffers.

IT. If the maximum for delay jitter is below the minimum tim e interval of the incoming information units (known as a ‘fast netwozxxk’), there are no problems with the temporal consistency of the transmitted information. If measures are provided to restore this temporal consistency, relatively o WO 03/026341 PCT/DE02/03584 large delay jitter can be tolerated as long &s the framework of the absolute permissible delay Zs not exceeded.

As well as QoS the general availability of services is also an important parameter that depends to a large degree orm the network and its characteristics. In the event of an error, e. g. in the case of failure of individual network components or connec=ting lines, is a backup path awailable and how quickly can it be brosught into use?

Do interruptions occur that the user can identify and. how long do these last? Does the network operator or even the use r have to intervene in any” way to restore the service in some i nstances? The reliability of the network in itself and the way in which it can help to bypass errors and where necessary restore the applications is of great sigraxificance here.

A standard network must therefore be considered subje«ct to qualification by initial conditions as proposed here =nd of course it should also be achieved in the most efficient manner possible, i.e. at the lowe st possible cost and in an economically advantageous manner.

The known network technologies satisfy the above specification partially at best. 1) The simplest approach is the tried and tested techraology of circuit switching, with which a dedicated connection (in the bidirectional instance or with multiple relations where necessary also two or more connections) (sometimes also referred to casually as a path) with & permanently assigned and absolutely reserved bandwidth is swittched for every communication relation.. Such connections are either configured explicitly as indivi dual physical lines (e.g. copper wires) or as (virtual) channels in what are known as transmission or switching systems, which allow multiple utilization of phiysical lines. A mix with differently implemented

[ W/O 03/026341 PCT/DE02/03584

1 inks is also possible.

The possible data throughput of such a ceonnection is determined by its own or its as signed bandwidth, the transport delay time is made up of the propag ation delay, i.e. the ddstance-dependent transit time on the line, and the switching dealays, i.e. the inherent processing times re sulting during switching of the digitally coded information (data) in the network nodes (switches). Switching here means transf-erring information (data) from a defined incoming line/channel te an outgoing l-ne/channel specified when the connection is being set up.

Both dexlay components can generally (i.e. when the systems are operating without interference) be assumed to be constamt for the period of a communication relation (with through-connected path or existing connection). When there is no interference therefore the same qurasi-optimum QoS is predefined and achievable for all applications (mo information losses, constant, generally relatively short, dezlay, no transpositions). However for this the connection must be permanently switched (and reserved) for the duration of the communication relation, even if the application only uses it very ire frequently (e.g. only sporadically). Reliabi lity/availability can ber improved by switching as quickly as possible to a previously provided alternative connection in the event of an error (double ca pacity required) or switching the backup cormnection immediately (d elay and expense, particularly when a plural ity of connections is af fected at the same time by one failure). 2) Packet switching technology aims at better utilization of ressources (bandwidth) by flexible sharing of 1 ines and (where neccessary virtual) channels or switching and t ransmission media by a plurality of communication relations.

Known, modern representatives are for example the connection -oriented ATM technology with fixed-length packets (also refeerred to as cells)

C WO 03/026341 PCT/DE02/03584 and the connectionless IP technologsy with variable length packets.

A) ATM technology is also promoted at the ITU-T under this name and with the objective of broadband ISDN (B-ISDN). ATM has mechanisms to provide a broad spectrum of service classes with defined and guaranteed QoS (at the statistical mean), even with very scant resourcess (available bandwidths). The resulting systems and networks are therefore very complex and expensive. Dimensioning and opeeration require highly qualified specialist personnel . ATM operates in a connection- oriented manner, with a network of ‘virtual’ paths and channels, assigned to each other in a hierarchical manner.

For a plurality of different s ervice classes bandwidths can be reserved in a connection-sp ecific manner and also ‘guaranteed’ based on the traf fic statistics used as a basis.

Different queuing and schedulimg mechanisms are used for this and these can be set in every mode for each path and channel (connection) by means of appropriate parameters. Fine- granular dimensioning and connection acceptance requirements can be used to limit information losses and the variable parts switching delays (these are essentially determined by queuing) based on statistical mules. Owing to the connection- oriented mode of operation, transposition of information units is unlikely during interference-free operation. As a result of the connection orientcation all inherent mechanisms have to be executed again durirag error handling. The basic concepts are therefore often very similar to those of circuit switching technology.

B) IP technology is more of a pragmatic approach that has become established in the data world due to its simple basic mechanisms. It has made massive progress in recent years so that the capacity (data through put, control efficiency) of systems and networks based on it is comparable to that of systems based on ATM technology . The success of IP technology

® WO 03/026341 PCT/DE02/03584 is significantly due to the fact that a large part of the services and applications are already based on packet- oriented internet protocols (IP) in the terminal. It is currently predicted that the growth in IP-based services will also be significantly greater in the future than in other technologies, so an extensive migration of all services to transport via IP-based networks seems probable. Unlike ATM networks IP networks operate in a connectionless manner and only provide a ‘best effort’ =ervice, with which it is difficult to predict and imposssible to guarantee an achievable QoS even with generously dimensioned networks.

C) The following solutions were also known to date: a) Using an ATM network as a core network. Edge devices transfer the IP data streams to ATM connections of appropriate service classes and transport takes place in corresponding connections in t he ATM network. Problems here are scalability, complexity and setting up and operating costs (see ATM technology abov-e). This solution is of more assistance in the core. The sa me disadvantages apply to (additional) use in the access . The following solution is an alternative in the access. b) Using a signaling protocol and setting up connections with reserved bandwidths via the IP network (integrated services -

IntServ, RSVP). This solution is feasible in principle both end-to-end (E2E), i.e. from te xrminal to terminal, and on subsections. It can be used fox each communication flow or (in the core} also for aggregated communication flows. It is however elaborate, expensive, mon-scaling (control costs) and inefficient, i.e. very similar to ATM technology.

® WO 03/026341 PCT/DE02/@D3584 c) MPLS: This approach is based on ATM technology. Paths (connections) are set up in the network, via which the traffic of individual (generally aggregated) flows is specifically routed. It is frequently proposed for QoS in conjunction with

RSVP and DiffServ (see below under d)) and can also be provided based on ATM transport. It reverts to the complexity of connection- oriented mechanisms with all the consequences already set out (from bandwidth control to monitoring the existence of t he connection), i.e. it is of similar complexity to ATM technol ogy. In conjunction with the DiffServ sol.ution it should in p articular alleviate the problem discussed there (specific traf fic control via paths). d) Differentiated Services (DiffServ): The data packetss are classified and marked in the edge device on the basis of their association with specific services, applications or communication relations, etc. (Flow-related) access con trol and monitoring (e.g. for availability of resources and compliance witlh the specified bandwidth and QoS characteristicss) can and should also take place. The pa ckets then follow the route through the network predefined by their packet header dnformation (e.g. destination address) aned the routing protocols, whereby they are processed (or prior_itized) in every node &ccording to their marking with appropriate ‘per hop’ behavior. The DiffServ approach allows the freedom of per hop behavior wi. thin a single routing domain, e.g. the (=ub) network of an operator, but requires complete edge processing between such domains (subnetworks). The DiffServ approach cannot prevent temporary and/or local bottlenecks, as tliere is generally no consideration of or harmonization with the routes predefined by t he routing protocols. Generally packets wwith the same destin ation follow the same set route from the point

C WO 03/026341 PCT/DE02/03584 when they meet ir a node. This can result significantly fn skewed loads and bottlenecks in the networks with correspondingly Rong (queuing) delays or even packet lossses.

Network and route engineering is also a complex task, whereby the aspects of reliability and availability (e.g. rerouti.ng in the event of error) are a further complication.

D) In principle almost all combinations of said approaches are conceivable and have to a large extent also been discussed.

All these approaches have in common the fact that (with t=he exception of DiffServ) they are based on paths and use bandwidths and where necessary further resources reserved along said paths. Even a purely DiffServ approach is always based at least om routes predefined by routing protocols .

This is generally associated with a major administrative burden with regard to preparing and (statically) setting up paths and routes in the network or a correspondingly high control burden for the dynamic selection and switching of the routes. Also storage devices must be kept available in every network node to lhiold path-specific and connection-specifi_c information, which can be lost or have to be reconfigured on other routes in t he event of error. Even with the purely

DiffServ approach the traffic follows the routes predefined by the routing protocols and these therefore have to be very carefully dimensi oned and monitored. Generally however it is not possible to predict exactly either all fluctuations in traffic volume or the responses of the routing protocols to possible events im the network.

One object of the inwention is therefore to highlight a way =n which services which comply reliably and efficiently with their specific QoS requirerments can be provided simply, pragmaticallly and economically in a service-integrated, packet-oriented and in

@® WO 03/026341. PCT/DE02/03584 particular IP-based network.

This object is achieved by the subjects claimed. Extensive traffi c distribution in the network is proposed. Among other things it achieves an optimally balanced QoS wit h best effort character for all services and applications. Tra ffic distribution according to the invention also allows the step beyond a single routing domain to a comprehensive total solution .

One important aspect of the invention is the dep arture from conven tional, established ways of thinking, for example by challe nging the subjectivity of characteristics such as QoS and reliab-ility and no longer associating them with just one path or route but defining them as overall characteristi-cs of the network soluti on, which thereby increases in autonomy aned is also more economical to operate. Considering QoS at network level QoS first allows it to be represented in connectionless operation. Such consid eration according to the invention is based for example on the fo llowing deliberations: a) Qua lity of service (QoS) is a relative concept. Even when information is transmitted with circuit-switched technology, data losses cannot be excluded (e.g. due to fzilure (-> bit error) or frame slippage). Such weaknesses cam however either be tolerated (e.g. in digital telephony) or tkiey are intercepted by appropriate protection measuress in the same (e.g. by means of redundancy) or higher layers (e.g. by repetition) (data technology). The decisive factor in the final effect is the (subjective) quality perception of the recdpient of the information. Realtime, interactive communication involving people for example always takes place via their sensory organs (operating in an analog manner) which can function with incomplete informatiora (otherwise (in partticular mobile) telephone calls, film, and television as they operate at present would definitely not le possible).

The requirements for the interactive» control of machines (e.g. remote control of robots) are in some instances significantly more stringent so that. a detailed consideration of each individual case may be necessary here. However under no circumstances can the physical limits, e.g. with regard to distance-dependent transit times, be undercut.

QoS therefore does not necessarily require an absolute guarantee (this does not in any case exist, even using paths and reservations) but compliance with the corresponding specific requirements for the respective service from the point of view of the recipient of thie information. In the case of packet-oriented transmissiora this primarily concerns the nature and scope of possible information losses, fixed and/or variable delays and the temporal consistency (sequence) of the information. ATM technology for example is based con switching nodes and transmission routes dimensioned according to the rules of statistics and the principle of connection-oriented transmission wit h correspondingly reserved resources along the path, whereby the correct distribution of resources along the paths is ensured by powerful but therefore also complex queuing and scheduling mechanisms in the network nodes. b) Modern high-speed (data) networks op erate at wire speed. IP- based networks such as the internet first see only packets and process all of these in the same manner a priori in that the first packet to arrive is also t he first to be forwarded; if there are not sufficient transmis sion resources available, the packets are first stored (queuing, buffer) and if there is no more storage space available, surplus incoming packets are discarded (best effort principle ). The network nodes in these networks, known as routers, were originally computers, i.e. the complete functionality of amalyzing and forwarding

® WO 03/026341 PCT/DE02/03584 the data packets was implemen ted in software programs.

Accordingly until recently such networks were also comparatively slow. However writh the assistance of correspondingly dimensioned suffers and appropriate data protection mechanisms in the higher protocol layers such as for example TCP, it was possi ble (although frequently with long delays) to achieve a suf ficiently reliable and workable transmission of non-time-crit ical information.

Technological progress allowe d the implementation of elementary router functions i n hardware (ASICs, FPGAs), thereby opening up the route to fast and therefore also quasi- realtime forwarding of data packets on higher-speed connecting lines. Practically the only remaining delaying element is then the unavoidable buffering to resolve conflicts in the event of the simultaneous arrival of a plurality of data packets routed to the same egress. These del ays however become increasingly less significant with increas ing bandwidth (or better: speed) of the connecting lines betwe en the routers, because then the waiting times caused by confl icts become increasingly shorter due to the faster outflow of data packets. This is particularly the case when di fferent traffic streams can be differentiated by appropriate marking and can be processed differently during queuing an-d scheduling (DiffServ, prioritization). c) Despite this technological progress, important aspects of relevance to the service remain unaffected, such as: - the aggregation of traffic sstreams on the routes in the network, with the result that even with careful control of the traffic streams at the network inputs, further into the network skewed loads, which im some instances adversely affect the QoS, cannot be predicted and therefore cannot be prevented either, or

@® WO 03/026341 PCT/DE02/03584 - the immense complexity and resulting long time required to reconfigure the routes in the event of error, as a result of which the availability of network and services can be significantly restricted for the user.

A communication network according to the invention, according to this new inventive consideration, comprises the following characteristics and functionalities (basic concept): - it operates in a packet-oriented and connectionless manner, - it offers a plurality of input and output ports, - it comprises a plurality of network nodes which are intermeshed so that there is (generally) a plurality of pat hs between different input and output ports, - it contains mechani sms that strive all the time (i.e. where possible at every decision point in the network) to achieve the most regular distribution possible of the traffic load in the network taking into account the respective destination (output port) of the data packets.

The disadvantages mentiomned above of existing network technologies resulting during their deployment as a service-integrated, packet=- oriented network, are therefore largely eliminated, as required, and the desired economic advantages are achieved at the same time: - The network should be connectionless and packet-oriented. Am

IP-based network in particular can therefore be deployed, ass this satisfies the specified requirements.

@® WO 03/026341 PCT/DE02/03584 - Uniform traffic distribution allows optimum use of resources with maximum quality and therefore the most economical dimensioning. This achieves a low-cost total solution. - A network operated in a connectionless manner requires no control power for connection setup and cleardown, no route selection, no route reconfiguration, no path recovery in the event of error, etc. It is therefore simple to control and economic to operate, as very little administrative intervention is required and the network is almost self- organizing. - The aggregation of traffic streams with the same destination is avoided by definition as a result of distribution, because even aggregated traffic streams are redistributed to different lines in the network during the course of their further transmission. - In the event of an error, i.e. failure of an outgoing line, complex reconfigur ation of the traffic streams affected by the failure to backup routes is not required. Instead it is sufficient no longer to distribute the traffic streams to the failed line. To el iminate the error it is therefore only necessary to reduce the degree of distribution. There is no need for reconfiguration. - Finally the overall effect of the solution is clearly pragmatic, as the fact that there is no need for complex reconfiguration and prioritization mechanisms significantly reduces the configuration effort for network management.

Further features can be included in different embodiments and configurations. Some fea tures and feature combinations associated

@® WO 03/026341 PCT/DE02/03584 with particul.arly desirable advantages are set out below, together with some possible alternative solutions:

The object off traffic distribution is to achieve tke most uniform distribution possible of the traffic load in the network. It can take place im different granularities, e.g. based on aggregated traffic streams, for each individual traffic strearm or based on individual data packets. Distribution becomes all the more efficient, thie finer the granularity. The distributcion decision should be talxen automatically on an ad hoc basis im every network node. The decision criterion used is the informaticon delivered with the data paclkzets, e.g. a combination of source and destination addresses, ira some cases also with further information, which is used for example for assignment to a specific traffic stream. In the case of Aistribution based on traffic streams, all the data packets that belong to the same traffic stream generally take the same route tlarough the network. The quality-enhancing effect of traffic distribution, by reliably preventing skewed loads and even the overloading of individual network sections, is hereby achieved primarily with an adequate statistical mass or traffic streams of the same type (in particular with similar bandwidtlas).

In the case ©f a predefined network topology with «(theoretically) regular linki-ng/intermeshing (see Figure 1), route information for traffic distribution and resulting ‘branch patterns’ can be preset to a more or less permanent extent in the network rodes.

In a real, ewoclved data network (see Figure 3) intermeshing is generally irrregular and rather incomplete. Also changes repeatedly occur in the network configuration or network topol ogy during operation. According to the invention a flexible up>date of the possible routces and branch patterns takes place for this purpose (as required or at regular intervals) and/or the node derives new

¢ WO 03/026341 PCT/DE02/03584 branch p-atterns from changed route information. «Corresponding protocol s from the internet environment (routing protocols such as

OSPF, BGP) or variants/developments derived from these are possible mechanisans for the distribution of route information. Of course this inf-ormation can also be predefined via a network controller (of any type) or a network management system.

In the case of the branch patterns further criteria, such as different bandwidths, different distance from desstination, route costs, etc. can also be included in the algorithmwmns for specific route se lection. This means for example that in the case of packet distribution between an STM-4 and an STM-1 link «corresponding weighting can be used to ensure that only every %S'" packet is passed to the SIM-1 link. The load can be distributed more precisely when individual packet lengths are also taken into aceount. Weighting the linkss according to appropriate criteria is also advantageous, in order to prevent the formation of route loops in a network with complex DHDntermeshing or for example to limit packet delay jitter.

Different packet delays on different routes can xesult in a change in the packet sequence. This is restored at the metwork egress (resequemcing), e.g. if an application should recuire this.

The best effort character of the inventive traff-dc distribution, with which services and applications are adversely affected by an increasirmg load (more or less tangibly depending on their charactewistics and requirements), can be significantly improved if the overall traffic load in the network is limited on the basis of the actual network capacity.

On the ore hand the bandwidth of the individual retwork access points both on the ingress side and the egress side could also be considerezd and be taken into account both separat-ely and as part of the overall picture. Based on the statistical traffic characteristics of the different services and applications and based on the topology of the network and the capacity and

@® WO 03/026341 PCT/DE02/03584 performance of the network nodes and connecting lines and assuming specific response patterns on the part of the subscriber or specific traffic characteristics resulting from these at the network access points, the network is dimensioned so that specific factors determining the QoS limit values, such as packet loss rate, packet delay or delay jitter, are only exceeded under such initial conditions with an easily definab»le, sufficiently small statistic probability.

On the other hand the traffic in a given network could be limited to comply with the corresponding initial conditions. For this all communication relations and data streams in the network are parameterized appropriately, reco rded individually whenever they occur and permitted or rejected as a function of the current load situation in the network (admissi on control).

Both mechanisms are in themselves however neither practical nor economical. Therefore for example overdimensioning of the networks to comply with the requirements o f more susceptible services is not economically justifiable in respe«<t of less susceptible services, nor is admission control practical for the majority of the traditional and also future intermet applications designed for a best effort environment.

A differentiated QoS that is tailored to the requirements of the respective service will result according to the invention from a differentiation and classificatiorm into different traffic classes which are processed and in particular prioritized in a correspondingly different manner. The number of traffic classes is at least two. Strict prioritization is preferred for processing in the network nodes (i.e. at the queuing points), as alternative :

¢ WO 03/026341 PCT/DE02/03584 methods (e.g. Weighted Fair Queuimg WFQ) which also guarantee resources for lower priority traf fic classes in all circumstances generally impair the higher prior ity traffic in the case of a heavy load and are by nature significan tly more complex than strict prioritization, with which no lower priority traffic is transmitted while high priority traffic is wa iting for transmission.

In a communication network of the type mentioned above the feature of prioritization can have the fo llowing advantageous applications and embodiments: All data streams are classified in corresponding priority classes according to the ir requirements. The lowest class is taken into account during network dimensioning (in the context of expected overall traffic volume) and essentially processed according to the best effort principle. For all communication relations or data streams in higher priority classes an admission control is carried out at the network ingress (in the input direction) and at the network egress (in the output direction). For this these data streams are logged at these two points with corresponding parameters (e.g. mean data and/or packet rate, peak rate, etc.) and evaluated. The decisions at the ingress and egress are independent of each other and only if both decisions are positive is the data stream admitted. The decision criterion used can for example be a threshold value that is determined as a function of the port capacity, the overall network capacity, the required quality in respect of possible packet delays and losses, etc., the respective priority class and if necessary further criteria. It is also possible thatt there could be a plurality of threshold values for each class based on different evaluation parameters, all of which have to le complied with individually or in a corresponding relationship with each other.

The admission control on the one hand limits the overall traffic volume of a specific priority class in the network and on the other hand it also restricts the associated traffic volume at each individual input and output port. The regular distribution of the

¢ WO 03/026341 PCT/DE02/03584 traffic in the network (ideally packet-based) and the correspondingly preferred processing mean that this traffic will always find adequate re sources (free link capacity, buffers) in the network when the thresholds are set correctly, in order to comply with the limit values f or both delay and loss in its quality requirements. The network can thereby be fully utilized and economically operated, because all the bandwidth not used by high priority traffic can be used at any time by low priority traffic.

Ideally preferred proce ssing is achieved by strict priority, i.e. where necessary total displacement of low priority traffic. Strict priority means minimum delay and minimum loss. Also it is clearly a simpler priority mechan ism than for example the leaky bucket method known from the prior art.

Compliance with the registered traffic parameters by the individual ’ data streams is monitor ed, because in the context of traffic distribution even a single data stream “going haywire” can significantly disrupt all the traffic throughout the network. The monitoring function (traffic enforcement, policing) can advantageously be designed to be relatively insensitive and therefore economical, because a random, short-lived, minor violation is compensate d for correspondingly by the traffic distribution according to the invention.

The monitoring function is advantageously applied to the individual data streams as they are registered. Alternatively any type of aggregation can be provided for each port, with which only an overall limit is verified and intervention takes place in response to violation of the ove rall limit within an aggregate randomly and where necessary through all the data streams contained there. Any and of course all relevant known mechanisms (e.g. leaky bucket) can

® WO 03/026341 PCT/DE02/03584 in principle be used and the same also applies to the response options (discarding of packets, marking of packets, disconnection/blocking of data streams, etc.). In some instances marking can also involve the transfer of the packets infringing the agreement (or even better the entire associated data stream) to a lower or the best effort class.

The principle of traffic distribution (in particular when this takes place at packet level) can also be deployed very advantageously to improve network and service reliability and availability. For this it is sufficient for the network nodes, when they identify an error (e.g. failed link, failed adjacent node) to remove the associated link(s) from the branch fan and continue distribution via the remaining links only. The decision can be made immediately and autonomously on independent identification of error status and on external identification subject to availability of information. If the network is adequat ely dimensioned, such a response results at worst in somewhat more displacement of best effort traffic but to no impairment of the quality of the high priority traffic. This particularly de sirable advantage is clearly achieved with the inventive combinatiom in a very simple, pragmatic and economical manner, as comparatively simple mechanisms are deployed compared with the prior art described above.

One very interesting variant of the basic concept results when the method for traffic distribution is only applied to the higher priority traffic class(es). The best effort traffic selects its routes based on the current internet principles, while the higher priority traffic is distributed uniforwnly in the network and fills it from the bottom up, so to speak. The best effort traffic therefore can be said to swim on a moderately full sea of higher priority traffic and is increasingly di splaced as the ‘tide’ rises.

One attractive advantage of this varian-t is that the QoS solution can be added on to available networks, =while the existing mechanisms continue unchanged.

The proposed principle can also be appl ied in a cell-based network, e.g. an ATM network.

The reliability of the network is furth er improved by automatic monitoring mechanisms in the router, pa rticularly in the context of the distribution method.

To increase reliability over the entire network as an option a type of fast feedback mechanism can be deplo—yed between the routers, which makes it possible for example to distribute the traffic differently further upstream in good ti-me when problems occur somewhere further downstream.

Admission control is where necessary comfigured so that it automatically offers the user the next lass down when a high priority traffic class is ‘overbooked’.

The resequencing function is generally -—provided at the network egress, e.g. as a standard function. Advantageously all current TCP applications, in which the resequencing function is generally not implemented, can therefore continue to be used unchanged.

The invention is described in more deta. il below with reference to further exemplary embodiments shown in the figures, in which:

Figure 1 shows an arrangement for impZlementing the method according to the invention wkiich is configured as an exemplary network to illustrate the basic principle for traffic distribution in a network that is meshed regularly over different stages (network levels),

Figure 2 shows an arrangement for implementing the method according to the invemtion which is configured as one of a plurality of possible specific embodiments of the invention,

Figure 3 shows an arrangement for implementing the method according to the invemtion which is configured as a real, evolved (data) metwork with irregular and rather incomplete meshing.

In the figures those transmission edges along which traffic streams are transmitted in a distributed manner in each instance are marked in each case by an arrowhead, wh ich also points in the direction of transmission.

One embodiment of the invention provides a connectionless, packet- oriented communication network, - with at least two different traffic classes, of which one is processed purely as best effort traffic, while the at least one other is strictly prioritized in respect of it (and in the case of a plurality of others, preferably also in respect of each other), - with network nodes, from which the traffic is distributed individually and autonomously with the objective of uniform traffic load distribution preferably in a packet-based manner according to specific rules to all or at least a plurality of paths in the direction of their destination (metwork egress), - in which the network nodes exchange/disseminate the information about available routes by means of corresponding protocols,

® WO 03/026341 PCT/DE02/03584 - in which the network nodes adjust their (traffic) distribution patterns immediately and autonomously in the event of an error, - that undertakes an admisssion control based on specific traffic parameters for the data streams of the at least one higher traffic class preferably at every ingress and egress (said admission control for example no longer admitting further traffic of said traffic class(es) from a total load reached of x% (x%, (x+d)%, (x+nd)%) of port capacity), - that only accepts a data stream of the at least one higher traffic class if both admission controls (at the input port and at the output port — independently!) have made a positive decision, - that monitors the registered traffic parameters of the data streams of the higher traffic class(es) at each ingress and where necessary interveraes with appropriate measures, and - that provides a resequeracing function at every egress for optional use by (all) tlhe data streams.

A further embodiment of the imvention is shown in the communication network shown in Figure 1. With the network 100 at least one traffic stream is transmitted in a distributed manner from a transmission node A configured as an ingress node to a transmission node B configured as an egress node. The distributed transmission thereby takes place in the network 100 in such a way that the part of the traffic stream(s) recezdved in each instance is transmitted from most of the transmission nodes of the network 100 to precisely two subsequent transmission nodes in a distributed manner. Only the two transmission nodes arranged immediately before the egress node

B transmit directly to the egxess node B in the absence of alternative residual routes without network-wide distribution, so that the parts transmitted in a distributed manner can be merged

@® WO 03/026341 PCT/DE02/03584 in this back into the original traffic str-eams. At the egress node

B the distributed traffic streams arrive i n the present instance from two different directions. The receive d parts of the traffic st reams are preferably resequenced into th. eir original sequence at the egress node by means of an assigned re-sequencing function RF.

This means that traffic streams can also bee transmitted in the network 100 between applications which are= dependent on transmission maintaining the original sequence before transmission, without requiring a change and/or modifica tion of the applications.

Figure 2 shows an alternative embodiment of the invention comfigured as a communication network 200. With the network 200 at least one traffic stream is transmitted im a distributed manner from a transmission node C configured as a.n ingress node to a transmission node D configured as an egresss node. Unlike the network 100 only some of the transmission edges of the network 200 are used for distributed transmission betwween the two nodes C and

D. This is based on the fact that in the mmetwork 200 not just any rowmite is used for distributed transmissiom from the node C to the node D but only those routes that are part-icularly suitable for this purpose. In the present example theses are those routes which, taking into account the topology of the network 200, do not take tow great a detour through the network 200 and therefore are all sulbject to a transmission delay, which is preferably within a relatively small, predefined tolerance rarmmge. Routes which lead from the node C to the node D, but the transmission delay of which dewiates too greatly from the tolerance ramnge, are not suitable for this exemplary distributed transmission.

The traffic streams are preferably transmi.tted in a packet-based mammner in every transmission node between the nodes C and D to the respective subsequent nodes. Also distribution is carried out taking into account utilization of the respectively remaining res idual routes and/or the length of the packets transmitted respectively to the residual routes. This results im a largely uni formly distributed transmission between the nodess C and D. If thi s principle is applied between all the ingress amd egress nodes of the network 200, the network 200 is filled from &the bottom up wit h a basic stock of distributed traffic, whereby witilization of the transmission nodes and transmission edges for the entire net work 200 is similar. No part of the network 200 xemains in ove rload for a longer period compared with the rema-inder of the net work 200.

If the incoming traffic is divided in the network 2«0 into two tra ffic classes, the higher priority traffic is tramsmitted in a pre ferred and distributed manner and the volume of the higher pri ority traffic is limited by means of admission ceontrols AC and tra ffic monitoring TE, the higher priority traffic «an be transmitted in the network 200 almost with realtime character. Best eff ort character is achieved for the lower priority traffic, whe reby its quality decreases as the higher prioritwy traffic inc reases and vice versa. Strict prioritization is totally adequate for prioritizing the traffic. Compared with other kmown pri oritization mechanisms this is characterized by Hts particular simplicity, as a result of which from an economic point of view it can be set up particularly advantageously in the transmission nod es.

Fur ther aspects of the invention are shown in the communication network 300 according to Figure 3. The network 300 <omprises a plurality of transmission nodes 301-315, whereby the transmission nod es 301-307 are configured as ingress and/or egress nodes. At lea st one traffic stream is transmitted in a distributed manner as fol lows from the transmission node E configured at east as the ing ress node 301 to the transmission node F configuzxed at least as the egress node 304:

301 320 308 321 309 308 322 311 323 313 309 324 310 325 314 311 327 303 329 312 313 330 312 331 315 310 326 311 327 313 314 333 313 334 305 032 0 V 33 | ~~ 304 000] | 337 0 [~~ 304 0]

It can clearly be identified that from every transmission node between the ncdes E and F, from which more than one residual route extends to the egress node F, the traffic sent to said transmission node is transmitted distributed to at least two residual routes.

Figure 3 also shows how the branch pattern changes when the transmission edge 325 fails. As a consequence in the predecessor node 309 the transmission edge 325 is deleted from the branch fan stored for transmission in the direction of the egress node F. No more traffic is then sent to the transmission nodes 314 and 305 located after the transmission edge 325. The transmission nodes 313, 312 and 315 also located after it however continue to receive traffic, which is transmitted distributed to other routes of the branch pattern not including the failed transmission edge 325. On failure of the transmission edge 325 the above branch pattern oo WO 03/026341 PCT/DE02/03584 changes as follows: 301 320 308 321 309 308 322 311 323 313 309 324 310 325 314 311 327 303 329 312 313 330 312 331 315 310 326 311 327 313 334 333 33 334 305

It can clearly be identified that the failure of the transmission edge 325 only results in a thinning out of the branch pattern and does not require reconfiguration of the network 300. In particular the egress node F is still accessed via two routes. It is clear that the invention is extremely resistant in a highly pragmatic manner to failures of transmission nodes or tiransmission edges. The higher the degree of meshing of the communicat-ion network, the more routes there are between the ingress and egresss nodes, so that even if the majority of the network fails, in most cases at least one route still remains, on which traffic streams can continue to be transmitted. Total interruption only occurs if the communication network more or less totally fails. In this case however even the complex reconfiguration of the routes known fmom the prior art

N

C WO 03/0263<31 PCT/DE02/03584 would be of little assistance. It would be posssible at most if in normal operation routes classed as unsuitable were still functiornal. In this case reconfiguration of the still functional transmission nodes to new, less optimum branch patterns would be possible in the event of at least partial failure of the existing branch patterns. The transmissions interrupteci as a result of the failure can then be resumed after reconfiguration if alternative routes lave been found.

It shoul.d be pointed out that the description of the components of the staradard communication network of relevance to the invention should in principle not be seen as restrictives. It is clear in particular to a person skilled in the relevant art that the terms used should be understood functionally and not physically. As a result the components can also be provided partially or wholly in software and/or distributed over a plurality of physical devices.

Claims (22)

RE @® 2001 P 17488 WO Revised claims

1. Method for transmitting traffic streams in a connectionless, packet -oriented commumication network (100, 200, 300) that comprises a plurality of transmission nodes (301-315), which are connected together in such a way that a plurality of routes exists between the transmiss ion nodes, whereby at least some of the transmission nodes ar e configured as ingress nodes (301-307, A, C, E) and/or as egress nodes (301-307, B, D, F) of the communication network, comprising the following steps: - the traffic streams are subdivided into at least two traffic classes - at least one without priority and at least one with priority, - the stream of prior ity traffic streams coming into the communication network is limited by means of an admission control (AC) that is carried out for every priority traffic stream at the respective ingress node and at the respective egress node, - the traffic streams are sent from the ingress nodes to the egress nodes, - for at least one of the transmission nodes receiving at least some of the traffic streams sent (301, 303-305, 308-315), more than one residual route extending from it to at least one of the egress nodes is identified, - the traffic streams sent to this egress node are transmitted from. said transmission node distributed to at least two of the identified residual routes.

2. Method according to Claim 1, wherein from each transmission node from which more than one residual route extends to an egress node, the traffic streams sent to said egress node are transmitted distributed to at least two of the residual routes. AMENDED SHEETS

C 2001 P 17488 WO

3. Method according to one of the preceding Claims, wherein distribution is effected from at least one of the transmission nodes for at least one traffic stream to a specific egress node in such a way that its packets are transmitted in a distributed manner.

4. Method according to one of the preceding Claims, wherein distribution is effected from at least one of the transmission nodes for at least one egress node in such a way that the traffic streams are transmitted to said egress node in a distributed manner, while the associated packets of each traffic stream are transmitted in a neon-distributed manner.

5. Method according to one of the preceding Claims, wherein distribution is effected from at least one of the transmission nodes for at least one egress node in such a way that different aggregations of traffic streams are transmitted in a distributed manner, while the associated traffic streams of each aggregation are transmitted im a non-distributed manner.

6. Method according to one of the preceding Claims, wherein the residual routes awe identified taking into account at least one branch fan in which the residual routes appropriate for distribution of the traffic streams are stored.

7. Method according to Claim 6, wherein an appropriate but failed and/or defective residual route is flagged accordingly in the branch fan, in particular is deleted from the branch fan. AMENDED SHEETS

® 2001 P 17488 WO

8. Method according to one of the poreceding Claims, wherein those residual routes are identified for distributed transmission that are within a predefined tolerance range in respect of their bandwidth, their disstance from the egress node, their cost and/or their current level. of use.

9. Method according to one of the preceding Claims, wherein the traffic streams are distributed taking into account the respective level of use of the appropriate residual routes and/or the respective scope of the parts of traffic streams already transmitted to the individual residual routes, in particular the length of the respectively transmitted packets.

10. Method according to one of the preceding Claims, wherein the priority traffic classes are transmitted in a strictly prioritized manner in the transmission nodes.

11. Method according to one of the preceding Claims, wherein the priority traffic streams are transmitted in a distributed manner.

12. Method according to one of the preceding Claims, wherein the two admission controls arce carried out independently of each other and the verified traffic stream is only admitted for transmission if the results of both aadmission controls are positive.

13. Method according to one of the preceding Claims, wherein it is monitored whether at le=ast one predefined traffic parameter is complied with by the pri ority traffic streams.

14. Method according to Claim 13, wherein violations of the traffic par-ameter that are identified accordingly by the traffic distributi.on are tolerated. AMENDED SHEETS

C 2001 P 17488 WO

15. Method according to one of Claims 13 or 14, wherein in the event of violations of the traffi< parameter that are not identified accordingly by the traffic distribution, the portion of traffic violating the traffic parameter is transmitted without priority.

16. Method according to one of the preceding Claims, wherein it is verified for at least some of the ports of the access nodes via which priority traffic streams are tramsmitted into the communic ation network whether at least one overall limit of that which may as a maximum be transmitted into the communication network is complied with.

17. Method according to one of the preceding C1 aims, wherein it is verified individually for each traffic stream at least fox some of the traffic streams whether the predefined traffic parameter is complied with.

18. Met hod according to one of the preceding Cl aims, wherein at least in the case of some of the egress nodes, at least for the traffic streams transmitted in a distributed manner, their original sequence, as it existed before their transmission in the communication network, is restored using a respectively assigned resequencing function (RF). AMENDED SHEETS

LY - 37a -

19. Communication network (100, 200, 300) comprising means that are set up to implement all the steps of a method according to one of the preceding method Claims.

20. Device, comprising - Means that are set up to implement those steps of a method according to one of the preceding method Claims that are effected by the device, - Means that are set up to implement intesractions of the device as stipulated according to the metlod with further devices by which the remaining steps of tlme method are implemented.

21. Device according to the preceding Claim, which is developed as a computer program peroduct and the means of which are configured as program codes which are executed by at least one processor to impl ement the method. AMENDED SHEET

® WO 03/026341 PCT/DE02/03584

22. Arrangement, in particular a communication network (100, 200, 300) compris ing - at least ome device according to Claim 21.