WO2002001796A2

WO2002001796A2 - System and method for communication session recovery

Info

Publication number: WO2002001796A2
Application number: PCT/US2001/018516
Authority: WO
Inventors: Arvind Puntambekar
Original assignee: Sycamore Networks, Inc.
Priority date: 2000-06-28
Filing date: 2001-06-08
Publication date: 2002-01-03
Also published as: WO2002001796A3; AU2001266776A1

Abstract

A system and method for communication session recovery is provided wherein a circuit is rerouted in response to a failure in the network. A method and apparatus is provided wherein a control function is lost that controls circuit maintenance and rerouting functions, the circuit can be routed despite the loss of control function. The method comprises detecting that a portion of an established connection between a source and destination system is not operating, detecting that a control session is not operating in a first intermediary networking system along the established connection, and in response to detecting that the control session is not operating, establishing a new control session in the first intermediary networking system. In one aspect, a disconnect message is forwarded, within the new control session, to a switch that established the connection.

Description

SYSTEM AND METHOD FOR COMMUNICATION SESSION RECOVERY

BACKGROUND OF THE INVENTION

Field of the Invention

The invention relates generally to network communications, and more specifically, to a system and method for recovering a communication session.

Related Art In general, a number of methods for communicating information in a network of computers are well-known. Typically, a network includes one or more systems including general-purpose computers such as PCs, network communication hardware including switches and routers, and perhaps other systems capable of sending and/or receiving information. These devices communicate over one or more communications media, such as copper cabling, optical fiber, wireless links or other communication medium capable of carrying information.

Network infrastructure communication entities such as switches and routers receive and transmit information destined between one or more end systems. Typically, information is transmitted in the form of transmitted messages called packets. Generally, packets include routing information and data, and may be in a number of different formats, depending on a communication protocol used to communicate the information. A protocol defines details of packet formats, provides a procedure for passing information between a sending and receiving system, and defines how systems handle error conditions. A conventional communication protocol used by systems to communicate information is the Transmission Control Protocol/Internet Protocol commonly referred to as TCP/IP, which has become the de facto standard for communication between systems. As is known, TCP/IP includes a group of different protocols, the group being commonly referred to as a protocol suite. For example, TCP/IP includes a reliable delivery mechanism protocol referred to as the Transmission Control Protocol (TCP). TCP/IP also includes a connectionless protocol for sending data between systems, referred to as the User Datagram Protocol (UDP).

TCP provides a reliable packet delivery service that includes recovery mechanisms - i.e., mechanisms for recovering from transmission errors, lost packets, or failures of intermediate network communication hardware between a sending and a receiving system. The TCP protocol handles such problems and allows an application on one system to establish a connection referred to as a "session" with an application on another system. In particular, TCP performs communication functions such as assembling a stream of data into packets and sending them, waiting for the receiving system to acknowledge reception and providing packet retransmission services, for retransmitting packets if necessary. TCP is well-known and is described in more detail in many sources, such as, for example, in the books titled Internetworking with TCP/IP, Volume I, Second Edition, by Douglas E. Comer, Prentice-Hall, NJ, 1991, and TCP/IP Illustrated, Volume I, The Protocols, by W. Richard Stevens, Addison Wesley Publishing, MA, 1994, both incorporated herein by reference.

Multiprotocol Label Switching (MPLS) is a method for forwarding packets within a communication network. MPLS uses short, fixed-length values carried by packets, the values being called labels, to determine a "next hop" upon which to forward a data packet. In MPLS, network entities such as routers and switches store and operate on the labels or "tags" within packets that further define a predetermined path through a network. More specifically, routers or switches forward packets along a next hop along the predetermined path defined by the label or "tag" in the packet. Two routers, called Label Switching Routers (LSRs) in MPLS, must agree on the meaning of labels used to forward traffic between the routers and through them. This common understanding is achieved by using a set of procedures, called a label distribution protocol (LDP), by which one LSR informs another of one or more label associations that the router has made. An Internet draft document titled LDP Specification by Loa Anderson, et al., filename draft-ietf-mpls-ldp-06.txt dated October 1999 and available from the Internet Engineering Task Force (IETF) on the Internet at http://www.ietf.org, describes one such label distribution protocol. General label switching concepts are described, for example, in the book titled Switching in IP Networks - IP Switching, Tag Switching and Related Technologies by Bruce Davie, et al., Morgan Kaufmann Publishers, Inc., San Francisco, CA, 1998, incorporated herein by reference. Other label distribution protocols and solutions for distributing labels are available and may be used.

A Label Distribution Protocol defines a set of procedures and messages by which LSRs establish Label Switched Paths (LSPs) through a communications network by mapping network-layer routing information directly to data-link layer switched paths. Two LSRs that use LDP to exchange label mapping information are referred to hereinafter as "LDP Peers" with respect to that information, and it is said that there is an "LDP Session" established between them. A single LDP session allows a peer to learn another peer's label mappings; i.e. the protocol is a bi-directional communication wherein each LSR communicates its label mapping to each of its peers.

MPLS/LDP signaling operates on a peer-to-peer basis for maintaining the state of each of the circuits that are setup in the network. The network in this context can be either electrical, optical, or other media type; there is no dependency on the actual physical medium of the network upon which the circuits are defined. Also, a circuit established using MPLS/LDP may traverse networks that implement more than one type of media.

LDP uses TCP for session communication. By using TCP, session messages are reliably delivered and distributed labels and state information associated with LSPs need not be refreshed periodically. However, UDP or any other protocol may be used to transfer label information.

LDP includes a mechanism by which an LSR can discover potential LDP peers. This discovery mechanism makes it unnecessary for operators to explicitly configure each LSR with its LDP peers. To discover another LSR, the LSR follows an LDP setup procedure to establish an LDP Session. In this procedure, LSRs establish a TCP session connection and use the connection to negotiate parameters for the LDP session, such as the label distribution method to be used. After LSRs agree as to the parameters, the session is operational, and the LSRs use the established TCP connection to distribute label information.

End-to-end circuit setup in a network can be achieved in many different ways. Two predominant methods include performing a hop-by-hop circuit setup and performing end-to end (source to destination) circuit setup. Currently, many networks implement an MPLS/LDP signaling method (a hop-by-hop method) for establishing circuits. In the MPLS/LDP method, a networking switch maintains a TCP session with a neighboring switch. The next switch along the circuit path also maintains a TCP session with a next switch along the path for the next hop and so on. Ultimately, a chain of switches is created wherein each switch maintains neighbor session-level connections with neighboring switches. Many systems have separate control functions and the data receipt and transmit functions in their implementations to maintain a high level of circuit availability in the event of loss of control functions. In particular, a Control Plane is generally described as one or more control functions that create, delete and maintain a communication connection. For example, using the TCP protocol, the Control Plane implementation of TCP includes control functions involved in TCP connection establishment and termination. When a TCP connection is established, an entity referred to as a Data Plane is responsible for receiving and transmitting data along the connection. The Control Plane is also involved in maintenance of the connection during its operation. In general, circuit setup is performed through the Control Plane and the Data Plane is operational after the circuit is established. Any physical level failures along the established circuit result in Data Plane functions signaling a fault to the Control Plane functions. The Control Plane is responsible for rerouting the circuit to an alternate path. In TCP/IP, a problem occurs if the TCP session terminates (loss of Control Plane functionality) for whatever reason, without generating an indication that the physical level has failed.

Circuit setup between a source system and a destination system may commence at the source, and is referred to as a source-routed communication. At the source system, such as a switch, signaling software in the switch requests from a routing protocol that an end-to-end path be established. The routing protocol on the switch typically accesses a routing database and returns this path information to the signalling software. A routing protocol used may be, for example, the Open Shortest Path First (OSPF) protocol and OSPF protocol routing information may be used to establish the path. Constraint-based Routing using Label Distribution Protocol (CR-LDP) is a protocol that combines a label- swapping mechanism such as MPLS with network layer (layer 3) routing (e.g., OSPF, BGP, or other network layer routing protocol.) In CR-LDP, an ingress switch receives a packet and performs a network layer lookup to determine where the packet will be forwarded. The ingress switch, sometimes referred to as a source switch, maintains a mapping between the network layer and label information. The ingress switch adds label information to the packet, and the thus-modified packet is forwarded to the destination. In particular, the ingress switch establishes the circuit using the routing information, receives data from an end system, and forwards the data onto the established circuit. The data may be forwarded by one or more intermediate network communication devices along the circuit to the destination. In this messaging system, the network communication devices send periodic messages from the source to the destination, and ensure the messages are received. If either end of the circuit does not respond after several retries, then one of the network communication devices performs a forced reroute of the circuit. There are several disadvantages of this method:

• Additional messaging is required to maintain the circuit in addition to the session level message mechanism, complicating the communication process.

• Additional messaging creates extra Control Plane data traffic in the network. Networks with limited bandwidth available for Control Plane data eventually lose

Control Plane data traffic packets. Because the amount of traffic generated by additional messaging is proportional to the number of circuits in the network, networks with many circuits experience higher proportional losses of Control Plane data. As the number of circuits increase, the amount of Control Plane data traffic increases and the probability of Control Plane data traffic loss increases also.

• Recovery time for a circuit is a function of the time necessary to communicate in the end-to-end messaging system. Because the end-to-end message system requires sending and receiving messages between source and destination hosts and detecting timeout occurrences, delays in error recovery are experienced. In mission critical networks, it is necessary to minimize circuit failures and timeout periods that result in the loss of critical data.

SUMMARY OF THE INVENTION These and other drawbacks of conventional communication systems and protocols are overcome by providing an apparatus and method for rerouting one or more circuits due to some failure in an intermediate section of the network. According to one aspect of the invention, if there is loss of a control function that controls circuit maintenance and rerouting functions, the circuit can be rerouted despite the loss of control function.

According to another aspect of the invention, a system is provided for communication session recovery, which (a) detects a data session loss along an established connection; (b) detects that a control session is not operating in an intermediary networking system along the connection; and (c) establishes responsive to detecting that a control session is not operating, a new control session in the intermediary networking system.

According to another aspect, a method for communication session recovery between the source system and destination system is provided. The method comprises steps of (a) detecting that a portion of an established connection between the source and destination is not operating; (b) detecting that a control session is not operating in a first intermediary networking system along the established connection; and (c) in response to step (b), establishing a new control session in the first intermediary networking system. According to another aspect of the invention, an apparatus is provided comprising: (a) receiving a request to release a connection; (b) verifying connectivity to a network communication device; (c) establishing connectivity to the network communication device if connectivity does not exist; and (d) forwarding the request to the network communication device. According to another aspect of the invention, a method for communication recovery is provided, comprising steps of: (a) receiving a request to release a connection; (b) verifying connectivity to a network communication device; (c) if connectivity does not exist, establishing connectivity to the network communication device; and (d) forwarding the request to the network communication device. Advantageously, because a connection is removed in response to a control channel and data channel failure, additional retries are not transmitted over the connection and other, additional messages are not created. The connection may also be rerouted if a failure in both channels is detected. Further, timeout periods for transmitting data are not exceeded, as the connection is rerouted through a same or different set of nodes.

Further features and advantages of the present invention as well as the structure and operation of various embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the drawings, like reference numerals indicate like or functionally similar elements. Additionally, the left-most one or two digits of a reference numeral identifies the drawing in which the reference numeral first appears. Brief Description of the Drawings

The invention is pointed out with particularity in the appended claims. The above and further advantages of this invention may be better understood by referring to the following description when taken in conjunction with the accompanying drawings in which similar reference numbers indicate the same or similar elements. In the drawings,

Figure 1 is a block diagram showing a conventional communications network; Figure 2 is a block diagram of a failure recovery in accordance with one embodiment of the invention; Figure 3 is a block diagram of an example apparatus for performing failure recovery;

Figure 4A shows a flow chart of a communication process in accordance with one embodiment of the invention;

Figure 4B shows a method for creating a circuit; Figure 4C shows an example of a circuit setup;

Figure 5 shows a flow chart of a process for communication session recovery; Figure 6A shows a TCP/IP message format; Figure 6B shows a TCP header format; Figure 7A shows an example LDP message format; Figure 7B shows an example LDP header format;

Figure 8 A shows an example Label Withdraw message format; Figure 8B shows an example Label Release message format; Figure 9 shows a state diagram of an LDP session operation; and Figure 10 shows an example time sequence of recovering a communication session.

Detailed Description

Figure 1 shows a block diagram of a conventional communications network 100 in which various embodiments of the invention may be implemented. The network typically includes one or more end systems 101, 102, commonly referred to as end nodes or hosts. These systems 101, 102 may be, for example, general purpose computers such as personal computers (PCs), workstations, servers, or other computer systems. Systems 101, 102 may also be specially-programmed, special purpose hardware. Systems 101, 102 generally include a processor (not shown) such as the PENTIUM family of microprocessors from the Intel Corporation, PowerPC processor, SPARC processor available from Sun Microsystems, or the like. Many other processors are available. Such a processor usually executes an operating system, which may be, for example, the WINDOWS 98, WINDOWS NT or WINDOWS 2000 operating systems available from the Microsoft Corporation, MacOS operating system available from Apple Computer, SOLARIS system available from Sun Microsystems, or UNIX-based operating systems such as Linux available from various sources. Other operating systems are available. Systems 101 and 102 are configured to transmit information to and receive information from each other using one or more networking protocols, such as TCP/IP. One or more intermediate network communication entities, referred to as nodes, receive and transmit information between systems 101 and 102. In Figure 1, nodes A-D (blocks 103-106, respectively) transmit packets between end system A (101) and system B (102). An example node is discussed in more detail below with respect to Figure 3. Nodes A-D and systems A and B are interconnected by a plurality of links 107-111. These links 107-111 may be any type of communication media, including copper or fiber optic cabling, or may include one or more systems for communicating information, such as a switch, router, or data repeating device.

Typically, data networking circuits are composed of multiple "hops" - basically a circuit established at a source system 101, 102 traverses a number of nodes to connect to the destination. As discussed above, Control Plane functionality is separated from Data Plane functionality in many networking systems to maintain high availability of data circuits. Thus, existing circuits will continue to operate even in the event of failure of the Control Plane, because the Control Plane is used for circuit setup and maintenance purposes. In the event of loss of the Control Plane functions, new circuits will not be able to be set up, but existing circuits continue to operate. In this multiple-hop environment, it becomes very important to maintain Control Plane operations for maintaining the health of the circuits in the network.

In a distributed network architecture, control functions associated with the Control Plane are typically performed by a device referred to as a "Control processor" or "module" in a node such as a switch, while the Data Plane functions are performed by a device referred to as an "Interface processor" such as individual interface cards or line cards in a switch. These processors are discussed in more detail below with reference to Figure 3. Because data forwarding and control functions are separated, it is possible for the Control Plane functions to fail, but the Data Plane functions continue operating, because the interface processors are functional and do not rely on Control Plane functions to forward data. But, if there is a failure in Data Plane functions during this period, it becomes difficult to propagate control signals indicating the failure to the source system, because the end-to-end control link is broken and this control link is used to notify other nodes that a failure occurred along the circuit. Because the link is broken, rerouting of an existing circuit fails. Generally, there are two failure considerations:

1. If Control Plane functions are performed by the control module, and that module fails, or needs to be removed for maintenance, how should existing circuits be rerouted?

2. Even if the Control Plane software (and control functions) are distributed, the software can fail in one or more nodes resulting in Control Plane disconnection. This failure of Control Plane software affects rerouting of existing circuits in the network and should be addressed.

More specifically, as shown in Figure 2, nodes A-F (201-206) have data channels 210-214 and control channels 215-219 between each node. A channel is described generally as the functions, protocols or procedures used to perform data forwarding or control functions. For example, if data channel 214 were to fail, data forwarding between nodes E and F is not available. To forward data between nodes E and F, node E establishes another path between nodes E and F, or if the circuit is a source-routed circuit, a new path is defined between the ingress switch (node A in Figure 2) and an egress switch (node F in Figure 2) by the ingress switch. For example, node A may define an alternate path from node A to node F through nodes G-H-J (220-222).

To accomplish rerouting in a source-routed network, a node that detects a data channel failure sends a "disconnect" message to the ingress switch along the established connection. This disconnect message may be, for example, a message used to deallocate a resource that was allocated for the circuit. In MPLS, this disconnect message may be an MPLS Label Withdraw message discussed further below with respect to Figure 8C. However, these disconnect messages are control messages, and if a control channel is not functioning among one or more of the upstream nodes, rerouting of the circuit cannot be performed because disconnect messages cannot be propagated to the switch that established the connection. Failure of a control channel may include, for example, failure of software or hardware involved in performing control functions or failure of the media used to convey control information. According to one aspect of the invention, an apparatus and method is provided that detects the receipt of a disconnect message and determines whether a control function used to propagate the message to other switches is operating. If not, a connection is established to the ingress switch, and the disconnect message is forwarded along the connection. Various aspects of the invention may be implemented as software in a node such as a switch, or may be provided as special-purpose hardware in one or more of the nodes.

According to another aspect of the invention, a detection mechanism is provided at a node such as a switch, and the mechanism is triggered during circuit error recovery procedures. The following steps are undertaken at the switch:

• When the switch receives a disconnect message, the switch determines if the switch has peer comiectivity, such as TCP Session Level connectivity, with a switch to which the switch needs to forward the disconnect message. If not, the switch determines the source switch that initiated the creation of the circuit from a local database. For example, in MPLS the disconnect message may be an LDP Label

Withdraw message.

• The switch identifies the source switch of the circuit (such as by address) and creates a temporary connection, such as a TCP session, to that switch.

• The switch forwards the received disconnect message to the source switch.

• The source switch performs normal circuit error recovery procedures ~ for example, the source switch clears the existing path and performs a new route lookup for the existing circuit.

Figure 3 shows a diagram of an example node which is suitable for performing various aspects of the present invention. Node 203 includes a control processor 303, and one or more interface processors such as 301, 302. Control processor 303 is responsible for circuit setup and management functions. In particular, control processor 303 may perform routing and signaling functions, management applications, and hardware monitoring functions. Control processor 303 may be, for example, a general-purpose processor such as a PENTIUM processor or it may be a special-purpose processor. Node 203 may include one or more control processors 303 to perform distributed control or for redundancy purposes.

Node 203 includes one or more interface processors 301, 302 that perform switching or routing of packets. Processors 301, 302 may also be general-purpose processors such as the PowerPC processor available from the Motorola Corporation. Other processors may be used. Processors 301, 302 perform data switching functions that are typically performed by "line cards" installed in node 203. Processors 301-303 may also have one or more associated memory devices 304, 308-309 that store program code executed by the processors 301-303, buffering, state, or other information needed to perform data forwarding and management functions. It should be understood that node 203 may have other internal structures and system hierarchies.

Interprocessor communication in node 203 may occur through one or more links, such as Ethernet, Fast Ethernet, or other communication means using one or more protocols, such as TCP/IP. For example, processors 301-303 may be interconnected by one or more control channels, other switching modules, such as a switch fabric that communicates information between interface processors 301, 302, or other communication means. Communication between processor 303 and processors 301, 302 may include control data such as routing or signaling data, or user data. Processors 301, 302 generally communicate with other systems or nodes through one or more network interfaces 305-307. Network interfaces may be any type of network interface, such as Ethernet, SONET, ATM, Frame Relay, or other network interface type.

Figure 4A shows a process 400 for communicating between two systems, such as systems A and B shown in Figures 1 and 2. At block 401, process 400 begins. At block

402, in response to a connection setup request transmitted from system A, intermediate nodes set up a connection between systems A and B. The connection may be, for example, a hop-by-hop circuit defined through one or more intermediate nodes. At block

403, intermediate nodes forward data along the connection established between systems A and B. During this data forwarding period, a control processor 303 may monitor the connection and perform error handling and connection rerouting functions, if necessary. At block 404, either system A or system B may initiate a closing of the connection, and any of control processors 301-303 may be involved in closing the connection and recovering resources in one or more nodes. At block 405, process 400 ends. As discussed above, label switching involves mapping particular labels to connections in a network. Labels in MPLS can be thought of as a linked list of connections across the network. Collectively, these connections form an end-to-end path or circuit defined between a source system and a destination system. These labels are defined and associated with an MPLS entity referred to as a Forwarding Equivalence Class (FEC), which is a definition of a data stream path to be applied for each type of data input to the network. Inputs may be defined in different manners depending on the application or use of the data. Typically, an FEC is associated with a particular source/destination TCP/IP address at an ingress point of the network, such as within an ingress node or switch. An FEC may be defined based upon a number of different parameters, including source system, destination system, quality of service requirements, or other requirements.

The circuit may be created by a configuration initialization of an interface processor, such as an interface processor 301, 302 becoming active, or by creating a new circuit. Figure 4B shows a method 450 for creating a circuit. At block 451 , process 450 begins. At block 452, the control processor 303 identifies the source and destination of the switch with calling parameters that specify how the circuit is to be established (e.g., security parameters, bandwidth requirements, interface numbers, etc.). At block 453, control processor 303 contacts the OSPF module to obtain a path (such as the optimal path) through the network, and the control processor 303 passes the circuit parameters to the OSPF module. At block 454, control processor 303 creates a circuit entry in the card, initiates a state machine that monitors the state of the circuit setup, and places the state machine in a "call setup state" indicating that the circuit is to be established. Also, at block 454, control processor 303 starts a timer, termed an "acknowledgement timer," that measures time elapsed between sending the connection setup message and the receipt (or non-receipt) of an acknowledgement (ACK) message associated with the connection setup message. At block 455, control processor 303 places the circuit entry in a "call in process" state indicating that a circuit is in the process of being established. At block 456, control processor 303 determines whether a setup acknowledgement message was received from the other end of the circuit within a predetermined time period. Control processor 303 kills the acknowledgement timer if processor 303 receives an acknowledgement message within the predetermined time period, or the acknowledgement timer expires indicating the circuit setup timed out. If the timer expires, control processor 303 attempts to establish the circuit again at block 457. When the circuit is established control processor 303 ensures that a circuit entry is created at every intermediate point to the final destination, i.e., the circuit is continuous from the source to the destination.

Figure 4C shows an end-to end circuit setup through a switch 460. As the circuit is defined (provisioned) in the interface processor 301 as shown — the circuit may be defined using an identification including Chassis c, Slot s, Port p, Channel n [c, s, p, n] information of a node such as switch 460. The circuit identification may be used in this instance to define a location within switch 460 where the circuit is defined. Other circuit identifications may be used. Interface processor 301 notifies the signaling software of control processor 303 regarding this circuit configuration. The signaling software requests from a routing protocol of the switch 460 a route to a destination node. In one aspect of the invention, the routing protocol may be OSPF.

The signaling software of the control processor 303 searches for a mapping for network layer traffic to be to be switched from a switch fabric processor 310 to the interface processor 301. Control processor 303 defines a unique LDP Label identifier for a circuit, for example, using the circuit identifier [c, s, p, n] described above. Switch 460 may include a switch fabric processor 310 that transfers information between interface processors 301, 302, control processor 303 or other processor types of switch 460. This processor 310 may be programmed to connect interface processors through uplink switch fabrics 461, 462. These uplink switch fabrics 461, 462 may include hardware capable of mapping interfaces in interface processors to connections defined in switch fabric processor 310. For example, a circuit connection Cl may be defined in uplink switch fabric 461 that maps network interface 305 to a circuit connection C3 defined in switch fabric processor 310. Connection C3 maps an ingress port of processor 310 to an egress port, the egress port being coupled to uplink switch fabric 462. Circuit connection C2, maps an ingress port of fabric 462 to an interface of processor 302, such as an interface that forms a trunk connection 463 to another switch (not shown). Control processor 303 selects a hardware mapping, e.g. circuit connection Cl, for connecting from the interface processor 301 to uplink switch fabric 461 of the interface processor 301. An LDP message (Label Request message) is created by control processor 303, and forwarded to interface processor 301 using a communication channel such as an Ethernet Control channel coupling the Control processor 303 and interface processor 301 at block 2. The Label Request Message also contains explicit path information, which is forwarded to the next switch or interface processor. Particularly, at block 3, interface processor 301 receives the LDP message, and forwards the message to interface processor 302. At block 4, interface processor 302 forwards the message to the next switch.

At interface processor 302, a similar procedure is performed whereby the connection request message is forwarded to the next switch. Control processor 303 reserves and selects connection C2 to connect from its uplink switch fabric 462 to trunk connection 463. At block 5, interface processor 302 receives, from the next switch, an LDP-Mapping message, and forwards the received message to control processor 303. At block 6, control processor 303 receives the LDP-Mapping message, and requests interface processors 301 and 302 to perform cross-connects between the processors 301, 302. In particular, control processor 303 signaling software sends cross-connect information to interface processors 301, 302, to program a hardware cross-connect between the interface processors 301, 302. At block 7, control processor 301 receives the LDP-Mapping message, completing the connection. Control Processor 303 also completes the hardware programming in the switch fabric processor 310, to connect circuit connections Cl & C2 and C3 as shown. Each of the processors (control processor 303 and interface processors 301, 302) maintains a table of connections established between the processors 301, 302, and one or more circuits may be set up in the manner described above. LDP session establishment is discussed in more detail below with respect to Figures 5 through 9.

Figure 5 shows a process 500 for recovering a communication session in accordance with one aspect of the invention. At block 501, process 500 begins. At blocks 502-503, a switch such as switch 460 shown in Fig. 4C monitors if a release request message is received from another switch or node along a predefined circuit or connection. A release request message is a request to deallocate a resource, such as the predefined circuit. For example, the release request message may be an LDP Label Withdraw message as described below with respect to Figure 8C. If a release request message is received, the switch determines if there is session-level connectivity to a next node along the circuit at block 503. If there is session-level connectivity, the switch forwards the request release to the next node at block 504. For instance, the acknowledgement message may be an LDP Label Release message as described below with respect to Figure 8D and determining session-level connectivity may include determining that a TCP control session exists between the switch. At block 505, the next node sends an acknowledgement that the request release was successfully received or processed at the next node. At block 506, the switch removes the connection from a database of connections maintained by the switch.

If there is no session-level connectivity to the next node, the switch establishes a control connection to the ingress node at block 509. At block 510, the switch forwards the release request to the ingress node along the established control connection. At block 511, the switch determines whether an acknowledgement is received at the switch. If so, the switch terminates the control connection to the ingress node at block 512, and removes the connection from its database at block 506. If the acknowledgement is not received, usually after a timeout period, the switch may invoke an error handler at block 573. The error handles retransmits the release request, determines whether the control connection is operational, or provides an error indication, as appropriate. At block 507, process 500 ends.

TCP Message Structure

As discussed above, label and state information may be communicated between nodes using the TCP protocol, which involves sending TCP messages called packets. As shown in Figure 6A, TCP packet 404 is encapsulated in an IP packet or datagram 403 as known in the art. That is, the TCP packet 404 is prepended with an IP header 402 which includes, among other things, routing information indicating the source and destination of the IP packet 403. The TCP packet also includes a TCP header 405, shown in more detail in Figure 6B. When the TCP protocol on a sending system sends a segment of information to a receiving system, the sending TCP protocol maintains a timer and waits for the receiving system to acknowledge reception of the segment. If an acknowledgement is not received in time, the segment is retransmitted. When the TCP protocol on a receiving system receives data from the sending system, the receiving system sends an acknowledgement message acknowledging receipt of the data.

TCP Header Format The TCP header 405 as shown in Figure 6B includes a source 406 and destination

407 port number to identify a sending and receiving application. These two port number values 406, 407 along with the source and destination IP addresses in the IP header 402 uniquely identify each connection. The sequence number 408 identifies a byte in the stream of data represents. When a new connection is being established, the S YN flag of flags 418 is turned on (i.e., the field has a defined value, chosen to be "1"). The sequence number field contains an initial sequence number (ISN) chosen by the sending system for the connection. Because every byte exchanged is numbered, the acknowledgement number 409 contains a next sequence number that the receiver sending the acknowledgement expects to receive. The header length 410 includes the length of the header in 32-bit words. There are six flag bits 418 in the TCP header. The option field 415 and data field 416 are optional fields that may be included in the TCP header 405.

To establish a TCP connection as is well-known in the art, the system requesting the connection (the sending system, sometimes referred to as the "client") sends a SYN segment specifymg a port number of the receiving system (sometimes referred to as the "server") that the client wants to connect to, and the client's initial sequence number (ISN). The server responds with its own SYN segment including the server's initial sequence number. The server also acknowledges the client's SYN by acknowledging the client's ISN plus one. The client acknowledges the server's SYN by acknowledging the server's ISN plus one. These three segments complete the session establishment, and the sequence is often referred to in the art as a three-way handshake. Opening and closing of TCP connections is discussed in more detail in the literature, for example, the book entitled TCP/IP Illustrated, Volume I - The Protocols by W. Richard Stevens, Addison Wesley Publishing, MA, 1994, incorporated herein by reference.

LDP Header Format

Figure 7A shows a block diagram of an example LDP header message format. Each LDP PDU can include one or more LDP messages. Each LDP PDU includes an LDP header followed by one or more LDP messages. The LDP header includes a version field 701 that contains a version number of the LDP protocol. The PDU length field 702 specifies the total length of the PDU, excluding the version 701 and length 702 fields. The LDP identifier field 703-704 uniquely identifies a label space of the sending LSR for which this PDU applies. The first four octets encode an IP address assigned to the LSR. The last two octets identify a label space defined within the LSR.

LDP Message Structure

As shown in Figure 7B, LDP messages commonly use a Type-Length- Value (TLV) encoding scheme. An LDP TLV is encoded as a 2-octet field that uses fourteen bits to specify a Type 707 and two bits 705, 706 to specify behavior when an LSR does not recognize the Type, followed by a two-octet Length field 708, followed by a variable length Value field 709. The Type field 707 encodes how the Value field 709 is to be interpreted. The Length field 708 specifies the length of the Value field 709 in octets. The Value field 709 is an octet string of Length octets that encodes information to be interpreted as specified by the Type field 707. In general, the LDP message format may be used to encode any information, and any message may be placed in a TLV message format. In should be understood that other formats for an LDP message may be used. Figures 8A through 8B show several exemplary LDP protocol message formats used in session communication. Many other messages having different formats may be used. In particular, below is described an LDP Hello message and an LDP Initialization message. Also, Figure 8A shows an LDP Label Withdraw message and Figure 8B shows an LDP Label Release message format. These messages and their use are described below in more detail with reference to Figures 9 and 10. LDP Hello messages are exchanged between nodes as part of LDP discovery.

For example, these hello messages may be, for example, hello messages as specified by the LDP Specification document described above. Particularly, the example format includes a number of fields, including a field encoded to identify the LDP message as a Hello message. A field indicates the message length. A message ID field, used to identify the message, may be for example, 32 bits long. A Common Hello Parameters TLV field specifies parameters common to Hello messages including hold time (time a sending LSR maintains a record of receiving a Hello message from a receiving LSR without receipt of another Hello message) and identification of the Hello message type, among others.

An LDP Initialization message may also be used in session communication, such as the LDP Initialization message specified by the LDP Specification document described above. Initialization messages are exchanged as part of an LDP session establishment procedure, discussed in more detail below with respect to Figure 9. LDP Initialization messages include a number of fields, one of which includes a value that identifies the LDP message as a Session Initialization message. The message also includes a field that indicates message length. The message also includes a message ID field, used to identify the message, may be for example, 32 bits long. Common Session Parameters TLV field specifies parameters proposed by a sending LSR for parameters to be negotiated for the LDP session. An Optional Parameters field is an optional field that may contain other parameters for the session, such as those specific to ATM, Frame Relay, or other connection type. Figure 8 A shows an exemplary format of an LDP Label Withdraw message. An

LSR sends a Label Withdraw message to an LDP peer to signal the peer that the peer may not use specific FEC-label mappings the LSR had previously advertised. LSRs that receive this message break mappings between the indicated FEC and label. Field 801 indicates that the LDP message is a Label Withdraw message. Field 802 indicates the length of the message. Message ID field 803, used to identify the message, may be, for example 32 bits long. FEC TLV field 804 identifies the FEC for which the FEC-label mapping is being withdrawn. Label TLV field 805, an optional field, specifies the label being withdrawn. The LDP Label Withdraw message also includes an Optional Parameters field 806 which may include other parameters for the session. An LSR transmits a Label Withdraw message under several different conditions.

These conditions include: (1) the LSR no longer recognizes a previously-known FEC for which the LSR has advertised a label; and (2) the LSR has determined (such as from its configuration or a circuit failure) to no longer label-switch a FEC with the label mapping being withdrawn. An LSR that receives a Label Withdraw message responds with a Label Release message.

Figure 8B shows an exemplary format of an LDP Label Release message. An LSR sends a Label Release message to an LDP peer to signal that peer that the LSR no longer needs specific FEC-label mappings previously requested of and/or advertised by the peer. Field 807 indicates that the LDP message is a Label Release message. Field 808 indicates the length of the message. Message ID field 809 is used to identify the message and may be, for example, a 32-bit field. FEC TLV field 810 identifies the FEC for which the FEC-label mapping is being released. Label TLV field 811, an optional field, specifies the label being released. The LDP Label Release message also includes an optional parameters field 812 which may include other parameters for the session. An LSR transmits a Label Release message to a peer when the LSR no longer needs a label previously received from or requested of that peer. An LSR transmits a Label Release message under several different conditions. These conditions include: (1) the LSR that sent the label mapping is no longer the next hop for the mapped FEC, and the LSR is configured for conservative operation; (2) the LSR receives a label mapping from an LSR which is not the next hop for the FEC, and the LSR is configured for conservative operation; and (3) the LSR receives a Label Withdraw message.

The LDP protocol defines messages, TLVs and procedures in the following general areas: peer discovery, session management, label distribution, and notification of errors and advisory information. To more fully understand various embodiments, one should understand LDP peer discovery and session management.

To engage in LDP Basic Discovery on an interface, an LSR periodically sends LDP Link Hello messages out the interface. LDP Link Hello messages are sent as UDP packets addressed to the well-known LDP discovery port for the "all routers on this subnet" group multicast address. An LDP Link Hello message sent by an LSR carries the LDP Identifier for the label space the LSR intends to use for the interface and any additional information associated with the circuit. Receipt of an LDP Link Hello message on an interface identifies a "Hello adjacency" with a potential LDP peer reachable at the link level on the interface, and identifies the label space the peer intends to use for the interface.

LDP sessions between non-directly connected LSRs are supported by LDP Extended Discovery. To engage in LDP Extended Discovery, an LSR periodically sends LDP Targeted Hello messages to a specific IP address. LDP Targeted Helios are sent as UDP packets addressed to the well-known LDP discovery port at the specific address. An LDP Targeted Hello message sent by an LSR carries an LDP Identifier for the label space the LSR intends to use and possibly additional optional information used to identify the circuit. Extended Discovery differs from Basic Discovery in several ways. First, a Targeted Hello message is sent to a specific IP address rather than to the "all routers" group multicast address for the outgoing interface. Second, unlike Basic Discovery, which is symmetric, Extended Discovery is asymmetric. One LSR initiates Extended Discovery with another targeted LSR, and the targeted LSR decides whether to respond to or ignore the Targeted Hello. A targeted LSR that chooses to respond does so by periodically sending Targeted Helios to the initiating LSR. Receipt of an LDP Targeted Hello identifies a "Hello adjacency" with a potential LDP peer reachable at the network level and the label space the peer intends to use.

LDP Session Establishment

The exchange of LDP Discovery Hello messages between two LSRs triggers LDP session establishment. Session establishment is a two-step process involving (1) a transport connection establishment and (2) an LDP session initialization. The following describes establishment of an LDP session between two LSRs: LSRl and LSR2. The description is provided from LSRl ' s position in the session establishment. The process assumes that exchange of Hello messages (referred to as "Helios") specifymg label space LSRL.a for LSRl and label space LSR2:b for LSR2 has already occurred.

Transport Connection Establishment The exchange of Helios results in the creation of a Hello adjacency at LSRl that serves to bind the link and the label spaces LSRl :a and LSR2:b.

(1.) If LSRl does not already have an LDP session for the exchange of label spaces LSRl :a and LSR2:b, it attempts to open a TCP connection for a new LDP session with LSR2. LSRl determines the transport addresses to be used at its end (Al) and LSR2's end (A2) of the LDP TCP connection. Address Al is determined as follows:

(a.) If LSRl uses the Transport Address optional object (TLV) in Helios it sends to LSR2 to advertise an address, Al is the address LSRl advertises via the optional object;

(b.) If LSRl does not use the Transport Address optional object, Al is the source IP address used in Helios it sends to LSR2. Similarly, address A2 is determined as follows:

(a.) If LSR2 uses the Transport Address optional object, A2 is the address LSR2 advertises via the optional object; (b.) If LSR2 does not use the Transport Address optional object, A2 is the source IP address in Helios received from LSR2.

(2.) LSRl determines whether it will play the active or passive role in session establishment by comparing addresses Al and A2 as unsigned integers. If the value of Al does not equal the value of A2, LSRl plays the active role; otherwise it is passive. (3.) If LSRl is active, it attempts to establish the LDP TCP connection by connecting to the well-known LDP port at address A2. If LSRl is passive, it waits for LSR2 to establish the LDP TCP connection to its well-known LDP port.

Session Initialization

After LSRl and LSR2 establish a transport connection such as through TCP, they negotiate session parameters by exchanging LDP Initialization messages. Parameters negotiated include LDP protocol version, label distribution method, timer values, VPI/VCI ranges for label-controlled ATM, DLCI ranges for label controlled Frame Relay, and any other parameters to be used for the connection. Successful negotiation completes establishment of an LDP session between LSRl and LSR2 for the advertisement of label spaces LSRl:a and LSR2:b.

The following describes the session initialization from LSRl's point of view. After the connection is established, if LSRl is playing the active role, it initiates negotiation of session parameters by sending an Initialization message to LSR2. If LSRl is passive, it waits for LSR2 to initiate the parameter negotiation. In general, when there are multiple links between LSRl and LSR2 and multiple label spaces to be advertised by each LSR, the passive LSR cannot know which label space to advertise over a newly established TCP connection until it receives the first LDP PDU on the connection. By waiting for the Initialization message from its peer the passive LSR can match the label space to be advertised by the peer (as determined from the LDP Identifier in the PDU header for the Initialization message) with a Hello adjacency previously created when Helios were exchanged.

When LSRl plays the passive role: (a.) If LSRl receives an Initialization message, it attempts to match the LDP

Identifier carried by the message PDU with a Hello adjacency.

(b.) If there is a matching Hello adjacency, the adjacency specifies the local label space for the session. Next, LSRl checks whether the session parameters proposed in the message are acceptable. If the proposed parameters are acceptable, LSRl replies with an Initialization message of its own to propose the parameters it wishes to use and a KeepAlive message to signal acceptance of LSR2's parameters. If the parameters are not acceptable, LSRl responds by sending a Session Rejected/Parameters Error Notification message and closing the TCP connection.

(c.) If LSRl cannot find a matching Hello adjacency, the LSRl sends a Session Rejected/No Hello Error Notification message and closes the TCP connection.

(d.) If LSRl receives a KeepAlive in response to its Initialization message, the session is operational from LSRl 's point of view. (e.) If LSRl receives an Error Notification message, LSR2 has rejected its proposed session and LSRl closes the TCP connection.

When LSRl plays the active role:

(a.) If LSRl receives an Error Notification message, LSR2 has rejected its proposed session and LSRl closes the TCP connection. (b.) If LSRl receives an Initialization message, LSRl checks whether the session parameters are acceptable. If so LSRl replies with a Keep Alive message. If the session parameters are unacceptable, LSRl sends a Session Rejected/Parameters Error Notification message and closes the connection.

(c.) If LSRl receives a KeepAlive message, LSR2 has accepted its proposed session parameters.

(d.) When LSRl has received both an acceptable Initialization message and a KeepAlive message, the session is operational from LSRl's point of view.

It is possible for a pair of incompatibly configured LSRs that disagree on session parameters to engage in an endless sequence of messages as each negatively acknowledges (NAKs) the other's Initialization messages with Error Notification messages. An LSR throttles its session setup retry attempts with an exponential backoff in situations where Initialization messages are being negatively acknowledged with Error Notification Messages (NAK'd).

Initialization State Machine

It is convenient to describe LDP session negotiation behavior in terms of a state machine. According to one embodiment an LDP state machine is defined to have five possible states and the behavior as a state transition diagram is shown in Figure 9. At NON-EXISTENT state 901, a TCP connection is not established, and an LDP session is not established. At arc 906, a TCP connection is established between two LSRs, LSRl and LSR2, resulting in (i.e., ending at) state INITIALIZED, 902. At INITIALIZED state 902, the TCP connection is established, and the LSRs may transmit or receive LDP messages. At arc 908, an LSR receives an acceptable Initialization message (passive role) and the state changes to OPENREC, 904. At arc 909, the LSR transmits an Initialization Message (active role) and transfers to the OPENSENT state 903 along arc 909. If an LSR is in the passive role and receives an acceptable Initialization message, the machine state changes to OPENREC state 904. If the LSR receives any other LDP message, the LSR transmits an Error Notification message (NAK) and closes the TCP connection (the machine changes to the NONEXISTENT state 901 along arc 907). If, at OPENREC state 904, the LSR receives a KeepAlive message, the machine state transfers to the OPERATIONAL state 905 along arc 912. If, while at OPENREC state 904, the LSR receives any other LDP message, the LSR transmits an Error Notification message and closes the TCP connection (i.e., the machine transfers to the NONEXISTENT state 901 along arc 913). If the machine is at OPENSEND state 903 and the LSR (active role) receives an acceptable Initialization message, the LSR transmits a KeepAlive message and the machine changes to the OPENREC state 904. If the machine is at the OPENSEND state 903 and the LSR receives any other LDP message, the LSR transmits an Error Notification message and closes the TCP connection (i.e., the machine changes to the NONEXISTENT state 901 along arc 914). If, at the OPERATIONAL state 905, the LSR receives or transmits a message requesting shutdown of the session, or does not receive an LDP message within a timeout period, the state machine transitions to the NONEXISTENT state 901 along arc 911. At OPERATIONAL state 905, the session is operational and the LSRs may exchange LDP messages.

Maintaining Hello Adjacencies

An LDP session with a peer has one or more Hello adjacencies. An LDP session has multiple Hello adjacencies when a pair of LSRs is connected by multiple links that share the same label space; for example, multiple Point -to-Point Protocol (PPP) links between a pair of routers. In this situation, the Hello messages sent by an LSR on each such link carry the same LDP Identifier. LDP includes mechanisms to monitor the necessity of an LDP session and its Hello adjacencies. LDP uses the regular receipt of LDP Discovery Helios to indicate a peer's intent to use the label space identified by the Hello. An LSR maintains a hold timer with each Hello adjacency, which it restarts when it receives a Hello that matches the adjacency. If the timer expires without receipt of a matching Hello from the peer, LDP concludes that the peer no longer wishes to label switch using that label space for that link (or target, in the case of Targeted Helios) or that the peer has failed. The LSR then deletes the Hello adjacency. When the last Hello adjacency for a LDP session is deleted, the LSR terminates the LDP session by sending a Notification message and closing the transport connection.

Maintaining LDP Sessions

LDP includes mechanisms to monitor the integrity of the LDP session. LDP uses the regular receipt of LDP PDUs on the session transport connection to monitor the integrity of the session. An LSR maintains a KeepAlive timer for each peer session, which it resets whenever the LSR receives an LDP PDU from the session peer. If the KeepAlive timer expires without receipt of an LDP PDU from the peer, the LSR concludes that the transport connection is bad or that the peer has failed, and it terminates the LDP session by closing the transport connection. After an LDP session has been established, an LSR ensures that its peer receive an LDP PDU from it at least every KeepAlive time period to ensure the peer restarts the session KeepAlive timer. The LSR may send any protocol message to meet this requirement. In circumstances where an LSR has no other information to communicate to its peer, the LSR sends a KeepAlive message. An LSR may choose to terminate an LDP session with a peer at any time. Should the LSR choose to do so, it informs the peer with a Shutdown message.

According to one process described with respect to the chart in Figure 10, the following is performed in a network such as that shown in Figure 2. Circuit setup commences at step 1000 following a route lookup via a routing protocol, such as OSPF. An end-to-end path is then defined by the terms of the intermediate switches and the links. TCP/IP Session Level connectivity is established with the first neighbor. A

MPLS/LDP Label Request message is forwarded to the neighbor, for further processing. The TCP/IP Session established is maintained between these two switches. A next switch along the path performs Session Level connectivity with its neighbor. That is, the switch establishes TCP/IP Session Level connectivity with a next hop switch and forwards the MPLS Label Request message to the next hop switch.

The process of establishing TCP/IP Session Level connectivity continues until the Request message is received by the destination switch. At the destination switch, the MPLS/LDP software creates the MPLS/LDP Label Mapping message and returns the same to the source switch, along the same route that the message took to get the Label Request message. According to one embodiment, the Label Request and Label Mapping messages take the same route through the network. This process forms the complete chain of connections that comprise the circuit, and the circuit will become 'Active' when the Label Mapping message arrives at the source switch. Any failure in this path such as that results in the MPLS/LDP software of the switch (node E in the example) detecting the failure. Two additional messages are generated - a Label Withdraw and a Label Release message for the upstream (step 1003) and downstream paths (step 1004). When the source switch A receives the Label Withdraw message, it will perform a reroute of the circuit. However, re-establishment of the circuit necessitates the Control channel to be active.

Assume that the Control channel failed between two switches such as node B and node C at step 1001 in the circuit path, and also assume that the Data Plane between these switches is still active and the circuit is performing data forwarding correctly. TCP/IP Session Level timeout period is fairly long in most systems - on the order of tens of seconds. During this time period, if there is some physical line failure (such as at step 1002), the distributed system will not be able to propagate the Label Withdraw and Release messages upstream and downstream past the Control channel failure. This inability to propagate these messages will result in an unusable circuit. One goal of the mechanism is to bypass the Session Level failure point such as at step 1001, and to communicate Label Withdraw and Release Messages with the source node. This mechanism establishes a temporary TCP/IP Session at step 1007 between the intermediate switch (node C) and the source switch (node A), so that rerouting can commence with minimum delays. In one embodiment, the TCP/IP session is a temporary connection that is deleted after the Label Withdraw and Label Release messages are communicated at steps 1008 and 1009, respectfully. At Step 1010, the TCP/IP session is terminated between nodes A and C, and the circuit may be routed, such as through nodes A, G, H, J and F as shown in Figure 2. While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of the present invention are not limited by any of the above exemplary embodiments, but are defined only in accordance with the following claims and their equivalents.

Claims

1. A system for communication session recovery, comprising:

(a) means for detecting a data session loss along an established connection;

(b) means for detecting that a control session is not operating in an intermediary networking system along the connection; and

(c) means for establishing, responsive to said means (b), a new control session in the intermediary networking system.

2. The system according to claim 1, wherein means (a) includes means for detecting a fault between a pair of intermediary networking systems along the connection.

3. The system according to claim 1, wherein means (b) includes means for determining that a control function is not operating among one or more intermediary networking systems along the connection.

4. The system according to claim 1, wherein means (c) further comprises means for establishing a new TCP session between an ingress system and the intermediary networking system.

5. The system according to claim 4, wherein the intermediary networking system establishes the new TCP session.

6. The system according to claim 4, wherein means (c) includes means for establishing a new TCP session in the intermediary networking system.

7. The system according to claim 1, wherein the control session and the new control sessions are TCP sessions.

8. The system according to claim 1, wherein the portion that is not operating is a failed physical link between a second and third intermediary networking system along the connection.

9. The system according to claim 1, wherein the portion that is not operating is a failed physical link between the first and a second intermediary networking system along the connection.

10. The system according to claim 1 , wherein the portion that is not operating is a failed data connection between first and second intermediary networking systems.

11. The system according to claim 1, further comprising (d) means for transmitting a request to a source switch to release the established connection.

12. A method for communication session recovery between the source system and destination system, the method comprising:

(a) detecting that a portion of an established connection between the source and destination is not operating; (b) detecting that a control session is not operating in a first intermediary networking system along the established connection; and

(c) in response to step (b), establishing a new control session in the first intermediary networking system.

13. The method according to claim 12, wherein step (a) includes a step of detecting a fault between a pair of intermediary networking systems along the connection.

14. The method according to claim 12, wherein step (b) includes a step of determining that a control function is not operating among one or more intermediary networking systems along the connection.

15. The method according to claim 12, wherein step (c) further comprises a step of establishing a new TCP session between an ingress system and the intermediary networking system.

16. The method according to claim 15, wherein the intermediary networking system establishes the new TCP session.

17. The method according to claim 12, wherein step (c) includes a step establishing a new TCP session in the intermediary networking system.

18. The method according to claim 12, wherein the control session and the new control sessions are TCP sessions.

19. The method according to claim 12, wherein the portion that is not operating is a failed physical link between a second and third intermediary networking system along the connection.

20. The method according to claim 12, wherein the portion that is not operating is a failed physical link between the first and a second intermediary networking system along the connection.

21. The method according to claim 12, wherein the portion that is not operating is a failed data connection between the first and a second intermediary networking system.

22. The method according to claim 12, further comprising a step of transmitting a request to the source switch to release the established connection.

23. A computer-readable medium encoded with a program that, when executed on a network communication device, performs a method for communication session recovery, the method comprising steps of:

(a) detecting that a portion of an established connection between a source and destination is not operating;

(b) detecting that a control session is not operating in a first intermediary networking system along the established connection; and

24. The medium according to claim 23, wherein step (a) includes a step of detecting a fault between a pair of intermediary networking systems along the connection.

25. The medium according to claim 23, wherein step (b) includes a step of determining that a control function is not operating among one or more intermediary networking systems along the connection.

26. The medium according to claim 23, wherein step (c) further comprises a step of establishing a new TCP session between an ingress system and the intermediary networking system.

27. The medium according to claim 26, wherein the intermediary networking system establishes the new TCP session.

28. The medium according to claim 23, wherein step (c) includes a step establishing a new TCP session in the intermediary networking system.

29. The medium according to claim 23, wherein the control session and the new control sessions are TCP sessions.

30. The medium according to step 23, wherein the portion that is not operating is a failed physical link between a second and third intermediary networking system along the connection.

31. The medium according to step 23, wherein the portion that is not operating is a failed physical link between the first and a second intermediary networking system along the connection.

32. The method according to step 23, wherein the portion that is not operating is a failed data connection between the first and a second intermediary networking system.

33. The method according to claim 23, further comprising a step of transmitting a request to a source switch to release the established connection.

34. An apparatus comprising:

(a) means for receiving a request to release a connection; (b) means for verifying connectivity to a network communication device;

(c) means for establishing connectivity to the network communication device if connectivity does not exist; and

(d) means for forwarding the request to the network communication device.

35. The apparatus according to claim 34, wherein the request is an MPLS/LDP Label withdraw message.

36. The apparatus according to claim 34, further comprising means for sending an MPLS/LDP Label Release message.

37. The apparatus according to claim 34, wherein the means for establishing connectivity comprises means for a step of creating a TCP session to the network communication device.

38. The apparatus according to claim 34, wherein the means for establishing connectivity comprises means for creating a connection to the network communication device using the TCP/IP protocol.

39. The apparatus according to claim 34, wherein the connection is an existing circuit defined through a plurality of switches.

40. The apparatus according to claim 34, wherein the means for verifying means for verifying if a TCP session to the network communication device exists.

41. The apparatus according to claim 34, wherein the means for forwarding the request comprises a step of forwarding an MPLS/LDP Label Withdraw message to the network communication device.

42. The apparatus according to claim 34, further comprising (e) means for sending an acknowledgement that the request has been granted.

43. The apparatus according to claim 34, wherein the means includes means for determining connectivity to a control function of the network communication device.

44. A method for communication recovery comprising: (a) receiving a request to release a connection;

(b) verifying connectivity to a network communication device;

(c) if connectivity does not exist, establishing connectivity to the network communication device; and

(d) forwarding the request to the network communication device.

45. The method according to claim 44, wherein the request is an MPLS/LDP Label Withdraw message.

46. The method according to claim 45, further comprising a step of sending an MPLS/LDP Label Release message.

47. The method according to claim 44, wherein step (c) comprises a step of creating a TCP session to the network communication device.

48. The method according to claim 44, wherein step (c) comprises a step of creating a connection to the network communication device using the TCP/IP protocol.

49. The method according to claim 44, wherein the connection is an existing circuit defined through a plurality of switches.

50. The method according to claim 44, wherein step (b) comprises a step of verifying if a TCP session to the network communication device exists.

51. The method according to claim 44, wherein step (d) comprises a step of forwarding an MPLS/LDP Label Withdraw message to the network communication device.

52. The method according to claim 44, further comprising a step of (e) sending an acknowledgement that the request has been granted.

53. The method according to claim 44, wherein step (b) includes a step of determining connectivity to a control function of the network communication device.