EP1807954A1

EP1807954A1 - N:1 redundancy scheme for modules with optical interfaces

Info

Publication number: EP1807954A1
Application number: EP05794533A
Authority: EP
Inventors: Derek Underwood; Joseph F. Galibois; Norman Stephen Schwartz
Original assignee: Siemens AG; Nokia Siemens Networks GmbH and Co KG
Current assignee: Nokia Solutions and Networks GmbH and Co KG
Priority date: 2004-10-14
Filing date: 2005-10-14
Publication date: 2007-07-18
Also published as: WO2006040354A1

Abstract

One for N (1:N) redundancy of I/O electronics modules is provided while interfacing to either non-redundant or 1+1 redundant lines. By using a mid-plane architecture which associates front modules with pairs of rear-mounted I/O driver modules and provides a redundancy bus to connect a spare front module to any pair of rear modules , single points of failure are avoided. The rear-mounted I/O modules operate 1+1 redundant to either align with normal SONET/SDH line protection arrangements or to provide 100% capacity redundancy for Ethernet interfaces. In one realization alternative, front and rear modules are paired such that each front module carries half the aggregate capacity of the I/O provided by the 1+1 redundant pair of rear I/O driver modules.

Description

DESCRIPTION

N:l Redundancy Scheme for Modules with Optical Interfaces

The present invention claims priority to the provisional application 60/618,624 filed on October 14, 2004.

Carriers require telecommunication equipment to incorporate redundant hardware with respect to central processors, switching fabrics, and I/O above a certain capacity. The simplest and most common means of implementing this is 1:1 redundancy. One attraction of this arrangement is that it is inherently consistent with the line redundancy normally used in SONET/SDH optical systems, which is commonly called 1+1 Linear APS.

However, because of the cost penalty that 1:1 redundancy incurs, vendors have developed systems that employ N:l redundant hardware. These work reliably for electrical I/O where protection switch relays are used to switchover a spare I/O module onto the same non-redundant cable that carried the signal before failure. These systems, however usually exhibit single points of failure which can take down an entire interface or system.

Optical systems, on the other hand, require a coupling of the N:l module redundancy with the 1+1 APS line redundancy in a way that maintains failure and maintenance state independence between line and hardware and avoids single points of failures. This has proven hard to achieve.

Most optical interface equipment uses 1:1 redundancy. Midplane architectures allow for the coupling of N:l redundant Front hardware modules with non-redundant electrical interfaces (e.g. as in Lucent's Stinger DSLAM) . Problematically, midplane architectures that couple 1:1 redundant lines to N:l redundant hardware have had single points of failure.

What is needed is the ability to couple 1:1 redundancy to N:l redundancy systems and, in addition, to do so without incurring single points of failure.

SUMMARY OF THE INVENTION

In midplane systems, the functionality is usually split between rear input/output (I/O or Line) cards and what are normally referred to as "Front Cards," where the majority of the functionality and cost resides. As mentioned before, the actual optical interface makes use of 1+1 Linear APS, which utilizes two physical connections for each port for reliability purposes. Attaching these connections to 1:1

Line (I/O) cards allows for no single-point failures in the transmission path. Each I/O card normally talks to a single Front card, so pairs of Front cards are required to make the entire system 1:1 redundant with no single failure points. This invention allows 1+1 APS interfaces to attach to 1:1 redundant I/O cards, which in turn attach to N:l redundant front cards, realizing a substantial cost savings (as long as N is equal to or greater then 2) without loss of the overall system reliability.

The invention can also support non-APS connections, such as used for Ethernet, and non-redundant I/O and/or Front card implementations, should this be desired.

By using a midplane architecture and associating pairs of front modules with pairs of rear-mounted I/O driver modules, with half the aggregate I/O capacity placed on each front module, single points of failure are avoided. The rear- mounted I/O modules operate 1:1 redundant to align with normal SONET/SDH line APS protection arrangements. As another consequence of eliminating all single-failure issues, this inventions also allows for Front or I/O cards to be taken out of service, upgraded, or replaced in live systems without any loss of traffic.

A high-speed redundancy system may be used to allow a 1:1 pair of rear-mounted I/O modules to connect to a spare front module in the event of failure or the need to take the front module into a maintenance state. This redundancy system can, in principle be either electrical or optical in nature and be comprised of either bussed or star-wired connections or combinations of any of the above.

The redundancy system can be comprised of one or more redundancy domains. A redundancy domain is defined as a set of N front cards that are commonly protected by a single front card to from an N:l redundancy group. One or more redundancy domains along with the required control and status systems form a complete redundancy system

In addition, the invention allows for different types of I/O connections such as ATM, Ethernet, and TDM to be mixed within the same redundancy domain, including, if desired, allowing one type of front card to be provisioned 1:1 redundant while another type is provisioned N:l. In all cases for maximum reliability the I/O (Line) cards are usually configured as 1:1 peers, although they can be configured in a non-redundant fashion if desired.

The invention can be applied to any midplane-based telecom or datacom system; for example, the Advanced Telecommunications Computing Architecture (ATCA) shelf as specified by PICMG3.0 contains the provision of a midplane in Zone 3, over which a redundancy system as described herein can be implemented..

The realization here allows both front and rear modules to access tracks in the midplane for interconnection between non-aligned positions for control and power as well as the I/O traffic. DETAILED DESCRIPTION

The general feature of the proposed midplane provides the necessary connections between front cards and Line cards, which, for example in an ATCA system are referred to as rear transition modules (RTM' s) . To support front card and RTM redundancy modes, the midplane should also provide cross connections between line cards and RTM' s, between line cards, and between RTM' s .

It should be appreciated that the midplane described here supports a sixteen-slot chassis. Another variation on the midplane may support other sizes, such as a fourteen or twenty-slot chassis. Reduced midplanes may be advantageous to allow the installation of other line cards that would not fit with a full midplane installed. In these cases, such a reduced version should support, for example, some slots that are not within any midplane redundancy domain

Figure Ia illustrates a functional description of the construction of such a midplane 100. In particular, a midplane unit connected, for example as a 1:1 node pair, is shown. Here, there is shown odd and even slots 102 a, b for the RTMs and odd and even slots 104 a, b for the Front Cards. As can be seen, the data paths 106 between the front cards and the RTM' s form a 1:1 node pair. This pairing supports 1:1 node RTM redundancy and 1:1 node front card redundancy.

The midplane concept also supports N:l line card redundancy. A special N:l redundant RTM would be installed behind the protection node line card, but the protection node line card is identical to the protected line cards and the N:l slot can also be used for 1:1 module redundancy. Figure Ib shows the data path connections at the N:l redundant card slot. Shown are the connections to the 1:1 pair slot, wherein the N:l slot is capable of supporting 1:1 line card redundancy, that is instead of N:l line card redundancy. With this configuration, regardless of which redundancy mode the N:l line card is working in, there is always 1+1 APS protection for the cables attached to the pair slot.

Figure 2 is a simplified diagram 200 depicting a single pair of 1:1 front cards. In this example, the APS fibers are connected to separate RTM' s, but the traffic from both RTM' s 202 a, b is routed over the midplane 204 to the working TDM front card 206 a. If the working TDM card fails, the traffic is routed over the midplane to the Protection TDM front card 206 b. In both cases, if either RTM or either fiber fail, the traffic is maintained via the remaining RTM and remaining fiber. Note that in this case, the actual redundancy bus connections are not required. Also note, that this figure shows two front cards and two RTM I/O cards servicing a single APS interface (which consists of two fibers.)

Figure 2b is a simplified diagram depicting two working TDM Front cards (for example) protected by a protection TDM card in a different slot. As in figure 2a, the two APS fibers

(working and protection) for each APS interface are attached to 1:1 redundant RTM cards 202 a, b, and the working and protection traffic is routed over the midplane 204 via the direct and crossed over links 208 to a single Front card, as indicated by the solid and dashed lines in the drawing. If any fiber or RTM fails, the other fiber or RTM will maintain the traffic to both Front cards, as in the previous figure. If one of the two working Front cards fail, then the working and protection traffic that was connected to that card is routed over the redundancy system to the protection card, preventing any loss of service. Note that in this example, three front cards 206 a, b, c and three RTMs provide fully protected service for two APS interfaces (one for each working front card) , which represents a savings of one Front and RTM card when compared to a 1:1 implementation (figure 2a) which would require four front cards and four RTMs. This saving obviously increases as N gets larger. For example, four interfaces require a total of eight Front and eight RTM cards in a 1:1 system, but only 5 front and 5 RTM cards in an N:l system.

A midplane could have one or more redundancy domains within a redundancy system. Each domain could be implemented as a bus or as a set of star-wired signals, or as a hybrid consisting of both bused and star-wired signals.

For descriptive purposes, in the following discussion a star- wired redundancy domain is described. Each star interface may comprise two channels between each node RTM and the N:l redundant RTM. The first channel may consist of four ports, each of which contains two sets of signals (one transmit and one receive), for example. The second channel, for example, would consist of eight, bused single-wire Signal Detects. When directed by the control and status system on the active hub (as will be explained later) , an RTM will steer one group of ports onto its redundancy Star interface. Four signal detects in this example are driven by each RTM, those corresponding to the group of ports being driven on the other channel.

As an example of the control and status system, in an ATCA- based implementation the redundancy system is under control of the active hub. There are two HUBs in the system but only one of them can be active at any time, and this card must be unambiguously identified to the rest of the system. This is done by identifying a set of slots whose residents are called ARC Masters. Each ARC master distributes a signal indicating its choice for the active HUB. All cards (including the ARC Masters) implement circuitry that examines the ARC signals and selects the Active HUB. An arbitrary redundancy interface may convey intent from ARC masters to ARC subscribers as to which hub's resources the ARC masters believe should be used. In general, both hubs, all node line cards and all RTM' s can be ARC subscribers.

When a node front card fails and N:l redundancy protection is being used, data channels must be quickly steered from the

RTM' s that were working with the failed node front card to the protection front card. The four working lines (in this example) and the four protection lines must be delivered by their RTM' s via interfaces to the protection front card. The steering mechanism in this solution is the MRC interface.

In an midplane-based ATCA system, for example, each node front card sends to each hub Health signals, conveying a summary of the card's ability to perform its functions. When a node front card can no longer perform its functions, the hub card uses control signals to each appropriate RTM to instruct it to drive its redundancy interfaces so that the traffic routed to the protection front card via the redundancy connections.

In an ATCA system, for example, the midplane of the solution provided here may be particularly applied to a zone 3 connector area. The zone 3 connector area in the ATCA specification is defined as a 95mm long region above the top of the ATCA backplane primarily intended for the attachment of RTM' s directly to their corresponding front cards. A direct connection like this is inadequate for supporting 1:1 I/O and does not support N:l redundancy at all, hence our midplane invention. Although any type of external physical connection could be used with the solution provided herein, it would be also desirable to support 1+1 linear APS protection for SONET/SDH I/O connections with either 1:1 or N:l redundant front cards. To achieve that, the described ATCA example solution implements a zone 3 midplane. This will allow I/O traffic to cross between slots and to allow the transport of the I/O traffic to a protection card as described above

Note that the design intent of the system is to allow a single protection card to provide redundancy for any number of identical working cards, which leads to substantial cost savings as the number of working cards increases. It will be appreciated that Figures 2a and 2b are simplified drawings. The actual zone 3 midplane consists of more signals then those shown in the drawing, as be delineated in the following table for a possible, example ATCA-based implementation:

Zone 3 Midplane Signals

The table above defines a lot of signals, but it must be remembered that in this example, not all of the signals in the table go to all of the zone 3 connectors. The following table provides, in summary, an example of a possible signal allocation on a slot type basis:

Zone 3 Connections by Slot

spares (tbd)

Non I/O 3 H - spares system redundancy (3), spares (tbd) card

This last table makes the assumption that there will NOT be a special slot set aside for a protection card. If the decision is made to dedicate a slot for I/O front card protection then the number of zone 3 pins required for the working front cards will be considerably reduced. This invention applies equally well to N:l systems that are implemented with special slots for the redundant card in a redundancy domain or systems that can use any slot to hold the redundant card in a redundancy domain.

As alluded to in the sections pertaining to the ATCA-based example system, the TDM Interface Card, the Optical RTM Card, and the zone 3 backplane, the TDM subsystem supports either a full 1:1 redundancy via duplicate TDM front Cards and Optical RTM' s, or a system that supports 1:1 RTM/APS redundancy and N:l TDM front card redundancy via the zone 3 midplane. The 1:1 system is intended to support hitless switching while the N:l system is intended to support no loss of stable calls at a much lower cost point. Figure2a shows the 1:1 implementation and Figure 2b shows the N:l implementation.

In Figure 2a (1:1 Redundancy) note that each front board/RTM duo supports one half of the Linear APS fiber connection. The zone 3 backplane connections are still present but are largely unused, with bearer traffic only traversing the two outermost long connections in the drawing that interconnect an RTM directly to the Front card in the same slot. In this configuration, signals between the 1:1 peer front cards, for example the update channels on an ATCA zone2 midplane, may be used for 1:1 synchronization and database support. The two front cards plus the two RTM' s form a fully redundant pair of boards that back each other up, and no single hardware failure (Fiber, RTM, or front card) will cause any loss of stable calls or an interruption of service. It will be noted that this is relatively a high level of reliability, at relatively high cost.

In Figure 2b (N: 1 redundancy) it should be noted that each of the front cards supports a complete linear APS fiber interconnect (working and protection fibers) . For both fiber interfaces, the working and protection fibers for a given channel are attached to different RTM' s so that there are no single points of failures in the APS protected components, but both of the working front TDM cards are protected by the single protection TDM front card via the zone 3 redundancy system. The system shown supports twice as many channels as the 1:1 system with three front cards and three RTMS, whereas a 1:1 system of the same capacity would require four front cards and four RTMs .

The cost savings of the N:l implementation increase substantially as the number of front TDM interface cards goes up. In a typical N:l implementation, a single protection TDM front card will protect up to N working TDM cards. Note that for example, the actual hardware implementations of a TDM front card, Optical RTM, and zone 3 midplane could support more then one APS interface per slot, in both 1:1 and N:l configurations. The above figures show only single interfaces for simplicity. Also note that the same RTM and TDM front cards support either of the two different redundancy configurations. . Also note that the drawings are applicable to any SONET/SDH interface using 1+1 linear APS and are also applicable to other redundant connections, such as those found on Ethernet interfaces.

Now turning to the physical layout of the midplane, a practical example is given here for an example ATCA-based system. It shall be appreciated that this example is but one of many layouts that may be applied. The outline of the ATCA Zone-3 midplane is dependent on the requirements of the manufacturer of the ATCA chassis. Hub slots contain, for example, (eight-row) Teradyne VHDM and (eight-row) VHDM-HSD connectors. Node line slots and all RTM' s contain (eight-row) , in the example, VHDM-HSD connectors. All slots use Teradyne VHDM-family power connectors .

Concerning power requirements, to the RTMs, one possibility in an example ATCA-based system may be that each front card provides two, 12-volt power signals to the midplane. Those power signals would then be cross-connected on the Zone-3 midplane and then diode -or'ed on each RTM. The RTM' s thus derive their power from the line card in front and/or the line card in the adjacent slot, which provides each RTM with 1:1 redundant power feeds. A scheme such as this example works equally well for 1:1 and N:l front cards, as long as the redundant card in an N:l system also can power the RTMs.

Beyond the use of a midplane, there are implementation options which will be set forth here. Generally the objectives are to place the maximum cost on the electronics modules while at the same time minimizing the total amount of electronics and maximizing reliability. The chosen approach here for optical I/O places all the optical components on the rear transition modules but the minimal amount of electronics. In this ATCA-based example, the interface between the two are the high-speed serial electronic representation of the optical signals which allows a common rear transition module to serve different optical rates, TDM, ATM and packet over SONET structures, and even optical gigabit and 10-gigabit Ethernet.

An equivalent example for electrical interfaces in an ATCA- based system may be to place the line drivers and transformers on the rear transition modules while keeping the framers on the front electronics module. Alternatively, SONET/SDH and high-rate electrical interface framers and gigabit Ethernet phys can be placed on the rear transition modules, although such an approach has the disadvantages of requiring a proprietary chip-specific interface between front and rear modules (typically parallel) and requires more power on the rear transition module possibly necessitating an on- card DC converter.

Two basic options exist for powering the rear transition modules used in any N:l redundant system. For example, the ATCA (PICMG3.0) philosophy is to power any electronics on the rear transition module from the front module with all the required DC-DC converters on the front module. In the example discussed here, we extend this so that the rear transition module can be powered either from the front module (s) it is working with under normal operation or from the spare front module under failure conditions in a diode-OR arrangement that prevents any power drain from the spare front module while a normal front module is able to supply power.

An alternative to the example discussed above may be to power the rear I/O module directly from a (for example) -48V power strip placed in the midplane.. This allows for higher-power components on the rear I/O modules but at the expense of requiring DC-DC converters on the rear I/O modules.

The decoupling of the front and rear modules would tend to involve new mechanisms for fault detection. Specifically, the rear I/O modules need to have a means of detecting failure of the front module and reporting such a failure to the shelf manager. Additionally, the risk of latent faults in the redundancy system must be minimized.

Options exist for how the switchover is controlled following detection of failure. One example approach recognizes that failover can only succeed if not only the shelf manager wants it to happen but also the spare module is ready for a switchover. One possibility is therefore to provide hardware control signals in the midplane from the spare front module to all rear modules to indicate its readiness and individual signals from shelf manager to each rear transition module to control switchover to the spare front module.

It shall be appreciated that the description here offers but examples and that changes to the invention may be made within the spirit and scope of the invention.

Claims

1. An apparatus for telecommunications equipment employing a mid-plane architecture that avoids single point (s) of failure, comprising:

front I/O electronics modules,

pairs of rear-mounted I/O driver modules,

a mid-plane providing a set of Y interconnections between the front I/O electronics modules and pair(s) of the rear-mounted I/O driver modules,

wherein each front I/O electronics module is associated with one pair of the rear-mounted I/O driver modules such that the front module sends and receives signals to both rear-mounted I/O driver modules in the pair,

wherein the full traffic capacity of the front module is distributed to one or more of the rear-mounted I/O driver modules of the associated pair such that single point (s) of failure are avoided.

2. The apparatus of claim 1, wherein the full traffic capacity of the front module is divided between the two rear- mounted I/O driver modules in the associated pair.

3. The apparatus of claim 1, wherein the full traffic capacity of the front module is transferred in its entirety through both rear-mounted I/O driver modules of the associated pair in an active-active worker-protection redundant configuration.

4. The apparatus of claim 1, wherein the full traffic capacity of the front module is transferred through only one of the rear-mounted I/O driver modules in the associated pair while the other of the associated pair operates in passive standby.

5. The apparatus of claim 1, wherein the rear-mounted I/O driver modules operate in a 1+1 redundant configuration to align with normal SONET/SDH 1+1 (worker and protection) line protection arrangements.

6. The apparatus of claim 1 where the interconnection between the front module and rear-mounted I/O driver modules is a serial or parallel electrical representation of a plurality of optical or electrical system input and output signals physically terminated at the rear-mounted I/O driver modules.

7. The apparatus of claim 1 where the interconnection between the front module and rear-mounted I/O driver module (s) is an optical signal representation of a plurality of optical or electrical system input and output signals physically terminated at the rear-mounted I/O driver modules.

8. The apparatus of claim 1 where the front I/O electronics modules are paired wherein the pair of front I/O electronics modules are associated with the pair of rear-mounted I/O driver modules via a mesh interconnection such that each of the front I/O electronics modules sends and receives signals to both of the rear-mounted I/O driver modules in using straight-through and crossover interconnections.

9. The apparatus of claim 8 wherein approximately half the aggregate capacity of the associated pair is placed on each of the front I/O electronics modules of the pair of front I/O electronics modules.

10. The apparatus of claim 1 wherein the front module monitors a condition of the two rear-mounted I/O driver modules and the integrity of the signals received from the rear-mounted I/O driver modules.

11. The apparatus of any of claims 4 through 10 wherein the interconnections between front and rear-mounted I/O driver modules consist of control, clock, and inventory bus signals in addition to the I/O signals.

12. The apparatus of any of claims 4 through 11 wherein the interconnections between front and rear-mounted I/O driver modules also include DC power adhering to the same Y interconnections through the mid-plane between front and rear-mounted I/O driver modules as the I/O and other signals.

13. The apparatus of any of claims 4 through 12, further comprising a high-speed electrical or optical redundancy bus, allowing a 1+1 pair of rear-mounted I/O driver modules to connect to a spare front module in the event of a failure or other removal from service of the front module.

14. The apparatus of claim 13 wherein the high-speed redundancy bus is a serial or parallel electrical representation of a plurality of optical or electrical system input and output signals physically terminated at the rear- mounted I/O driver modules together with any or all control, clock, and inventory interfaces between front and rear- mounted I/O driver modules.

15. The apparatus of claim 13 wherein the high-speed redundancy bus is an optical signal representation of a plurality of optical or electrical system input and output signals physically terminated at the rear-mounted I/O driver modules together with control signals and clock.

16. The apparatus of claim 13 wherein each rear-mounted I/O driver module monitors the health of the one or more front

I/O electronics modules it connects together with the integrity of the signals it receives and reports anomalies to a system manager for control of front module switchover.

17. The apparatus of claim 16 wherein each rear-mounted I/O driver module acts upon control instructions received from the system manager to switch the signals from a front module going out of service to the spare front module or back from the spare front module to a front module being returned to service.

18. The apparatus of claim 13, wherein a second spare module can be installed to create two N+l redundancy groups and thereby provide sparing of two technically different front module types in a same chassis.

19. The apparatus of claim 13 or claim 18, wherein rear- mounted I/O driver modules may be non-redundant.

20. The apparatus of claims 8 and 13, wherein one or more pairs of front I/O electronics modules in the chassis operate in 1+1 active-active redundancy to provide for hitless failover while other front I/O electronics modules in the same chassis are configured in one or more 1:N redundancy groups .

21. The apparatus of claim 1, wherein the mid-plane architecture is integrated in an Advanced Telecommunications Computing Architecture (ATCA) shelf as specified by PICMG 3.0.

22. The apparatus of claim 1, wherein the rear-mounted I/O driver modules operate in a 1+1 redundant configuration to provide a 100% capacity redundancy on packet interfaces that do not ride on SONET/SDH infrastructure such as Ethernet.

23. A method for telecommunications equipment employing a mid-plane architecture that avoids single point (s) of failure, the mid-plane architecture including front I/O electronics modules and at least one pairs of rear-mounted I/O driver modules, comprising the steps of: providing a set of Y interconnections between the front I/O electronics modules and pairs of the rear-mounted I/O driver modules,

associating each front I/O electronics module with one pair of the rear-mounted I/O driver modules such that the front module sends and receives signals to both rear-mounted I/O driver modules in the pair

where the full traffic capacity of the front module is distributed to one or more of the rear-mounted I/O driver modules of the associated pair such that single point (s) of failure are avoided.