HIGH SPEED COMBINATION ROUTER-SWITCH
Field of the Invention
The present invention relates to a combination router and switch in computer communications. More particularly, it relates to a router and switch, which can operate with different communication protocols and with reduced data loss.
Discussion of Related Art
Since the advent of the telegraph and telephone, people have been setting up networks to transfer information from one location to another. Different types of networks have been created to transfer different types of information in varying manners. For example, with the telegraph each message was passed to every location on the wire connecting the locations. The operator had to determine whether the message was direct to his location. Some operators might need to relay a message onto a separate network of wires when the destination was further away. With telephone networks, switches were introduced to provide point-to-point connections. The system would determine the destination location and set switches to provide a direct wire connection between the originating and destination locations.
With the advent of computers, more complex networks have been devised. Point-to-point systems proved inadequate with large quantities of small information transfers. In particular, significant resources and overhead is used to set up point-to-point connections within the system. When large amounts of data are being transferred at once, the overhead is relatively small compared to the data. However, with small amounts of data in each communication, the overhead becomes a large percentage of the total transmission. Therefore, new structures for interconnecting locations have been developed to handle different types of information transfers. However, different structures continue to be used based upon the desired structures and patterns of the information traffic. Differences can be illustrated in the structures of the seven-layer ISO representation of communications. The second and third layers are most relevant to the present invention. The first layer, physical, represents the mechanism for connecting devices. Typically, the physical layer includes different types of media, such as twisted-pair, coaxial cable, fiber optic cable or wireless medium. The second layer, data link, represents how the physical layer is interconnected to transfer information. Common systems include asynchronous transfer mode (ATM), frame relay, point-to-point, and multiprotocol label switching (MPLS). Finally, the third layer, network, defines how the structures in the data link layer are controlled. For example, the
network defines the settings for switches to transfer the information to the desired location. Internet protocol (IP) is one type of network structure. However, with many different choices for the data link and network layers, integration into a single network is difficult. Furthermore, each type of network has certain advantages and disadvantages, which could be beneficial in connection with transferring different kinds of information. Therefore, a need exists for a system that can incorporate the advantages of different network designs based upon the desired transfer. FIG. 1 illustrates three principal portions of a network: access, aggregation and backbone. The access portion provides a connection between customer devices and the network. Depending upon the type of network, the access portion may include a variety of devices having different communication formats. Typical devices include telephones, modems, computers, and different types of digital accesses. For example, the access portion may include a frame relay, a telephone line, an ISDN connection or a xDSL connection. With different data types (from different devices), the network has different organizations.
The aggregation portion connects each of the access portion devices to each other, and to the backbone portion. Depending upon the devices, different structures may be used for the network, as illustrated in FIG. 2A. Since different protocols are used, the same equipment cannot handle all types of data transfers. Thus, as illustrated in FIG. 2 A, information in the form of switched voice or data, such as from a telephone connection, uses a point-to-point switched network common for telephony. Frame relay structures can be transferred through either an ATM network of switches or through a system operating with routers. xDSL data is passed through a router-based system, such as the Internet. With the different structures, each type of data needs its own network.
The aggregation portion and the backbone portion may be formed of a combination of routers and switches, as shown in FIG. 2B. Routers are used to determine a path through the network to the destination location. Each router includes the arrangement of at least a portion of the entire network. The router monitors messages sent to it to retrieve information about the final destination. Using the stored arrangement of the network, the router determines the next hop in a transmission to either another router or to the final destination. Subsequent routers also select next hops until the final destination is reached. Switches are used to transfer data coming on one data channel to another, outgoing data channel. The data channels may be different wire, or simply different paths or channels operating over a single wire. When a router selects a next hop, the switches along a path must be set to pass the message to the next router.
There are many different types of routers and switches, which operate in different manners. FIG. 1 illustrates a network wherein switches and routers are paired. In such a network, data messages are called packets. Each router, after determining the next hop for a packet, sends to its single switch to transfer the packet. Alternatively, the information may traverse several different switches between hops. Also, under some structures, the access devices perform the routing function, and only switches are used between access devices. In addition to different physical structures, different systems transfer different forms of packets. Some, like the ATM switch, use fixed sized units or cells. A longer packet is divided into cells by the routing function, and each cell is sent in sequence to the network. Within solely a switched network, each switch simply forwards each cell independently, i.e., without regard to the other cells of the packet. It is the responsibility of the network's egress routing function to reassemble all of the cells back into a complete packet. When routers are used in the network, such as with IP, the next hop is determined based upon the information in the packet. The router receives and stores all of the cells corresponding to a packet. The packet is then reformed, checked for errors, and processed to determine the next hop. The router will then form the packet back into cells and transfer it through the network to the next router. This process results in significant delays in passing the packets through the network.
A new system has been proposed which provides "cut-through" switching for IP packets on a network including routers and ATM switches. When a router recognizes an IP packet, it informs both the transmitting side and receiving side devices to use a different path. Typically, in this context, a path is implemented as a particular channel or VPI/VCI value between ATM devices. Once the new path is set up, the entire IP packet is sent without the usual processing. While the entire packet passes more quickly than in conventional routers, the time to set up and take down the new path slows the process down. Also, this system requires changes in the existing equipment within a network, for devices that can communicate to change paths. It cannot operate with other types of routers and switches.
Furthermore, conventional routers use a local view of the system; they only route packets to immediate next hops. If a link goes down, then the router sends a message back to the originating device and other routers to reroute future packets through different links. However, the packets that have already been sent towards that router are typically lost, or need to be resent. Finally, different types of information have different transfer requirements. For example, some data may need to be very accurate, but the time to transfer the data may not be important.
Other data, may not need to be as accurate, but the time to transfer it may be very important. In other cases, certain quantities of data may need to be transferred at specified rates. Therefore, some networks, such as ATM networks, have a quality of service feature, which transfers data according to specified priorities. However, IP structures only include simplified mechanisms to differentiate types of service. The IP structure includes a priority designation. This structure is significantly less robust than ATM quality of service, and does not allow for guaranteed delivery or bandwidth capabilities. Therefore, a need exists for a system that permits IP data to be transferred with improved quality of service requirements.
Summary of the Invention
The present invention overcomes the deficiencies of the prior art, to a great extent, through the use of an integrated router and switch. According to one aspect of the invention, the router portion of the combination is by-passed when it is not necessary. This reduces the time through the integrated device and improves performance. According to another aspect of the invention, on-the-fly, inline routing is used for packets. Packet types may include, but are not limited to the IP. When sufficient information from the packet has been retrieved, a new path may be determined. For example, in an ATM switch environment, the first cell or cells of an IP packet are retrieved to obtain the information in the IP header. The router portion of the integrated device makes a routing decision and selects a new path. In conjunction with setting up new paths, according to one embodiment of the invention, ATM quality of service can be set for the path based upon various features of the IP packet, such as source address, destination address, and priority information. After the path is selected, all remaining cells of the IP packet are passed through the integrated router/switch on the path without the need to store and reform the entire packet. Information in the IP packet can be adjusted on-the-fly as necessary to move to the next hop. According to another aspect of the invention, dynamic routing is used to by-pass links that fail during a transmission.
Brief Description of the Drawings
FIG. 1 illustrates the arrangement of routers and switches within a network according to existing art.
FIG. 2A illustrates connections within a central office according to existing art. FIG. 2B illustrates connections within a point of presence according to existing art.
FIG. 3 illustrates the arrangement of an integrated router and switch according to an embodiment of the present invention.
FIG. 4 illustrates the arrangement of an interface card according to an embodiment of the present invention. FIG. 5 illustrates the use of the present invention in a network.
FIG. 6 illustrates the structure of a switching core for interconnecting multiple integrated router/switches of the present invention.
FIGS. 7 A and 7B illustrate adaptive routing according to the present invention.
Detailed Description
FIG. 3 illustrates the hardware of an integrated router and switch that jointly operate according to an embodiment of the present invention. The integrated router/switch 300 includes various modules or cards for input, output and switching functions. Interface cards 310, 311 connect to the inputs and outputs of the integrated router/switch. The routing function is performed in the interface cards 310, 311 as discussed below. According to an embodiment of the invention, up to four interface cards 310, 311 can be connected within a single integrated router/switch 300. Two switch modules 320, 321 are in the integrated router/switch 300 to switch between the different interface cards. One switch module 321 acts as a redundant backup in case of failure of the other switch module 320. The switch modules 320, 321 are connected to each of the interface cards 310, 311 by four 622 Mbps lines 340, 341 , 342, 343. Thus, each integrated router/switch 300 provides switching capability of 10 Gbps. The switch module 320 is connected to an uplink card 350 in order to interconnect multiple integrated router/switches for higher data transfer rates.
Each of the interface cards is connected to a management card 360. The management card 360 maintains the routing tables necessary for determining the next hop. The management card may communicate with management cards of other connected integrated router/switches to determine the routing table. Alternatively, as discussed below, the routing tables may be generated by a switching core and transferred to each management card. According to an embodiment of the invention, the connection 370 between the interface cards 310, 311 and the management card 360 is a Fast Ethernet line. A peripheral card 365 connected to the management card 360 provides the management ports for monitoring and controlling operations. The peripheral card 365 may be connected to one or more management outputs 368. The
management outputs 368 could include a Fast Ethernet connection to transferring data to or from the management card 360. It may also include a terminal or management system, so that an operator can check and control operation. It may also include a hard drive or other storage device for maintaining data generated or used by the management card or integrated router/switch.
The structure of an interface card 310 is illustrated in FIG. 4. The interface card includes four cell forwarding engines (i.e., CFEs 421, 422, 423 and 424), which provide the integrated routing function. Each of the cell forwarding engines is connected to a line card 410, 411, 412,
413. The line cards 410, 411, 412, 413 are connected to the different input devices, such as computers, computer networks, or other routers and switches. Different line cards are used for different types of inputs, and can include multiple ports. The cell forwarding engines 421, 422, 423, 424 are connected to a processor 430. The processor 430 controls operation of the interface card within the integrated router/switch. The processor 430 is connected through the Fast Ethernet line 370 to the management card 360, and through a control bus 470 to the switches. The management card 360 communicates with the network to maintain a routing table. The routing table is also transferred to the processor 430 in each of the interface cards 310, 311, where it is stored in a memory 440 connected to the processor 430. Any changes to the network structure are reflected in the routing table. The processor 430 also sets up paths to each next hop as needed by the CFEs, and stores information about the paths in the memory 440. The cell forwarding engines 421 , 422, 423, 424 access the routing table and path information in the memory 440 through direct connections 441 to determine a next hop and the appropriate path for that hop and quality of service. The cell forwarding engines also include a forwarding database derived from the routing table. The forwarding database includes information about existing paths in relation to certain qualities of service. Routing information is determined by the cell forwarding engines and transmitted to the processor 430 which sets up the paths in the switch 320.
Additionally, the interface card 310 includes connections to bypass the cell forwarding engines, in the forward direction. This allows the integrated router/switch to operate with both protocols that use routing and that do not use routing, without significant delays. If a cell arrives that is not to be routed, it is passed to the switch 320 through the forward direction bypass 450, rather than to the cell forwarding engine. The switch forwards the cell based upon the ATM paths previously defined. The data outputted from the switch travels through the interface card
on the reverse direction bypass 461 to the line card 410. Since routing is not needed after switching, the cell forwarding engine 421 is not used.
FIG. 5 illustrates the use of the integrated router/switch in a network. The integrated router/switch 300 can be used to replace any router and switch combination within a network. A central office or point of presence (POP) 500 provides a connection between external devices and the network. A POP 500 includes an access portion 510 and one or more routers, switches or integrated router/switches 300. The access portion connects the POP 500 to external devices (not shown) such as telephones, modems, computers, etc. As noted above, different types of devices using different protocols can be connected to a network. Therefore, the access portion includes different connections, such as a frame relay 511 , a dial access 512, and xDSL access 513. A integrated router/switch 300 can operate with different inputs and can provide a single connection to each of the access portion 510 devices. The integrated router/switch 300 provides communications between the external devices connected to the POP 500. It also provides connections to a network backbone 550. A network backbone 550 is a collection of routers 540 and switches 541 which transfers messages between different POPs 500, 501, 502. Since the integrated router/switch 300 has features of both routers and switches, it can be used in the network backbone in place of routers and switches. Alternatively, integrated router/switches 520, 530 can be used with other routers 540 and switches 541 to form the backbone. Therefore, the integrated router/switch can be incorporated into existing networks without changing the entire structure.
The integrated router/switch 300 of the present invention is scalable to obtain a greater number of connections and increased bandwidth. FIG. 6 illustrates a switching core 610 for interconnecting multiple integrated router/switches 300. As illustrated in FIG. 6, the switching core 610 includes a switch 640, multiple downlink cards 651, 652, 653, 654, a processor 660, a memory 670, and a peripheral card 680. The downlink cards 651 , 652, 653, 654 in the switching core 610 connect to the uplink cards in the router/switch 300. When the next hop is on another integrated router/switch, the cells are transferred from an uplink card of one integrated router/switch to the switch 640 of the switching core 610 and back out to an uplink card of another integrated router/switch. When a switching core is utilized, the routing information is collected and stored in the memory 670 of the switching core. The processor 660 communicates through the switch to other devices on the network to determine the network configuration and create the routing table. The
routing table can then be downloaded from the memory 670 through the switch and downlink cards to each of the integrated router/switches 300. In this manner, the routing information is maintained much more efficiently than in existing systems. In order to maintain proper routing information, the structure of the network must be communicated to each router or node in the network. When each router determines and maintains its own routing table, the necessary messages to communicate the entire structure is O(n2), where n is the number of routers. However, with the switching core, the collection of integrated router/switches 300 connected to the switching core 610 acts as a single node. Furthermore, the integrated router/switches 300 and switching core 610 do not need to be colocated to achieve this result. Therefore, a single switching core, and its associated integrated router/switches, can operate as a local network over a dispersed area.
Additionally, as illustrated in FIG. 6, a single switching core 610 can be connected with other switching cores 620, 611 to provide redundancy and increased interconnectivity. Each switching core 610 includes a peripheral card 680 that provides a Fast Ethernet interface. The Fast Ethernet interface can connect a redundant switching core 611 to the primary switching core 610. If there is an error in the switching core 610, the switching core 611 contains all of the information necessary to continue operation. Additionally, the redundant switching core 611 contains connections (not shown) to the integrated router/switches in the same manner as for the primary switching core 610. According to one embodiment of the invention, a switching core 610 includes sufficient downlink cards 651, 652, 653, 654 and switch fabric 640 to connect to four integrated router/switches (64 lines). If greater switching capacity is needed, additional switching cores 620 can be connected to one or more lines from one of the integrated router/switches 300. In this manner, each integrated router/switch can communicate through one or more switching cores and integrated router switches to reach the proper line card 410 for an output. When multiple switching cores are used, only one memory 670 needs to be used to maintain the routing table for the entire structure. The routing table information could then be transmitted from the first memory to memory locations in the other switching cores and to all integrated router/switches. A single switching core with four integrated router/switches can provide 40 Gbps capability. Sixteen switching cores 610, 620, 630 can be combined to obtain a total bandwidth of 640 Gbps within a single structure.
Operation of the integrated router/switch will now be described. Cells of incoming IP packets are received from a line card 410 and passed through a cell forwarding engine 421. The cell forwarding engine 421 reads the header of the IP packet contained in one or more cells.
Assuming that an appropriate path in the switch 320 has already been set up, the cells are modified by the cell forwarding engine, as necessary, to represent the path and to accommodate on-the-fly routing. As additional cells of the packet are received, they are also modified, as necessary, by the cell forwarding engine. After each cell (or cells) is processed by the cell forwarding engine, it is passed to the switch 320. In the switch 320, the cells are redirected to the appropriate line for an outgoing port of a line card 410. Of course, if a switching core is used, the cells may be directed to the uplink cards to be switched by the switching core.
If an appropriate path has not been set up in the switch 320, the cell forwarding engine determines the next hop and transfers the relevant packet and next hop information to the processor 430. The processor 430 sets up the path in the switch 320 through the control bus 470. Then the cell forwarding engine continues its processing, as described. Typically, the integrated router/switch operates like an ATM switch with a combined router. The software running the integrated router/switch sets up paths in the same manner as for any ATM switch and router. The cell forwarding engines 421 read the headers for the cells and determine the outgoing virtual path and virtual channel for each cell using the stored forwarding table or routing table. However, as noted above, the router function can be bypassed if not needed. For non-routed ATM cells, once the ATM paths are set up, and the switch has been configured, cells can pass without delay. Since the routing does not have to be checked for each cell, the latency of the cell is improved and consistent.
The integrated router/switch also performs IP routing in two different modes. In a first mode, the integrated router/switch performs IP routing over ATM, as with most conventional ATM switches and routers. This utilizes a store-and-forward technique. Incoming cells are reformed into complete IP packets. The IP information in each packet is examined and the next hop for the packet is determined. The packet is then broken back down and the cells transferred to the next hop. As noted previously, this store-and-forward technique can be slow, particularly if any cells are lost. Alternatively, the integrated router/switch 300 performs "on-the-fly" routing of IP packets. When a cell forwarding engine 420 recognizes the first cell or cells which contain the IP header of an IP packet, it selects a path to the next hop for that IP packet. This path applies to
the particular IP packet being received. Other paths may also pass to the same location, but the cells of the IP packet must be serially passed in the correct order so that the IP packet can be formed again.
Certain bits of the IP packet are checked and modified to perform proper routing. For example in IP version 4, the Time-to-Live (TTL) field is typically decremented by one and the IP header check sum is recalculated and modified in the first ATM cell as it passes through the CFE. If the IP packet is modified the ATM AAL5 CRC would be checked and a new CRC computed and transmitted in the last cell of the packet. Therefore, the cells merely pass through the integrated router/switch 300 without the need to store-and-forward the IP packet. On-the-fly routing has been described in the context of IP packets which are divided into multiple ATM cells. A similar process can also be generally applied to routing of ordinary IP packets to reduce retransmission delays. With ordinary IP packets, the entire packet would be received at once, not as separate cells. However, reception and transmission of the packet takes some period of time. Typically, the entire packet is received and stored before it is retransmitted to the next hop. According to the present invention, a router can begin transmission of the packet to the next hop once the header has been received and before the entire packet is received. As with a packet divided into cells, the cell forwarding engine, or other routing engine, would modify the appropriate portions of the packet for routing and retransmit them. If errors occur later in the packet, similar to in later cells, then an error indication can be sent and the transmission stopped. Therefore, on-the-fly routing works the same for contiguous portions of a large packet as with consecutive cells which form a packet.
Furthermore, the integrated router/switch can operate to provide an IP multicast, without the need to perform an ATM multicast. For an IP multicast destination, the cell forwarding engine 421 could select a path that switch 320 could pass to multiple locations. The egress paths from switch 320 are not required to be part of an ATM multicast connection. Instead, those paths could be ATM point-to-point connections.
The integrated router/switch operates on layers 2 and 3 in the ISO system. It provides data links by transferring cells, and it provides networking services through routing, error checking, and quality of service. With on-the-fly routing, some of the standard error checking for IP packets is handled differently. Since cells are forwarded before all cells of a packet have been received, the entire packet cannot be discarded if it has errors. The integrated router/switch monitors errors in the cells, as with an IP router, while continuing to forward the cells. If errors
are detected, then the packet is determined to be bad and an error indicator is set in the trailer of the last cell of the packet. The router at the next hop will recognize the error and discard the packet. Also, once errors are detected, most of the remaining cells of the packet can be discarded without forwarding to reduce extraneous use of the switch and network. If multiple bad packets are received, then the integrated router/switch will revert back to store-and-forward mode for IP processing. With store-and-forward mode, the integrated router/switch can determine whether the packet or cells are bad before forwarding them, which reduces extraneous use of the switch and network.
Also, the integrated router/switch can provide enhanced quality of service by setting up specific paths for specific IP communications. Standard IP routing has minimal quality of service capability, mainly relating to priority designations. However, quality of service is a significant feature of an ATM network. By using on-the-fly routing, IP communications can be treated like ATM communications and processed in the same manner with a quality of service feature. The use of quality of service allows the system to accommodate a variety of input types, such as telephone, video and audio communications, which have different transmission requirements than regular network and Internet protocols. Different bases can be used for setting the quality of service, such as the type of data in the IP packet, the source address, or destination address. For example, IP flow types, such as high or low priority, can receive certain designated quality of service designations. Additionally, the information in the packet, such voice, data, video, or audio, can be used to select appropriate quality of service, such as guaranteed bandwidth or constant bit rates. Quality of service features allows users with different requirements to access to be distinguished. Each user can obtain the access it needs for its communications at costs with reflect the required levels.
The use of ATM switching improves utilization of multiple paths. In many instances, a single router will communicate with the same next hop for different communications. Since significant overhead time is spent setting up new paths, the integrated router/switch can reduce overhead by continuously maintaining paths to each next hop. The processor 430 will create a set or pool of paths to each next hop, which is then stored in the memory 440. When a CFE determines a next hop for a packet, it may request a new path from the processor. If a path in the pool is available, it will be assigned to the data flow for that packet. If no path is available, a default path will be used for the communication. When a transmission is complete, paths that
were part of the pool remain active, but are noted as being available for another communication. If a special path was set up for a communication, it is removed once the transmission ends.
Using a new path for each transmission to a next hop can result in large numbers of paths being required, called virtual channel explosion, which may overload the switching system of the network. Additionally, not all network products allow for multiple paths. The integrated router/switch 300 of the present invention can adjust operation in order to accommodate these difficulties. As the number of paths increase, or when multiple paths to a location are not permitted, the system can change to use a single path. In this mode, the cells of each packet will be collected and then transmitted in order on a single path. Additionally, different modes can be applied to different information flows. Thus, some transmissions may be given individual paths and others may operate over a single path. In this manner, services may be differentiated based upon necessities of each transmission, and charges to users of the system can be adjusted accordingly.
The integrated router/switch can also provide enhanced quality of service when just a single path to a location is used. In this case, the cell forwarding engine 420 selects a path for which switch 320 processes using an enhanced quality of service. Cell forwarding engine 420 may modify the cells of the packet (e.g., change the Type-Of-Service bits for IP version 4 or the Priority bits for IP version 6) to indicate the desired enhanced quality of service. The next hop may detect the cell modifications, and also process the packet using an enhanced quality of service.
Finally, since the integrated router/switch 300 operates as both a router and switch, it can adaptively adjust the network structure when errors occur. For example, as illustrated in FIGS. 7A and 7B, it can react to link failure. FIG. 7A illustrates a path from 1 to 2 through the network between switches A, B, C and D. However, if the link between C and D fails, cells and packets are lost at C. The failure must be reported back to A to send the new packets through another route. While the failure is being reported, all data passing from A is lost. On the other hand, as illustrated in FIG. 7B, with the present invention, when the link between C and D fails, C changes the paths to send the cells back to B and A, to be forwarded to D through E. In this manner, fewer cells if any are lost. Having described at least one embodiment, various alterations, modifications and improvements will be apparent to those of skill in the art. Such variations are considered to be part of this invention, which is not limited except by the claims attached hereto.