US20170155599A1 - Virtual cluster switching - Google Patents

Virtual cluster switching Download PDF

Info

Publication number
US20170155599A1
US20170155599A1 US14/912,585 US201114912585A US2017155599A1 US 20170155599 A1 US20170155599 A1 US 20170155599A1 US 201114912585 A US201114912585 A US 201114912585A US 2017155599 A1 US2017155599 A1 US 2017155599A1
Authority
US
United States
Prior art keywords
switch
vcs
switches
port
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/912,585
Inventor
Suresh Vobbilisetty
Dilip Chatwani
Shiv Haris
Phanidhar Koganti
Vidyasagara R. Guntaka
Jesse B. Willeke
Senthilkumar Narayanasamy
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Avago Technologies International Sales Pte Ltd
Original Assignee
Brocade Communications Systems LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/087,239 external-priority patent/US8867552B2/en
Priority claimed from US13/092,460 external-priority patent/US9769016B2/en
Priority claimed from US13/092,701 external-priority patent/US8989186B2/en
Priority claimed from US13/092,724 external-priority patent/US9001824B2/en
Priority claimed from US13/092,580 external-priority patent/US9716672B2/en
Priority claimed from US13/092,752 external-priority patent/US9270486B2/en
Priority claimed from US13/092,877 external-priority patent/US9231890B2/en
Priority claimed from US13/098,360 external-priority patent/US8625616B2/en
Application filed by Brocade Communications Systems LLC filed Critical Brocade Communications Systems LLC
Priority to US14/912,585 priority Critical patent/US20170155599A1/en
Priority claimed from PCT/US2011/034917 external-priority patent/WO2011140028A1/en
Publication of US20170155599A1 publication Critical patent/US20170155599A1/en
Assigned to Brocade Communications Systems LLC reassignment Brocade Communications Systems LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: BROCADE COMMUNICATIONS SYSTEMS, INC.
Assigned to AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED reassignment AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Brocade Communications Systems LLC
Assigned to BROCADE COMMUNICATIONS SYSTEMS, INC. reassignment BROCADE COMMUNICATIONS SYSTEMS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHATWANI, DILIP, VOBBILISETTY, SURESH
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46Interconnection of networks
    • H04L12/4641Virtual LANs, VLANs, e.g. virtual private networks [VPN]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/25Routing or path finding in a switch fabric
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46Interconnection of networks
    • H04L12/4604LAN interconnection over a backbone network, e.g. Internet, Frame Relay
    • H04L12/462LAN interconnection over a bridge based backbone
    • H04L12/4625Single bridge functionality, e.g. connection of two networks over a single bridge
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46Interconnection of networks
    • H04L12/4633Interconnection of networks using encapsulation techniques, e.g. tunneling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0803Configuration setting
    • H04L41/0813Configuration setting characterised by the conditions triggering a change of settings
    • H04L41/0816Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/30Peripheral units, e.g. input or output ports
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/35Switches specially adapted for specific applications
    • H04L49/356Switches specially adapted for specific applications for storage area networks
    • H04L49/357Fibre channel switches
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/65Re-configuration of fast packet switches
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/70Virtual switches

Definitions

  • the present disclosure relates to network design. More specifically, the present disclosure relates to a method for a constructing a scalable switching system that facilitates automatic configuration.
  • switch stacking One way to increase the throughput of a switch system is to use switch stacking.
  • switch stacking multiple smaller-scale, identical switches are interconnected in a special pattern to form a larger logical switch.
  • switch stacking requires careful configuration of the ports and inter-switch links. The amount of required manual configuration becomes prohibitively complex and tedious when the stack reaches a certain size, which precludes switch stacking from being a practical option in building a large-scale switching system.
  • a system based on stacked switches often has topology limitations which restrict the scalability of the system due to fabric bandwidth considerations.
  • the switch includes one or more ports on the switch configured to transmit packets encapsulated based on a first protocol.
  • the switch further includes a control mechanism.
  • the control mechanism forms a logical switch based on a second protocol, receives an automatically assigned identifier for the logical switch without requiring manual configuration of the identifier, and joins a virtual cluster switch.
  • the virtual cluster switch comprises one or more physical switches which are allowed to be coupled in an arbitrary topology. Furthermore, the virtual cluster switch appears to be one single switch.
  • the first protocol is a Transparent Interconnection of Lots of Links (TRILL) protocol
  • TRILL Transparent Interconnection of Lots of Links
  • the logical switch formed by the control mechanism is a logical Fibre Channel (FC) switch.
  • FC Fibre Channel
  • the identifier assigned to the logical switch is an FC switch domain ID.
  • control mechanism is further configured to maintain a copy of configuration information for the virtual cluster switch.
  • the configuration information for the virtual cluster switch comprises a number of logical switch identifiers assigned to the physical switches in the virtual cluster switch.
  • the switch includes a media access control (MAC) learning mechanism which is configured to learn a source MAC address and a corresponding VLAN identifier of an ingress packet associated with a port and communicate a learned MAC address, a corresponding VLAN identifier, and the corresponding port information to a name service.
  • MAC media access control
  • One embodiment of the present invention provides a switching system that includes a plurality of switches configured to transport packets using a first protocol.
  • Each switch includes a control mechanism.
  • the plurality switches are allowed to be coupled in an arbitrary topology.
  • the control mechanism automatically configures the respective switch within the switching system based on a second protocol without requiring manual configuration, and the switching system appears externally as a single switch.
  • a respective switch in the switching system receives an automatically configured identifier associated with a logical switch formed on the respective switch.
  • the logical switch is a logical FC switch.
  • the identifier is an FC switch domain ID.
  • the packets are transported between switches based on a TRILL protocol.
  • the respective switch is assigned a TRILL RBridge identifier that corresponds to the FC switch domain ID.
  • a respective switch maintains a copy of configuration information of all the switches in the switching system.
  • the switching system includes a name service which maintains records of MAC addresses and VLAN information learned by a respective switch.
  • FIG. 1A illustrates an exemplary virtual cluster switch (VCS) system, in accordance with an embodiment of the present invention.
  • VCS virtual cluster switch
  • FIG. 1B illustrates an exemplary VCS system where the member switches are configured in a CLOS network, in accordance with an embodiment of the present invention.
  • FIG. 2 illustrates the protocol stack within a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 3 illustrates an exemplary configuration of a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 4 illustrates an exemplary configuration of how a virtual cluster switch can be connected to different edge networks, in accordance with an embodiment of the present invention.
  • FIG. 5A illustrates how a logical Fibre Channel switch fabric is formed in a virtual cluster switch in conjunction with the example in FIG. 4 , in accordance with an embodiment of the present invention.
  • FIG. 5B illustrates an example of how a logical FC switch can be created within a physical Ethernet switch, in accordance with one embodiment of the present invention.
  • FIG. 6 illustrates an exemplary VCS configuration database, in accordance with an embodiment of the present invention.
  • FIG. 7A illustrates an exemplary process of a switch joining a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 7B presents a flowchart illustrating the process of a default switch joining a VCS, in accordance with one embodiment of the present invention.
  • FIG. 7C presents a flowchart illustrating the process of a switch re-joining a previously joined VCS, in accordance with one embodiment of the present invention.
  • FIG. 7D presents a flowchart illustrating the general operation of a switch during a join process, in accordance with one embodiment of the present invention.
  • FIG. 8 presents a flowchart illustrating the process of looking up an ingress frame's destination MAC address and forwarding the frame in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 9 illustrates how data frames and control frames are transported through a VCS, in accordance with one embodiment of the present invention.
  • FIG. 10 illustrates a logical VCS access layer (VAL) which includes an automatic port profile manager, in accordance with one embodiment of the present invention.
  • VAL logical VCS access layer
  • FIG. 11 illustrates an example of the operation of automatic migration of port profiles (AMPP), in accordance with one embodiment of the present invention
  • FIG. 12A illustrates exemplary port profile contents, in accordance with one embodiment of the present invention.
  • FIG. 12B illustrates three exemplary port profiles, in accordance with on embodiment of the present invention.
  • FIGS. 13A and 13B illustrates how forwarding is achieved between VMs based on port profile membership, in accordance with one embodiment of the present invention.
  • FIG. 14 presents a flowchart illustrating the process of creating and applying a port profile, in accordance with one embodiment of the present invention.
  • FIG. 15 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • FIG. 16 illustrates an exemplary configuration of advanced link tracking in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 17 illustrates an example where advanced link tracking allows virtual machines to re-route egress traffic when a link fails, in accordance with one embodiment of the present invention.
  • FIG. 18 presents a flowchart illustrating the process of advance link tracking, in accordance with one embodiment of the present invention.
  • FIG. 19 illustrates an exemplary switch that facilitates virtual cluster switching and advanced link tracking, in accordance with one embodiment of the present invention.
  • FIG. 20 illustrates an exemplary VCS network environment with virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 21A illustrates an exemplary data structure for storing virtual port grouping information, in accordance with one embodiment of the present invention.
  • FIG. 21B illustrates logical virtual networks within a VCS as a result of virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 22 presents a flowchart illustrating the process of enforcing virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 23 illustrates an exemplary switch that facilitates virtual cluster switching with virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 24 illustrates an example of name service operation in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 25 presents a flowchart illustrating the process of distributing learned MAC information by the Ethernet name service in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 26 presents a flowchart illustrating the process of distributing information of a learned MAC address via an MCT, in accordance with one embodiment of the present invention.
  • FIG. 27 presents a flowchart illustrating the process of updating the link state in an MCT group, in accordance with one embodiment of the present invention.
  • FIG. 28 illustrates an exemplary switch that facilitates formation of a virtual cluster switch with Ethernet and MCT name services, in accordance with an embodiment of the present invention.
  • FIG. 29 illustrates an exemplary equal cost multi-pathing configuration in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 30 illustrates an example of VCS congestion notification, in accordance with one embodiment of the present invention.
  • FIG. 31 illustrates an exemplary hybrid trunk in accordance with one embodiment of the present invention.
  • FIG. 32 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • FIG. 33 illustrates an exemplary network architecture which includes CNE devices for facilitate cross-data-center communications, in accordance with one embodiment of the present invention.
  • FIG. 34 illustrates an exemplary implementation of CNE-enabled VCSs, in accordance with one embodiment of the present invention.
  • FIG. 35A presents a diagram illustrating how CNE devices handle broadcast, unknown unicast, and multicast (BUM) traffic across data centers, in accordance with one embodiment of the present invention.
  • FIG. 35B presents a diagram illustrating how CNE devices handle unicast traffic across data centers, in accordance with one embodiment of the present invention.
  • FIG. 36 illustrates an example where two CNE devices are used to construct a vLAG, in accordance with an embodiment of the present invention.
  • FIG. 37 illustrates an exemplary CNE device, in accordance with one embodiment of the present invention.
  • the problem of building a versatile, cost-effective, and scalable switching system is solved by running a control plane with automatic configuration capabilities (such as the Fibre Channel control plane) over a conventional transport protocol, thereby allowing a number of switches to be inter-connected to form a single, scalable logical switch without requiring burdensome manual configuration.
  • a control plane with automatic configuration capabilities such as the Fibre Channel control plane
  • VCS virtual cluster switch
  • the automatic configuration capability provided by the control plane running on each physical switch allows any number of switches to be connected in an arbitrary topology without requiring tedious manual configuration of the ports and links. This feature makes it possible to use many smaller, inexpensive switches to construct a large cluster switch, which can be viewed as a single logical switch externally.
  • a virtual cluster switch is not the same as conventional switch stacking.
  • switch stacking multiple switches are interconnected at a common location (often within the same rack), based on a particular topology, and manually configured in a particular way. These stacked switches typically share a common address, e.g., IP address, so they can be addressed as a single switch externally.
  • switch stacking requires a significant amount of manual configuration of the ports and inter-switch links. The need for manual configuration prohibits switch stacking from being a viable option in building a large-scale switching system.
  • the topology restriction imposed by switch stacking also limits the number of switches that can be stacked. This is because it is very difficult, if not impossible, to design a stack topology that allows the overall switch bandwidth to scale adequately with the number of switch units.
  • a VCS can include an arbitrary number of switches with individual addresses, can be based on an arbitrary topology, and does not require extensive manual configuration.
  • the switches can reside in the same location, or be distributed over different locations.
  • VCS automatic and dynamic configurability
  • a network operator to build its switching system in a distributed and “pay-as-you-grow” fashion without sacrificing scalability.
  • the VCS's ability to respond to changing network conditions makes it an ideal solution in a virtual computing environment, where network loads often change with time.
  • TRILL Transparent Interconnection of Lots of Links
  • FC Fibre Channel
  • RBridge and switch
  • RBridge does not limit embodiments of the present invention to TRILL networks only.
  • the TRILL protocol is described in IETF draft “RBridges: Base Protocol Specification,” available at http://tools.ietf.org/html/draft-ietf-trill-rbridge-protocol, which is incorporated by reference herein
  • VCS virtual cluster switch
  • VCS virtual cluster switching
  • VCS virtual cluster switching
  • RBridge refers to routing bridges, which are bridges implementing the TRILL protocol as described in IETF draft “RBridges: Base Protocol Specification.” Embodiments of the present invention are not limited to the application among RBridges. Other types of switches, routers, and forwarders can also be used.
  • frame or “packet” refer to a group of bits that can be transported together across a network. “Frame” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. “Packet” should not be interpreted as limiting embodiments of the present invention to layer-3 networks. “Frame” or “packet” can be replaced by other terminologies referring to a group of bits, such as “cell” or “datagram.”
  • FIG. 1A illustrates an exemplary virtual cluster switch system, in accordance with an embodiment of the present invention.
  • a VCS 100 includes physical switches 101 , 102 , 103 , 104 , 105 , 106 , and 107 .
  • a given physical switch runs an Ethernet-based transport protocol on its ports (e.g., TRILL on its inter-switch ports, and Ethernet transport on its external ports), while its control plane runs an FC switch fabric protocol stack.
  • TRILL protocol facilitates transport of Ethernet frames within and across VCS 100 in a routed fashion (since TRILL provides routing functions to Ethernet frames).
  • the FC switch fabric protocol stack facilitates the automatic configuration of individual physical switches, in a way similar to how a conventional FC switch fabric is formed and automatically configured.
  • VCS 100 can appear externally as an ultra-high-capacity Ethernet switch. More details on FC network architecture, protocols, naming/address conventions, and various standards are available in the documentation available from the NCITS/ANSI T11 committee (www.t11.org) and publicly available literature, such as “Designing Storage Area Networks,” by Tom Clark, 2nd Ed., Addison Wesley, 2003, the disclosures of which are incorporated by reference in their entirety herein.
  • a physical switch may dedicate a number of ports for external use (i.e., to be coupled to end hosts or other switches external to the VCS) and other ports for inter-switch connection.
  • VCS 100 appears to be one switch to a device from the outside, and any port from any of the physical switches is considered one port on the VCS.
  • port groups 110 and 112 are both VCS external ports and can be treated equally as if they were ports on a common physical switch, although switches 105 and 107 may reside in two different locations.
  • the physical switches can reside at a common location, such as a data center or central office, or be distributed in different locations. Hence, it is possible to construct a large-scale centralized switching system using many smaller, inexpensive switches housed in one or more chassis at the same location. It is also possible to have the physical switches placed at different locations, thus creating a logical switch that can be accessed from multiple locations.
  • the topology used to interconnect the physical switches can also be versatile.
  • VCS 100 is based on a mesh topology. In further embodiments, a VCS can be based on a ring, tree, or other types of topologies.
  • the protocol architecture of a VCS is based on elements from the standard IEEE 802.1Q Ethernet bridge, which is emulated over a transport based on the Fibre Channel Framing and Signaling-2 (FC-FS-2) standard.
  • FC-FS-2 Fibre Channel Framing and Signaling-2
  • FIG. 1B presents an exemplary VCS with its member switches connected in a CLOS network, in accordance with one embodiment of the present invention.
  • a VCS 120 forms a fully non-blocking 8 ⁇ 8 switch, using eight 4 ⁇ 4 switches and four 2 ⁇ 2 switches connected in a three-stage CLOS network.
  • a large-scale switch with a higher port count can be built in a similar way.
  • FIG. 2 illustrates the protocol stack within a virtual cluster switch, in accordance with an embodiment of the present invention.
  • Switch 202 includes an ingress Ethernet port 206 and an inter-switch port 208 .
  • Switch 204 includes an egress Ethernet port 212 and an inter-switch port 210 .
  • Ingress Ethernet port 206 receives Ethernet frames from an external device.
  • the Ethernet header is processed by a medium access control (MAC) layer protocol.
  • MAC medium access control
  • FDB forwarding database
  • FDB 214 is maintained locally in a switch, which would perform a lookup based on the destination MAC address and the VLAN indicated in the Ethernet frame. The lookup result would provide the corresponding output port. However, since VCS 200 is not one single physical switch, FDB 214 would return the egress switch's identifier (i.e., switch 204 's identifier). In one embodiment, FDB 214 is a data structure replicated and distributed among all the physical switches. That is, every physical switch maintains its own copy of FDB 214 .
  • the forwarding of the Ethernet frame between ingress switch 202 and egress switch 204 is performed via inter-switch ports 208 and 210 .
  • the frame transported between the two inter-switch ports is encapsulated in an outer MAC header and a TRILL header, in accordance with the TRILL standard.
  • the protocol stack associated with a given inter-switch port includes the following (from bottom up): MAC layer, TRILL layer, FC-FS-2 layer, FC E-Port layer, and FC link services (FC-LS) layer.
  • the FC-LS layer is responsible for maintaining the connectivity information of a physical switch's neighbor, and populating an FC routing information base (RIB) 222 . This operation is similar to what is done in an FC switch fabric.
  • FC-LS protocol is also responsible for handling joining and departure of a physical switch in VCS 200 .
  • the operation of the FC-LS layer is specified in the FC-LS standard, which is available at http://www.t11.org/ftp/t11/member/fc/ls/06-393v5.pdf, the disclosure of which is incorporated herein in its entirety.
  • Path selector 218 performs a fabric shortest-path first (FSPF)-based route lookup in conjunction with RIB 222 , and identifies the next-hop switch within VCS 200 .
  • FSPF fabric shortest-path first
  • each physical switch includes an address manager 216 and a fabric controller 220 .
  • Address manager 216 is responsible for configuring the address of a physical switch when the switch first joins the VCS. For example, when switch 202 first joins VCS 200 , address manager 216 can negotiate a new FC switch domain ID, which is subsequently used to identify the switch within VCS 200 .
  • Fabric controller 220 is responsible for managing and configuring the logical FC switch fabric formed on the control plane of VCS 200 .
  • VCS protocol architecture
  • FC switch fabric with an Ethernet/TRILL transport.
  • Each physical switch from an external point of view, appears to be a TRILL RBridge.
  • the switch's control plane implements the FC switch fabric software.
  • embodiments of the present invention facilitate the construction of an “Ethernet switch fabric” running on FC control software. This unique combination provides the VCS with automatic configuration capability and allows it to provide the ubiquitous Ethernet services in a very scalable fashion.
  • FIG. 3 illustrates an exemplary configuration of a virtual cluster switch, in accordance with an embodiment of the present invention.
  • a VCS 300 includes four physical switches 302 , 304 , 306 , and 308 .
  • VCS 300 constitutes an access layer which is coupled to two aggregation switches 310 and 312 .
  • the physical switches within VCS 300 are connected in a ring topology.
  • Aggregation switch 310 or 312 can connect to any of the physical switches within VCS 300 .
  • aggregation switch 310 is coupled to physical switches 302 and 308 .
  • FIG. 4 illustrates an exemplary configuration of how a virtual cluster switch can be connected to different edge networks, in accordance with an embodiment of the present invention.
  • a VCS 400 includes a number of TRILL RBridges 402 , 404 , 406 , 408 , and 410 , which are controlled by the FC switch-fabric control plane. Also included in VCS 400 are RBridges 412 , 414 , and 416 . Each RBridge has a number of edge ports which can be connected to external edge networks.
  • RBridge 412 is coupled with hosts 420 and 422 via 10GE ports.
  • RBridge 414 is coupled to a host 426 via a 10GE port.
  • These RBridges have TRILL-based inter-switch ports for connection with other TRILL RBridges in VCS 400 .
  • RBridge 416 is coupled to host 428 and an external Ethernet switch 430 , which is coupled to an external network that includes a host 424 .
  • network equipment can also be coupled directly to any of the physical switches in VCS 400 .
  • TRILL RBridge 408 is coupled to a data storage 417
  • TRILL RBridge 410 is coupled to a data storage 418 .
  • TRILL RBridges Although the physical switches within VCS 400 are labeled as “TRILL RBridges,” they are different from the conventional TRILL RBridge in the sense that they are controlled by the FC switch fabric control plane. In other words, the assignment of switch addresses, link discovery and maintenance, topology convergence, routing, and forwarding can be handled by the corresponding FC protocols. Particularly, each TRILL RBridge's switch ID or nickname is mapped from the corresponding FC switch domain ID, which can be automatically assigned when a switch joins VCS 400 (which is logically similar to an FC switch fabric).
  • TRILL is only used as a transport between the switches within VCS 400 . This is because TRILL can readily accommodate native Ethernet frames. Also, the TRILL standards provide a ready-to-use forwarding mechanism that can be used in any routed network with arbitrary topology (although the actual routing in VCS is done by the FC switch fabric protocols). Embodiments of the present invention should be not limited to using only TRILL as the transport. Other protocols (such as multi-protocol label switching (MPLS) or Internet Protocol (IP)), either public or proprietary, can also be used for the transport.
  • MPLS multi-protocol label switching
  • IP Internet Protocol
  • a VCS is created by instantiating a logical FC switch in the control plane of each switch.
  • a virtual generic port (denoted as G_Port) is created for each Ethernet port on the RBridge.
  • G_Port assumes the normal G_Port behavior from the FC switch perspective.
  • the specific transition from a G_Port to either an FC F_Port or E_Port is determined by the underlying link and physical layer protocols. For example, if the physical Ethernet port is connected to an external device which lacks VCS capabilities, the corresponding G_Port will be turned into an F_Port. On the other hand, if the physical Ethernet port is connected to a switch with VCS capabilities and it is confirmed that the switch on the other side is part of a VCS, then the G_Port will be turned into an E_port.
  • FIG. 5A illustrates how a logical Fibre Channel switch fabric is formed in a virtual cluster switch in conjunction with the example in FIG. 4 , in accordance with an embodiment of the present invention.
  • RBridge 412 contains a virtual, logical FC switch 502 .
  • logical FC switch 502 Corresponding to the physical Ethernet ports coupled to hosts 420 and 422 , logical FC switch 502 has two logical F_Ports, which are logically coupled to hosts 420 and 422 . In addition, two logical N_Ports, 506 and 504 , are created for hosts 420 and 422 , respectively.
  • logical FC switch 502 On the VCS side, logical FC switch 502 has three logical E_Ports, which are to be coupled with other logical FC switches in the logical FC switch fabric in the VCS.
  • RBridge 416 contains a virtual, logical FC switch 512 .
  • logical FC switch 512 Corresponding to the physical Ethernet ports coupled to host 428 and external switch 430 , logical FC switch 512 has a logical F_Port coupled to host 428 , and a logical FL_Port coupled to switch 430 .
  • a logical N_Port 510 is created for host 428
  • a logical NL_Port 508 is created for switch 430 .
  • the logical FL_Port is created because that port is coupled to a switch (switch 430 ), instead of a regular host, and therefore logical FC switch 512 assumes an arbitrated loop topology leading to switch 430 .
  • Logical NL_Port 508 is created based on the same reasoning to represent a corresponding NL_Port on switch 430 .
  • logical FC switch 512 On the VCS side, logical FC switch 512 has two logical E_Ports, which to be coupled with other logical FC switches in the logical FC switch fabric in the VCS.
  • FIG. 5B illustrates an example of how a logical FC switch can be created within a physical Ethernet switch, in accordance with one embodiment of the present invention.
  • fabric port refers to a port used to couple multiple switches in a VCS.
  • the clustering protocols control the forwarding between fabric ports.
  • edge port refers to a port that is not currently coupled to another switch unit in the VCS. Standard IEEE 802.1Q and layer-3 protocols control forwarding on edge ports.
  • a logical FC switch 521 is created within a physical switch (RBridge) 520 .
  • Logical FC switch 521 participates in the FC switch fabric protocol via logical inter-switch links (ISLs) to other switch units and has an FC switch domain ID assigned to it just as a physical FC switch does. In other words, the domain allocation, principal switch selection, and conflict resolution work just as they would on a physical FC ISL.
  • ISLs logical inter-switch links
  • the physical edge ports 522 and 524 are mapped to logical F_Ports 532 and 534 , respectively.
  • physical fabric ports 526 and 528 are mapped to logical E_Ports 536 and 538 , respectively.
  • logical FC switch 521 when logical FC switch 521 is created (for example, during the boot-up sequence), logical FC switch 521 only has four G_Ports which correspond to the four physical ports. These G_Ports are subsequently mapped to F_Ports or E_Ports, depending on the devices coupled to the physical ports.
  • Neighbor discovery is the first step in VCS formation between two VCS-capable switches. It is assumed that the verification of VCS capability can be carried out by a handshake process between two neighbor switches when the link is first brought up.
  • VCS presents itself as one unified switch composed of multiple member switches.
  • the VCS configuration is based on a distributed database, which is replicated and distributed over all switches.
  • each VCS member switch maintains a copy of the VCS configuration database, and any change to the database is propagated to all the member switches.
  • a network administrator can configure any part of the VCS from anywhere by, for example, executing a command line interface (CLI) command from any member switch.
  • CLI command line interface
  • a VCS configuration database includes a global configuration table (GT) of the VCS and a list of switch description tables (STs), each of which describes a VCS member switch.
  • a member switch can have a VCS configuration database that includes a global table and one switch description table, e.g., [ ⁇ GT> ⁇ ST>].
  • a VCS with multiple switches will have a configuration database that has a single global table and multiple switch description tables, e.g., [ ⁇ GT> ⁇ ST0> ⁇ ST1> . . . ⁇ STn ⁇ 1>].
  • the number n corresponds to the number of member switches in the VCS.
  • the GT can include at least the following information: the VCS ID, number of nodes in the VCS, a list of VLANs supported by the VCS, a list of all the switches (e.g., list of FC switch domain IDs for all active switches) in the VCS, and the FC switch domain ID of the principal switch (as in a logical FC switch fabric).
  • a switch description table can include at least the following information: the IN_VCS flag, indication whether the switch is a principal switch in the logical FC switch fabric, the FC switch domain ID for the switch, the FC world-wide name (WWN) for the corresponding logical FC switch; the mapped ID of the switch, and optionally the IP address of the switch.
  • WWN FC world-wide name
  • each switch's global configuration database is associated with a transaction ID.
  • the transaction ID specifies the latest transaction (e.g., update or change) incurred to the global configuration database.
  • the transaction IDs of the global configuration databases in two switches can be compared to determine which database has the most current information (i.e., the database with the more current transaction ID is more up-to-date).
  • the transaction ID is the switch's serial number plus a sequential transaction number. This configuration can unambiguously resolve which switch has the latest configuration.
  • VCS configuration database 600 describes the VCS configuration when the switch is part of a VCS.
  • Default switch configuration table 604 describes the switch's default configuration.
  • VCS configuration database 600 includes a GT 602 , which includes a VCS identifier (denoted as VCS_ID) and a VLAN list within the VCS.
  • VCS_ID VCS identifier
  • STs such as ST0, ST1, and STn.
  • Each ST includes the corresponding member switch's MAC address and FC switch domain ID, as well as the switch's interface details. Note that each switch also has a VCS-mapped ID which is a switch index within the VCS.
  • each switch also has a VCS-mapped ID (denoted as “mappedID”), which is a switch index within the VCS.
  • mappedID is unique and persistent within the VCS. That is, when a switch joins the VCS for the first time, the VCS assigns a mapped ID to the switch. This mapped ID persists with the switch, even if the switch leaves the VCS. When the switch joins the VCS again at a later time, the same mapped ID is used by the VCS to retrieve previous configuration information for the switch. This feature can reduce the amount of configuration overhead in VCS.
  • the persistent mapped ID allows the VCS to “recognize” a previously configured member switch when it re-joins the VCS, since a dynamically assigned FC fabric domain ID would change each time the member switch joins and is configured by the VCS.
  • Default switch configuration table 604 has an entry for the mappedID that points to the corresponding ST in VCS configuration database 600 . Note that only VCS configuration database 600 is replicated and distributed to all switches in the VCS. Default switch configuration table 604 is local to a particular member switch.
  • the “IN_VCS” value in default switch configuration table 604 indicates whether the member switch is part of a VCS.
  • a switch is considered to be “in a VCS” when it is assigned one of the FC switch domains by the FC switch fabric with two or more switch domains. If a switch is part of an FC switch fabric that has only one switch domain, i.e., its own switch domain, then the switch is considered to be “not in a VCS.”
  • the logical FC switch fabric formation process allocates a new switch domain ID to the joining switch. In one embodiment, only the switches directly connected to the new switch participate in the VCS join operation.
  • the global configuration database of a joining switch is current and in sync with the global configuration database of the VCS based on a comparison of the transaction IDs of the two databases (e.g., when a member switch is temporarily disconnected from the VCS and re-connected shortly afterward), a trivial merge is performed. That is, the joining switch can be connected to the VCS, and no change or update to the global VCS configuration database is required.
  • the VCS assigns an FC switch domain ID to the logical FC switch formed within the joining switch.
  • the joining switch can be automatically assigned an IP address (which in one embodiment can be internal to the VCS) corresponding to the FC switch domain ID.
  • the joining switch's IP address can be 127.10.10.FC_domain_ID.
  • each port on the joining switch can be automatically assigned a MAC address which is unique to that port, on that switch.
  • a port on the joining switch can be assigned a MAC address of OUI:FC_domain_ID:0:0, where OUI is the Organizationally Unique Identifier for the port, and FC_domain_ID is the switch domain ID assigned to the logical FC switch.
  • the IP and MAC address assignment process can be part of the software within the switch which controls the FC switch formation and the join/merge process.
  • the automatically assigned switch IP address and port MAC address for a given VCS member switch allow any port on the switch to be remotely configured from anywhere within the VCS.
  • a configuration command of a given member switch can be issued from a host connected to any member switch in the VCS and identifies the switch to be configured by its IP address and optionally one or more port MAC addresses.
  • Such configuration command might include information on VLAN configuration, quality-of-service (QoS) configuration, and/or access-control configuration, each of which can be port-specific.
  • the change to a switch's configuration is tentatively transmitted to the switch. After the switch confirms and validates the change, a commit-change command is transmitted to all the member switches in the VCS, so the global configuration database can be updated throughout the VCS.
  • the change is tentatively transmitted to all the member switches in the VCS, and the commit-change command is only sent out after all the switches confirm and validate the tentatively change.
  • FIG. 7A illustrates an exemplary process of a switch joining a virtual cluster switch, in accordance with an embodiment of the present invention.
  • a switch 702 is within an existing VCS, and a switch 704 is joining the VCS.
  • both switches 702 and 704 trigger an FC State Change Notification (SCN) process.
  • SCN FC State Change Notification
  • both switches 702 and 704 perform a PRE-INVITE operation.
  • the pre-invite operation involves the following process.
  • both neighbors on each end of the link present to the other switch a VCS four-tuple of ⁇ Prior VCS_ID, SWITCH_MAC, mappedID, IN_VCS> from a prior incarnation, if any. Otherwise, the switch presents to the counterpart a default tuple. If the VCS_ID value was not set from a prior join operation, a VCS_ID value of ⁇ 1 is used. In addition, if a switch's IN_VCS flag is set to 0, it sends out its interface configuration to the neighboring switch. In the example in FIG. 7 , both switches 702 and 704 send the above information to the other switch.
  • a driver switch for the join process is selected.
  • switch 702 After switch 702 is selected as the driver switch, switch 702 then attempts to reserve a slot in the VCS configuration database corresponding to the mappedID value in switch 704 's PRE-INVITE information. Next, switch 702 searches the VCS configuration database for switch 704 's MAC address in any mappedID slot. If such a slot is found, switch 702 copies all information from the identified slot into the reserved slot. Otherwise, switch 702 copies the information received during the PRE-INVITE from switch 704 into the VCS configuration database. The updated VCS configuration database is then propagated to all the switches in the VCS as a prepare operation in the database (note that the update is not committed to the database yet).
  • the prepare operation may or may not result in configuration conflicts, which may be flagged as warnings or fatal errors.
  • Such conflicts can include inconsistencies between the joining switch's local configuration or policy setting and the VCS configuration. For example, a conflict arises when the joining switch is manually configured to allow packets with a particular VLAN value to pass through, whereas the VCS does not allow this VLAN value to enter the switch fabric from this particular RBridge (for example, when this VLAN value is reserved for other purposes).
  • the prepare operation is handled locally and/or remotely in concert with other VCS member switches. If there is an un-resolvable conflict, switch 702 sends out a PRE-INVITE-FAILED message to switch 704 . Otherwise, switch 702 generates an INVITE message with the VCS's merged view of the switch (i.e., the updated VCS configuration database).
  • switch 704 Upon receiving the INVITE message, switch 704 either accepts or rejects the INVITE. The INVITE can be rejected if the configuration in the INVITE is in conflict with what switch 704 can accept. If the INVITE is acceptable, switch 704 sends back an INVITE-ACCEPT message in response. The INVITE-ACCEPT message then triggers a final database commit throughout all member switches in the VCS. In other words, the updated VCS configuration database is updated, replicated, and distributed to all the switches in the VCS.
  • a “joining switch” refers to a switch attempting to join a VCS.
  • a “neighboring VCS member switch” or “neighboring member switch” refers to a VCS member switch to which the joining switch is connected.
  • a VCS is pre-configured (e.g., the global configuration database) with the MAC address of a joining switch with an optionally pre-allocated mapped ID for the joining switch.
  • the joining switch may be allowed to carry any value in the VCS_ID field of its existing configuration.
  • the neighboring VCS member switch can assign an FC switch domain ID and the proper VCS_ID to the joining switch in the INVITE message.
  • the joining switch may be pre-provisioned to join an existing VCS (e.g., with the parameters in the default switch configuration table, such as mappedID, VCS_ID, and IN_VCS, populated with values corresponding to the VCS). If the pre-provisioned parameters do not guarantee a slot with the same mappedID in the global configuration database when the switch joins the VCS, the switch can revert to the default joining procedure described below.
  • a default switch is one that has no records of any previous joining with a VCS.
  • a switch can become a default switch if it is forced into a factory default state.
  • a joining default switch can present its initial configuration information (for example, its interface configuration details) to a neighboring VCS member switch.
  • a slot in the VCS configuration database is selected based on a monotonically incrementing number, which is used as the mapped ID for the joining switch.
  • the corresponding FC switch domain ID which is allocated to the joining switch and the joining switch's MAC is updated accordingly in this slot.
  • the neighboring VCS member switch then initiates a prepare transaction, which propagates to all VCS member switches and requires an explicit validation of the joining switch's configuration information from each VCS member switch. If the prepare transaction fails, a PRE-INVITE-FAILED message is sent to the joining switch and the joining process is aborted.
  • FIG. 7B presents a flowchart illustrating the process of a default switch joining a VCS, in accordance with one embodiment of the present invention.
  • the VCS member switch reserves a slot in the global configuration database (operation 722 ).
  • the reserved slot corresponds to a mappedID value which is also assigned to the joining switch.
  • the logical FC switch within the neighboring VCS member switch allocates an FC switch domain ID for the joining switch (operation 724 ). This operation resembles a similar fabric login operation in the FC switch fabric join process.
  • the neighboring VCS member switch then tentatively updates the reserved slot in the global configuration database with the allocated FC switch domain ID and the joining switch's MAC address (operation 726 ).
  • the neighboring VCS member switch transmits the joining switch's tentative configuration to all member switches in the VCS (operation 728 ), and determines whether the joining switch's configuration information is confirmed and validated by all VCS member switches (operation 730 ). If the joining switch's configuration is confirmed, the neighboring member switch then commits the changes to the global configuration database and completes the join process (operation 732 ). Otherwise, the join process is aborted and the tentative changes to the global configuration database are discarded (operation 734 ).
  • the neighboring VCS member switch first checks whether the joining switch's VCS_ID is the same as the existing VCS_ID on the member switch. If the two VCS_ID values are the same, the neighboring member switch attempts to find a slot in the global configuration database with the same mappedID value which was received from the joining switching during the tuple-exchange process. If such a slot in the global database is available, the slot is reserved for the joining switch.
  • the global configuration database is searched for a match to the joining switch's MAC address. If a match is found in another slot, the configuration information from that slot is copied to the reserved slot. Subsequently, the join process continues as described in FIG. 7A . If the two VCS_ID values are different, the system performs a join process as described blow for the use case “switch joins another VCS.”
  • FIG. 7C presents a flowchart illustrating the process of a switch re-joining a previously joined VCS, in accordance with one embodiment of the present invention.
  • a neighboring member switch first determines whether the joining switch's existing VCS_ID is the same as the neighboring member switch's VCS_ID (operation 740 ). If the two VCS_IDs are different, the joining process proceeds to the operations illustrated in FIG. 7D (operation 741 ). If the two VCS_IDs are the same, which means that the joining switch has previously been part of the same VCS, the neighboring member switch further determines whether there is a slot in the global configuration database with the same mappedID as the joining switch (operation 742 ).
  • the neighboring member switch reserves a slot in the global configuration database with the same mappedID (operation 744 ). Otherwise, the neighboring member switch reserves a slot in the global configuration database with a new mappedID, which is also communicated to the joining switch (operation 746 ).
  • the neighboring member switch determines whether the global configuration database contains a slot with the same MAC address as the joining switch (operation 748 ). If there is such a slot, which means that the global configuration database contains a slot which has been used previously for the same joining switch's configuration information, such information is copied from the identified slot to the reserved slot (operation 750 ). Otherwise, the neighboring member switch proceeds to complete the join process as illustrated in FIG. 7A .
  • This use case occurs when a switch is disconnected from one VCS and then connected to a different VCS without being reset to the default state.
  • This scenario can also occur when a switch is connected to a VCS while it is participating in another VCS. In such cases, there will be a VCS_ID mismatch in the join process.
  • the IN_VCS field in the joining switch's configuration table might or might not be set. If the IN_VCS field is not set, which means that the joining switch is not currently participating in a VCS, the join process can assign the switch a new VCS_ID corresponding to the VCS the switch is joining.
  • the join process is disallowed.
  • the joining switch can complete the joining process after being set to the default state.
  • the switch with a lower FC switch domain ID would have its IN_VCS field set to “1” and drives the join process.
  • two VCSs are allowed to merge together. Similar to the FC switch fabric formation process, the logical FC switches in both VCSs would select a new principal FC switch. This newly selected principal FC switch then re-assigns FC switch domain IDs to all the member switches. After the FC switch domain IDs are assigned, a “fabric up” message which is broadcast to all the member switches starts the VCS join process.
  • each member switch can join the VCS (which initially only contains the switch with the principal FC switch) using the “switch joins another VCS” procedure described above.
  • a switch When a switch is removed from a VCS, its neighboring member switch typically receives a “domain-unreachable” notification at its logical FC switch. Upon receiving this notification, the neighboring member switch disables this switch from the global VCS configuration database and propagates this change to all other member switches. Optionally, the neighboring member switch does not clear the slot previously used by the removed switch in the global configuration database. This way, if the departure of the switch is only temporary, the same slot in the configuration database can still be used when the switch re-joins the VCS.
  • the logical FC infrastructure in the member switches can detect the disconnection of the switch(es) and issues a number of “domain-unreachable” notifications.
  • a comparison between the switch's configuration information and the corresponding slot information in the global VCS configuration database allows the switch to be added to the VCS using the same slot (i.e., the slot with the same mappedID) in the global configuration database.
  • FIG. 7D presents a flowchart illustrating the general operation of a switch during a join process, in accordance with one embodiment of the present invention. This operation applies to both a joining switch and a neighboring VCS member switch.
  • the logical FC switch within a switch receives an SCN notification on a newly detected switch domain ID on a link (operation 760 ).
  • the system then exchanges the VCS four-tuple with the neighbor switch (operation 762 ).
  • the system then waits for an INVITE from the neighbor member switch (operation 766 ). After the INVITE is received, the system determines whether the configuration information in the INVITE causes any un-resolved conflict with the local switch (operation 768 ). If there is an un-resolved conflict, the system rejects the INVITE (operation 770 ). Otherwise, the system accepts the INVITE (operation 772 ).
  • the system then further determines whether there is an existing slot in the global configuration database with the same mappedID as the joining switch (operation 774 ). If such a slot exists, the system then sends the INVITE to the joining switch (operation 775 ) and determines whether there is any un-resolved conflict between the configuration information stored in this slot and the information provided by the joining switch (operation 780 ). If so, the system revokes the INVITE (operation 782 ). Otherwise, the system updates the global configuration database with the joining switch's configuration information and propagates the update to all other member switches (operation 784 ).
  • the system allocates an interim slot in the global configuration database (operation 776 ), and sends an INVITE to the joining switch (operation 778 ). After receiving an INVITE acceptance from the joining switch (operation 779 ), the system then updates the global configuration database (operation 784 ) and completes the join process.
  • each VCS switch unit performs source MAC address learning, similar to what an Ethernet bridge does.
  • Each ⁇ MAC address, VLAN ⁇ tuple learned on a physical port on a VCS switch unit is registered into the local Fibre Channel Name Server (FC-NS) via a logical Nx_Port interface corresponding to that physical port. This registration binds the address learned to the specific interface identified by the Nx_Port.
  • FC-NS Fibre Channel Name Server
  • Each FC-NS instance on each VCS switch unit coordinates and distributes all locally learned ⁇ MAC address, VLAN ⁇ tuples with every other FC-NS instance in the fabric. This feature allows the dissemination of locally learned ⁇ MAC addresses, VLAN ⁇ information to every switch in the VCS.
  • the learned MAC addresses are aged locally by individual switches.
  • FIG. 8 presents a flowchart illustrating the process of looking up an ingress frame's destination MAC address and forwarding the frame in a VCS, in accordance with one embodiment of the present invention.
  • a VCS switch receives an Ethernet frame at one of its Ethernet ports (operation 802 ).
  • the switch then extracts the frame's destination MAC address and queries the local FC Name Server (operation 804 ).
  • the switch determines whether the FC-NS returns an N_Port or an NL_Port identifier that corresponds to an egress Ethernet port (operation 806 ).
  • the switch forwards the frame to the identified N_Port or NL_Port (operation 808 ). Otherwise, the switch floods the frame on the TRILL multicast tree as well as on all the N_Ports and NL_Ports that participate in that VLAN (operation 810 ).
  • This flood/broadcast operation is similar to the broadcast process in a conventional TRILL RBridge, wherein all the physical switches in the VCS will receive and process this frame, and learn the source address corresponding to the ingress RBridge.
  • each receiving switch floods the frame to its local ports that participate in the frame's VLAN (operation 812 ).
  • FIG. 9 illustrates how data frames and control frames are transported in a VCS, in accordance with an embodiment of the present invention.
  • a VCS 930 includes member switches 934 , 936 , 938 , 944 , 946 , and 948 .
  • An end host 932 is communicating with an end host 940 .
  • Switch 934 is the ingress VCS member switch corresponding to host 932
  • switch 938 is the egress VCS member switch corresponding to host 938 .
  • host 932 sends an Ethernet frame 933 to host 940 .
  • Ethernet frame 933 is first encountered by ingress switch 934 .
  • switch 934 Upon receiving frame 933 , switch 934 first extracts frame 933 's destination MAC address.
  • Switch 934 then performs a MAC address lookup using the Ethernet name service, which provides the egress switch identifier (i.e., the RBridge identifier of egress switch 938 ). Based on the egress switch identifier, the logical FC switch in switch 934 performs a routing table lookup to determine the next-hop switch, which is switch 936 , and the corresponding output port for forwarding frame 933 . The egress switch identifier is then used to generate a TRILL header (which specifies the destination switch's RBridge identifier), and the next-hop switch information is used to generate an outer Ethernet header.
  • the egress switch identifier i.e., the RBridge identifier of egress switch 938
  • switch 934 encapsulates frame 933 with the proper TRILL header and outer Ethernet header, and sends the encapsulated frame 935 to switch 936 .
  • switch 936 Based on the destination RBridge identifier in the TRILL header of frame 935 , switch 936 performs a routing table lookup and determines the next hop. Based on the next-hop information, switch 936 updates frame 935 's outer Ethernet header and forwards frame 935 to egress switch 938 .
  • switch 938 Upon receiving frame 935 , switch 938 determines that it is the destination RBridge based on frame 935 's TRILL header. Correspondingly, switch 938 strips frame 935 of its outer Ethernet header and TRILL header, and inspects the destination MAC address of its inner Ethernet header. Switch 938 then performs a MAC address lookup and determines the correct output port leading to host 940 . Subsequently, the original Ethernet frame 933 is transmitted to host 940 .
  • the logical FC switches within the physical VCS member switches may send control frames to one another (for example, to update the VCS global configuration database or to notify other switches of the learned MAC addresses).
  • control frames can be FC control frames encapsulated in a TRILL header and an outer Ethernet header.
  • switch 944 can sends a TRILL-encapsulated FC control frame 942 to switch 946 .
  • Switch 946 can forward frame 942 just like a regular data frame, since switch 946 is not concerned with the payload in frame 942 .
  • Today's server virtualization infrastructure (e.g. a Hypervisor, also called virtual machine monitor) associates a server side (e.g. Hypervisor or adapter) Virtual Ethernet Bridge (VEB) port profile to each Ethernet MAC address used by a virtual machine (VM) to access the network through a VEB port.
  • VEB's port profile attributes includes: the types of frames allowed on the port (e.g. all frames, only frames tagged with certain VLAN values, or untagged frames), the VLAN identifiers that are allowed to be used, and rate limiting attributes (e.g. port or access-control based rate limits).
  • the VEB's port profile migrates with it.
  • today's server virtualization infrastructure provides automated port profile migration of the server's VEB port(s) that are associated with a VM.
  • Today's layer-2 networks do not provide a mechanism for automatically migrating switch access and traffic controls associated with an end-point device (e.g., a VM), when that device migrates from one switch to another.
  • the migration may be physical, such as an Operating System image (application, middleware, Operating System and associated state) that is running on one physical system and is migrated to another system.
  • the migration may be also be virtual, such as an Operating System image (OS image) that is running over a Hypervisor on one system and is migrated to run over a Hypervisor on another system.
  • OS image Operating System image
  • Embodiments of the present invention provides a mechanism for automatically migrating port profiles resident in a switch and associated with an OS image to a port on a second switch, when that OS image migrates from one physical end-host system to another end-host system, which is attached to the second switch.
  • FIG. 10 illustrates a logical VCS access layer (VAL) which includes an automatic port profile manager, in accordance with one embodiment of the present invention.
  • a VCS 1000 is coupled with a number of physical server systems, such as system 1002 .
  • Each physical server system runs a number of virtual machines (VMs, also called virtual servers).
  • system 1002 includes four VMs, one of which is VM 1004 .
  • a VM may be dedicated to a certain application (e.g., instant messaging services, directory services, data base applications, etc.) and may have its own requirement on the network.
  • a VM running mission-critical applications may require a separate VLAN within VCS 1000 and may have more strict QoS requirement (such as guaranteed port bandwidth, low latency, and guaranteed packet delivery).
  • a VM running non-critical applications may have much lower requirements.
  • the switches within VCS 100 which are coupled externally to the physical end-host systems form a logical VCS access layer (VAL) 1010 .
  • the automatic migration of port profiles (AMPP) is implemented in VAL 1010 .
  • various port profiles which are often tailored to different requirements of the VMs, are created and distributed to all the member switches in VCS 1000 .
  • the VM's source MAC address is recognized and used to identify the corresponding port profile, which is then applied to the appropriate ingress switch port.
  • the MAC-address detection mechanism can quickly identify the new physical switch port to which the VM is coupled to, and apply the same port profile to the new port.
  • FIG. 11 illustrates an example of the operation of AMPP, in accordance with one embodiment of the present invention.
  • a VCS 1100 includes two switches 1120 and 1122 , which are coupled to two physical servers, 1116 and 1118 , respectively.
  • Physical server 1116 hosts four VMs, 1102 , 1104 , 1106 , and 1008 .
  • Each VM has a virtual port (VP, or virtual network interface card, VNIC).
  • VM 1102 has a VP 1110 .
  • a respective VP is assigned a virtual MAC address.
  • the four VPs are logically coupled to a virtual switch 1112 which is provided by a hypervisor 1114 .
  • Virtual switch 1112 is responsible for dispatching outgoing and incoming traffic through a physical NIC 1117 .
  • an Ethernet frame generated by a respective VM has the virtual MAC of the corresponding VP as its source address.
  • virtual switch 1112 functions as an aggregation point that provides a link to the ingress member switch in VCS 1100 .
  • Physical server 1118 has a similar architecture.
  • a VM can migrate from one physical server to another (e.g., “VMotion” function provided by VMware). This migration can be event-driven or pre-scheduled. Such migration is often used to cope with changing dynamics in a number of parameters, such as server load, power consumption, resource utilization, etc.
  • one or more port profiles can be created to specify a number of requirements/restrictions/limitations that should be enforced at a VCS switch port corresponding to one or more VMs.
  • a port profile for VM 1102 (which can be identified by the virtual MAC address of VP 1110 ) can be created and distributed to every member switch of VCS 1100 .
  • switch 1120 When VM 1102 sends its first Ethernet frame to the network, switch 1120 would learn this source MAC address. Upon learning VP 1110 's MAC address, switch 1120 then searches its port profile database and identifies the matching port profile. Subsequently, the identified port profile is applied to the port on switch 1120 which is coupled to system 1116 .
  • the same port profile is applied to the port where the matching MAC address is the destination MAC address of a frame. This way, the same network parameters are enforced at both ingress and egress ports of the VCS.
  • the port profile might include “soft” parameters. In other words, the requirements and limitations in the port profile may be specific to certain MAC addresses, and may not be “hard” limitations on the physical parameters of the switch port, since traffic from/to multiple VMs is handled by the same physical switch port.
  • VCS 1100 provides a mechanism that distributes all the port profiles and the port-profile-to-MAC mapping information to all the member switches.
  • the port profiles can be created using a command line interface (CLI) or other network management software.
  • CLI command line interface
  • the target switch port in the VCS can automatically activate the correct port profile configuration upon migration of a VM (such as a VMware VMotion).
  • FIG. 12A illustrates exemplary port profile contents, in accordance with one embodiment of the present invention.
  • a port profile can contain the entire configuration needed for a VM to gain access to a LAN or WAN, which can include: Fibre Channel over Ethernet (FCoE) configuration, VLAN configuration, QoS related configuration, and security related configuration (such as access control lists, ACLs).
  • FCoE Fibre Channel over Ethernet
  • VLAN virtual local area network
  • QoS related configuration such as access control lists, ACLs
  • security related configuration such as access control lists, ACLs
  • a port profile can be capable of operating as a self contained configuration container. In other words, if a port profile is applied to a new switch without any additional configuration, the port profile should be sufficient to set the switch's global and local (interface level) configuration and allow the switch to start carrying traffic.
  • a VLAN configuration profile within a port profile can define:
  • a QoS configuration profile within a port profile can define:
  • FCoE configuration profile within a port profile defines the attributes needed for the port to support FCoE, which can include:
  • a security configuration profile within a port profile defines the security rules needed for the server port. However, the security rules can be different at different ports, so some of the locally configured ACLs can be allowed to override conflicting rules from a port profile.
  • a typical security profile can contain the following attributes:
  • each port profile can have one or more MAC addresses associated with it.
  • FIG. 12B illustrates three exemplary port profiles, in accordance with on embodiment of the present invention.
  • port profile PP-1 is associated with 5 MAC addresses. These MAC address can be virtual MAC addresses assigned to different VMs.
  • the port-profile-to-MAC mapping information is distributed throughout the VCS.
  • a port profile can be activated on a server port in three ways: (1) when a hypervisor binds a MAC address to a port profile ID; (2) through regular MAC learning; and (3) through a manual configuration process via a management interface.
  • FIG. 13 illustrates how forwarding is achieved between VMs based on port profile membership, in accordance with one embodiment of the present invention.
  • port profile 1 has two members: MAC-1 and MAC-3.
  • Port profile 2 has four members: MAC-2, MAC-4, MAC-5, and MAC-6. All the VMs belong to the same VLAN X.
  • a policy group ID GID
  • All the MAC addresses mapped to the same port profile should belong to the same policy group which dictates the forwarding boundary. This configuration allows enforcing different forwarding domains within a VLAN, as illustrated in FIG. 13B .
  • the system then ensures that both the source MAC address and destination MAC address are part of the same port profile.
  • FIG. 14 presents a flowchart illustrating the process of creating and applying a port profile, in accordance with one embodiment of the present invention.
  • the system receives a user created port profile with the corresponding VM's MAC address (operation 1402 ). This MAC address is then associated with the port profile and can be later used to identify the profile.
  • the system determines whether the new profile creates dependencies on other existing profiles or conflicts (operation 1404 ). If so, the system allows the user to resolve the conflicting configuration and/or dependencies (operation 1406 ).
  • the system distributes the port profile and the corresponding VM MAC address to every member switch throughout the VCS fabric (operation 1408 ).
  • the system detects a matching virtual MAC address from the received ingress packets (operation 1410 ). Based on the learned MAC address, the system then activates the corresponding port profile on the switch port (operation 1412 ).
  • FIG. 15 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • the VCS member switch is a TRILL RBridge 1500 running special VCS software.
  • RBridge 1500 includes a number of Ethernet communication ports 1501 , which can transmit and receive Ethernet frames and/or TRILL encapsulated frames.
  • Also included in RBridge 1500 is a packet processor 1502 , a virtual FC switch management module 1504 , a logical FC switch 1505 , a VCS configuration database 1506 , an AMPP management module 1507 , and a TRILL header generation module 1508 .
  • packet processor 1502 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames.
  • Virtual FC switch management module 1504 maintains the state of logical FC switch 1505 , which is used to join other VCS switches using the FC switch fabric protocols.
  • VCS configuration database 1506 maintains the configuration state of every switch within the VCS.
  • TRILL header generation module 1508 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • AMPP management module 1507 Upon learning a new MAC address from a VM, AMPP management module 1507 identifies the port profile corresponding to the learned MAC, and applies the identified port profile. This dynamic port profile application process can respond to VM migration within a very short period of time, thereby facilitating automatic port configuration in a virtual computing environment.
  • Today's server virtualization infrastructure e.g. a Hypervisor, also called virtual machine monitor
  • a Hypervisor also called virtual machine monitor
  • VEBs virtual Ethernet bridges
  • Each virtual switch serves a number of virtual machines.
  • VEBs virtual Ethernet bridges
  • the number of communication sessions among the virtual machines can be quite large.
  • a network link or port fails, the failure would typically disrupt the reachability to one or more virtual machines. This disruption can affect the communication sessions of some of the virtual machines.
  • reachability disruption only triggers a topology change and/or MAC address learning update in the network, and the source virtual machines are not notified about these updates.
  • Embodiments of the present invention facilitate advanced link tracking by monitoring any reachability disruption in the network and notifying the affected hypervisor.
  • the hypervisor can re-configure the connectivity of the virtual machines under its control to bypass the failed link or port.
  • this advanced link tracking function can be carried out in a logical VCS access layer.
  • FIG. 16 illustrates an exemplary configuration of advanced link tracking in a VCS, in accordance with one embodiment of the present invention.
  • a VCS 1600 includes four switches (which can be RBridges), 1620 , 1622 , 1624 , and 1626 .
  • a physical server 1618 is coupled to both switches 1622 and 1624 via two network interface cards (NICs), 1603 and 1605 , respectively.
  • Physical server 1618 hosts four VMs, 1622 , 1624 , 1626 , and 1628 , which are managed by a hypervisor 1601 .
  • Hypervisor 1601 provides two virtual switches, 1602 and 1604 .
  • Each VM has two virtual ports (VPs), and is coupled to both virtual switches 1602 and 1604 via the VPs.
  • VPs virtual ports
  • each VM within physical server 1618 is dual-homed with virtual switches 1602 and 1604 .
  • This configuration provides redundancy to each VM, so that when one of the physical NICs (i.e., NIC 1603 or 1605 ) fails, hypervisor 1601 can instruct the VMs to use the other working NIC.
  • VMs 1622 and 1624 are configured to communicate via virtual switch 1602
  • VMs 1626 and 1628 are configured to communicate via virtual switch 1604 .
  • VCS 1600 Also coupled to VCS 1600 is physical servers 1617 , which has a similar configuration as server 1618 .
  • Server 1617 includes four VMs, 1632 , 1634 , 1636 , and 1638 . These four VMs are each dual-homed with virtual switches 1642 and 1644 , which are provided by hypervisor 1641 .
  • Virtual switch 1642 is coupled to VCS member switch 1620 via a NIC 1643
  • virtual switch 1644 is coupled to VCS member switch 1626 via a NIC 1645 .
  • VMs 1632 and 1634 communicate with VCS 1600 via virtual switch 1642 and NIC 1643
  • VMs 1636 and 1638 communicate with VCS 1600 via virtual switch 1644 and NIC 1645 .
  • VCS member switch 1626 the traffic between VMs 1622 and 1624 and VMs 1636 and 1638 is normally carried by VCS member switch 1626 .
  • this reachability update information is not only reflected in the VCS topology update (which is handled by the routing protocol within VCS 1600 ), but also communicated to hypervisor 1601 via NIC 1603 .
  • This update can allow hypervisor 1601 to quickly re-configure VMs 1622 and 1624 , so that these two VMs use virtual switch 1604 and NIC 1605 to access VCS 1600 .
  • This way, the traffic from VMs 1622 and 1624 can still reach VMs 1636 and 1638 via switch 1624 , switch 1620 , NIC 1643 , and virtual switch 1642 .
  • the new data path bypasses the failed link between switches 1620 and 1626 . This re-configuration can take place shortly after the link failure is detected, thereby facilitating fast recovery at the source VMs.
  • FIG. 17 illustrates an example where advanced link tracking allows virtual machines to re-route egress traffic when a link fails, in accordance with one embodiment of the present invention.
  • two servers 1702 and 1704 are coupled to a VCS 1700 .
  • Server 1702 hosts four VMs, 1706 , 1708 , 1710 , and 1712 , all of which are dual-homed with virtual switches 1714 and 1716 .
  • VMs 1706 and 1708 access VCS 1700 via VS 1714
  • VMs 1710 and 1712 access VCS 1700 via VS 1716 .
  • Server 1704 have a similar configuration as server 1702 .
  • VCS 1700 can communicate with a third entity which maintains the connectivity-pattern information among all the VMs (such as the vCenter by VMware) to obtain information on the affected VMs.
  • VCS 1700 can notify every external port of the lost reachability, and let the individual hypervisor to determine whether re-configuration of the VM-to-VS connectivity is necessary.
  • FIG. 18 presents a flowchart illustrating the process of advance link tracking, in accordance with one embodiment of the present invention.
  • the system first detects a link (or port) failure in the VCS (operation 1802 ). The system then determines whether the failure affects reachability of an end host (operation 1804 ). If the failure does not affect reachability of any end host, it is assumed that VCS can recover from the failure after its topology converges and the routing protocol updates every switch's forwarding table. If the reachability of an end host is affected, the system then optionally identifies ingress port(s) which are in communication with the affected end host(s) (operation 1806 ). Subsequently, the system notifies the end hosts via the ingress ports of the reachability disruption (operation 1808 ).
  • FIG. 19 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • the VCS member switch is a TRILL RBridge 1900 running special VCS software.
  • RBridge 1900 includes a number of Ethernet communication ports 1901 , which can be coupled to one or more servers hosting virtual machines and which can transmit and receive Ethernet frames and/or TRILL encapsulated frames.
  • Also included in RBridge 1900 is a packet processor 1902 , a virtual FC switch management module 1904 , a logical FC switch 1905 , a VCS configuration database 1906 , an advanced link tracking module 1907 , and a TRILL header generation module 1908 .
  • packet processor 1902 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames.
  • Virtual FC switch management module 1904 maintains the state of logical FC switch 1905 , which is used to join other VCS switches using the FC switch fabric protocols.
  • VCS configuration database 1906 maintains the configuration state of every switch within the VCS.
  • TRILL header generation module 1908 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • advanced link tracking module 1907 Upon learning about disrupted reachability in the VCS, advanced link tracking module 1907 identifies the port(s) which are affected by the disruption, and notifies the hypervisor of the disruption. This notification can allow the hypervisor to expedite the re-configuration of the affected VMs and minimize service disruption. Furthermore, advanced link tracking module 1907 also monitors the health of all the links corresponding to ports 1901 . Upon detection of any link or port failure, advanced link tracking module 1907 can notify other switches in the VCS of the link state change and any reachability disruption.
  • Today's server virtualization infrastructure e.g. a Hypervisor, also called virtual machine monitor
  • a Hypervisor also called virtual machine monitor
  • VEBs virtual Ethernet bridges
  • Each virtual switch serves a number of virtual machines.
  • VCS virtual Ethernet bridges
  • the number of communication sessions among the virtual machines can be quite large.
  • different virtual machines belong to different customers, and different customers usually require dedicated and isolated network resources for security and quality of service purposes.
  • each customer's virtual machines are placed in the same group based on the MAC address and VLAN tag of their associated virtual ports.
  • the grouping information is replicated and stored at every VCS member switch.
  • the member switch can enforce the grouping policy by looking up the group ID corresponding to the frame's source MAC address and VLAN tag. If the frame's destination MAC address combined with the VLAN tag belong to the same group, then the frame is forwarded to the proper next-hop switch. Otherwise, the frame is dropped. This configuration ensures that each customer's virtual machines only communicate within themselves, thereby creating virtual partitions within a VCS.
  • FIG. 20 illustrates an exemplary VCS network environment with virtual port grouping, in accordance with one embodiment of the present invention.
  • a VCS 2000 includes four switches (which can be RBridges), 2020 , 2022 , 2024 , and 2026 .
  • a physical server 2018 is coupled to both switches 2022 and 2024 via two network interface cards (NICs), 2003 and 2005 , respectively.
  • Physical server 2018 hosts four VMs, 2022 , 2024 , 2026 , and 2028 , which are managed by a hypervisor 2001 .
  • Hypervisor 2001 provides two virtual switches, 2002 and 2004 .
  • Each VM has two virtual ports (VPs), and is coupled to both virtual switches 2002 and 2004 via the VPs.
  • VPs virtual ports
  • each VM within physical server 2018 is dual-homed with virtual switches 2002 and 2004 .
  • This configuration provides redundancy to each VM, so that when one of the physical NICs (i.e., NIC 2003 or 2005 ) fails, hypervisor 2001 can instruct the VMs to use the other working NIC.
  • VMs 2022 and 2024 are configured to communicate via virtual switch 2002
  • VMs 2026 and 2028 are configured to communicate via virtual switch 2004 .
  • VCS 2000 Also coupled to VCS 2000 is physical servers 2017 , which has a similar configuration as server 2018 .
  • Server 2017 includes four VMs, 2032 , 2034 , 2036 , and 2038 . These four VMs are each dual-homed with virtual switches 2042 and 2044 , which are provided by hypervisor 2041 .
  • Virtual switch 2042 is coupled to VCS member switch 2020 via a NIC 2043
  • virtual switch 2044 is coupled to VCS member switch 2026 via a NIC 2045 .
  • VMs 2032 and 2034 communicate with VCS 2000 via virtual switch 2042 and NIC 2043
  • VMs 2036 and 2038 communicate with VCS 2000 via virtual switch 2044 and NIC 2045 .
  • VMs 2024 , 2028 , 2032 , and 2036 belong to the same customer (as indicated by the shading in each drawing element in FIG. 20 ), and that these VMs are restricted to communicate only within themselves.
  • the customer would set a particular VLAN value to these VMs.
  • the customer VLAN value might not work within VCS 2000 , because the customer VLAN value might conflict with existing VLAN settings within VCS 2000 .
  • One way to solve this problem is to place the MAC addresses corresponding to the virtual ports of a particular customer's virtual machines, together with the customer's VLAN value, in a customer-specific group, and assign a unique group ID to this group.
  • This grouping information can be a data structure that maps a ⁇ MAC, VLAN TAG ⁇ tuple to a group ID. This data structure is replicated and distributed to every member switch in VCS 2000 .
  • FIG. 21A illustrates an exemplary data structure for storing virtual port grouping information, in accordance with one embodiment of the present invention.
  • each ⁇ MAC, VLAN TAG ⁇ tuple maps to a group ID, which is used to restrict forwarding of frames between different groups.
  • FIG. 21B illustrates logical virtual networks within a VCS as a result of virtual port grouping, in accordance with one embodiment of the present invention.
  • two servers 2102 and 2104 are coupled to a VCS 2100 .
  • Server 2102 hosts four VMs, 2106 , 2108 , 2110 , and 2112 , all of which are dual-homed with virtual switches 2114 and 2116 .
  • Server 2104 hosts four VMs, 2122 , 2124 , 2126 , and 2128 , all of which are dual-homed with virtual switches 2118 and 2120 .
  • VMs 2108 , 2112 , 2124 , and 2128 belong to the same customer and are expected to communicate only within themselves.
  • a VM administrator 2124 configures VMs 2108 , 2112 , 2124 , and 2128 with the same VLAN tag corresponding to their common customer.
  • the MAC address of the virtual ports for these machines, together with the designated VLAN tag, are assigned with the same group ID.
  • VMs 2106 , 2110 , 2122 , and 2126 and their respective VLAN tags are assigned a different group ID. This grouping information is then communicated to VCS 2100 and stored at every member switch.
  • the virtual port grouping creates two virtual networks 2120 and 2121 within VCS 2100 .
  • FIG. 22 presents a flowchart illustrating the process of enforcing virtual port grouping, in accordance with one embodiment of the present invention.
  • a VCS member switch receives in ingress frame from an end host (operation 2202 ). The switch then identifies the group ID based on the source MAC address and VLAN tag of the received frame (operation 2204 ). In addition, the switch further determines whether the destination MAC address and VLAN tag of the received frame point to the same group ID (operation 2206 ). If so, the switch identifies the output port based on the destination MAC address and VLAN tag (operation 2208 ), and forwards the frame to the identified output port (operation 2210 ). Otherwise, the frame is discarded (operation 2212 ).
  • FIG. 23 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • the VCS member switch is a TRILL RBridge 2300 running special VCS software.
  • RBridge 2300 includes a number of Ethernet communication ports 2301 , which can be coupled to one or more servers hosting virtual machines and which can transmit and receive Ethernet frames and/or TRILL encapsulated frames.
  • Also included in RBridge 2300 is a packet processor 2302 , a virtual FC switch management module 2304 , a logical FC switch 2305 , a VCS configuration database 2306 , virtual port grouping module 2307 , and a TRILL header generation module 2308 .
  • packet processor 2302 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames.
  • Virtual FC switch management module 2304 maintains the state of logical FC switch 2305 , which is used to join other VCS switches using the FC switch fabric protocols.
  • VCS configuration database 2306 maintains the configuration state of every switch within the VCS.
  • TRILL header generation module 2308 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • Virtual port grouping module 2307 is responsible for maintaining a copy of the grouping information and enforcing the grouping policies. As described in conjunction with FIG.
  • virtual port grouping module 2307 can communicate with a VM administrator (such as the VMware Vcenter) to obtain the virtual port grouping information (e.g., the complete data structure illustrated in FIG. 21A ).
  • a VM administrator such as the VMware Vcenter
  • VCS allows an interconnected fabric of RBridges to function as a single logical switch.
  • the VCS name services facilitate fast distribution of run-time network state changes, including newly learned MAC addresses (which is referred to as “Ethernet name service” or “Ethernet NS” in this disclosure) and multi-chassis trunk (MCT) port state updates (which is referred to as “MCT name service” or “MCT NS” in this disclosure). More details on MCT are provided in U.S.
  • the Ethernet NS provides the ability to distribute various information across the VCS.
  • the MAC information learned at one member switch is distributed to all other member switches, which facilitates fast MAC moves (for example, during migration of virtual machines) and global MAC learning.
  • layer-2 multicast information which can be a multicast MAC address with corresponding switch/port identifiers and VLAN tag, can be distributed to facilitate efficient VCS-wide multicast.
  • Ethernet NS provides a distribution mechanism and does not maintain a central storage of the MAC-related knowledge base. In other words, the Ethernet NS knowledge database is replicated and stored distributively among all the VCS member switches.
  • Ethernet NS also provides VCS-wide distribution of multicast MAC-to-RBridge/Port mapping information which can be obtained by Internet Group Management Protocol (IGMP) snooping.
  • IGMP Internet Group Management Protocol
  • Ethernet NS distributes this information to all RBridges, thereby allowing the VCS to behave as a single switch. By tracking and forwarding IGMP join and leave information, the Ethernet NS can efficiently track the multicast MAC information and maintain an accurate layer-2 multicast group.
  • MCT multi-chassis trunking
  • FIG. 24 illustrates an example of name service operation in a VCS, in accordance with one embodiment of the present invention.
  • a VCS 2400 includes four member switches (Rbridges), 2402 , 2404 , 2406 , and 2408 .
  • Rbridges member switches
  • 2402 Assume that an end host 2414 is coupled to switch 2402 during operation. When end host 2414 sends its first Ethernet frame, switch 2402 would not recognize the source MAC address of this ingress frame. Upon receiving this ingress frame, switch 2402 then determines the port (or interface) on which the frame arrives and the frame's VLAG tag.
  • switch 2402 assembles an Ethernet NS update frame which indicates the learned MAC address (which corresponds to end host 2414 ), its switch identifier (which in one embodiment is the RBridge ID of switch 2402 ), the port identifier, and the VLAG tag for the frame.
  • this frame is an FC registered state change notification (RSCN) encapsulated in a TRILL header.
  • switch 2402 can obtain the information of all other member switches in the VCS by looking up the global configuration database.
  • switch 2402 can send the Ethernet NS update frame to switches 2404 , 2408 , and 2406 , respectively.
  • each member switch updates its own MAC database accordingly. In this way, when one of the member switches receives an Ethernet frame destined to end-host 2414 , it can forward that frame to switch 2402 (instead of flooding the frame to all of its ports).
  • MCT group 2416 is formed by an end host 2412 which is dual-homed with switches 2406 and 2408 .
  • switch 2406 is the primary RBridge in MCT group 2416 .
  • switch 2406 assigns a virtual RBridge 2410 to MCT group 2410 .
  • switch 2406 notifies the rest of VCS 2400 about the MAC address of end host 2412 .
  • the NS update associated the MAC address of end host 2412 indicates the identifier of virtual RBridge 2410 (instead of the identifier of either switch 2406 or switch 2408 ).
  • VCS 2400 can associate end host 2412 with virtual RBridge 2410 .
  • a member switch in VCS 2400 would forward the frame toward virtual RBridge 2410 (i.e., by setting RBridge 2410 as the destination RBridge in the TRILL header).
  • switch 2406 is also responsible for distributing the link state information with respect to the virtual connectivity between virtual RBridge 2410 and switches 2406 and 2408 (indicated by the dotted lines).
  • primary RBridge 2406 is responsible for updating the rest of the VCS 2400 that host 2412 's MAC address is no longer associated with virtual RBidge 2410 . Instead, the MAC address of host 2412 is now associated with the switch to which host 2412 remains connected. In a further embodiment, it can be the responsibility of the switch that remains connected to host 2412 to distribute the updated MAC address association to the rest of VCS 2400 .
  • FIG. 25 presents a flowchart illustrating the process of distributing learned MAC information by the Ethernet name service in a VCS, in accordance with one embodiment of the present invention.
  • a VCS member switch detects an ingress frame with a new source MAC address (operation 2502 ). The switch then identifies the port on which the ingress frame is received (operation 2504 ). Subsequently, the switch assembles an Ethernet NS update frame with the learned MAC address, the switch identifier, port identifier, and VLAN tag (operation 2506 ). The switch then distributes the Ethernet NS update frames to all member switches in the VCS (operation 2508 ).
  • FIG. 26 presents a flowchart illustrating the process of distributing information of a learned MAC address via an MCT, in accordance with one embodiment of the present invention.
  • one of the switches in a MCT group detects an ingress frame with a new source MAC address (operation 2602 ).
  • the switch determines whether the end host which generates the frame is dual-homed with the MCT group (operation 2604 ).
  • the switch can make this determination by communicating with the other switch of the MCT group.
  • the switch can inspect the link aggregation group (LAG) ID of the ingress frame to determine whether the end host is transmitting using a LAG.
  • LAG link aggregation group
  • the switch then assembles an Ethernet NS update frame with the MAC address, the virtual RBridge identifier corresponding to the MCT, a port identifier, and the VLAG tag of the frame (operation 2606 ).
  • the switch assembles an Ethernet NS updated frame with the MAC address, the local physical switch identifier (as opposed to the virtual RBridge ID), the identifier of the port on which the frame is received, and the frame's VLAN tag (operation 2607 ).
  • the switch then distributes the Ethernet NS update frames to all the member switches in the VCS (operation 2608 ).
  • FIG. 27 presents a flowchart illustrating the process of updating the link state in an MCT group, in accordance with one embodiment of the present invention.
  • one of the MCT partner switches detects a link or port failure which is part of the MCT group (operation 2702 ). Note that this failure can be detected locally (which means a port on the local switch or a link coupled to a local port has failed), or be detected remotely (which means that the failure occurs on the partner switch and the local switch is notified of the failure by the partner switch).
  • the switch determines whether the MCT end host is still connected to the local switch (operation 2704 ).
  • the local switch optionally notifies the other partner switch in the MCT of the failure (operation 2710 ) and takes no further actions, assuming that the partner switch will assume responsibility of updating the link state (using, for example, the same procedure illustrated in FIG. 27 ).
  • the switch then assembles an NS update frame with the end host's MAC address, the local switch's identifier (e.g., the physical RBridge ID of the local switch), the identifier of the port thought which the end host is connected, and the proper VLAN tag (operation 2706 ).
  • the switch then distributes the NS update frames to all member switches in the VCS (operation 2708 ).
  • FIG. 28 illustrates an exemplary switch that facilitates formation of a virtual cluster switch with Ethernet and MCT name services, in accordance with an embodiment of the present invention.
  • the VCS member switch is a TRILL RBridge 2800 running special VCS software.
  • RBridge 2800 includes a number of Ethernet communication ports 2801 , which can transmit and receive Ethernet frames and/or TRILL encapsulated frames.
  • Also included in RBridge 2800 is a packet processor 2802 , a virtual FC switch management module 2804 , a logical FC switch 2805 , a VCS configuration database 2806 , a name services management module 2807 , and a TRILL header generation module 2808 .
  • packet processor 2802 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames.
  • Virtual FC switch management module 2804 maintains the state of logical FC switch 2805 , which is used to join other VCS switches using the FC switch fabric protocols.
  • VCS configuration database 2806 maintains the configuration state of every switch within the VCS.
  • TRILL header generation module 2808 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • NS management module 2807 Based on the extracted MAC addresses of incoming frames, NS management module 2807 distributes the NS update frames to the rest of the VCS. NS management module 2807 also maintains a copy of NS database 2809 .
  • NS database 2809 stores all the learned MAC address information from every member switch in the VCS.
  • FIG. 29 illustrates an exemplary equal cost multi-pathing configuration in a VCS, in accordance with one embodiment of the present invention.
  • a VCS 2930 includes switches 2934 , 2936 , 2938 , 2944 , 2946 , and 2948 .
  • a traffic flow enters VCS 2930 via ingress switch 2934 and exits VCS 2930 via egress switch 2948 .
  • switch 2934 's forwarding engine when determining the output interface, can return two possible output ports, corresponding to data paths 2950 and 2952 , respectively. If load balancing is desired, traffic from switch 2934 to switch 2948 can be split between the two data paths.
  • the traffic splitting can be based on any field in a frame header.
  • the traffic can be split based on VLAN tags.
  • the traffic can also be split based on layer-4 port numbers or application types.
  • the VCS can achieve loss-less packet transport by implementing a number of flow control mechanisms.
  • the VCS member switches implement a priority-based flow control (PFC)-like mechanism to ensure a loss-less packet transport between neighboring switches.
  • PFC priority-based flow control
  • traffic class is identified by the VLAN tag priority values.
  • Priority-based flow control is intended to eliminate frame loss due to congestion. This is achieved by a mechanism similar to the IEEE 802.3x PAUSE, but operating on individual priorities. This mechanism enables support for higher layer protocols that are highly loss sensitive while not affecting the operation of traditional LAN protocols utilizing other priorities. Details of PFC can be found in the IEEE 802.1Qbb standard, available at http://www.ieee802.org/1/pages/802.1bb.html, which is incorporated by reference herein.
  • the VCS member switch also employ enhanced transmission selection mechanisms, which support allocation of bandwidth amongst different traffic classes.
  • enhanced transmission selection will allow other traffic classes to use the available bandwidth.
  • the bandwidth-allocation priorities can coexist with strict priorities.
  • the VCS can prioritize traffic to provide different service characteristics to traffic classes. It is possible to share bandwidth between priorities carrying bursty loads rather than servicing them with strict priority while allowing strict priority for time-sensitive and management traffic requiring minimum latency. Also, when traffic at a given priority level does not use its allocation, it is possible to allow other priorities to use that bandwidth. Details of ETS can be found in the IEEE 802.1Qas standard, available at http://www.ieee802.org/1/pages/802.1az.html, which is incorporated by reference herein.
  • the VCS member switches can also implement congestion notification mechanisms to facilitate source-oriented flow control.
  • FIG. 30 illustrates an example of VCS congestion notification, in accordance with one embodiment of the present invention.
  • a source host 3002 is transmitting frames to a destination host 3004 via a VCS 3000 .
  • congestion occurs at an intermediary switch 3006 .
  • As frames accumulate at switch 3006 its corresponding queue 3008 becomes full.
  • a queue monitoring mechanism within switch 3006 is triggered when the content of queue 3008 passes a predetermined threshold.
  • switch 3006 can randomly select a frame in queue 3008 , extract the source MAC address of the selected frame to construct a congestion notification frame, and send this notification frame to the source device corresponding to the extracted source MAC address (which is host 3002 ).
  • Switch 3006 can perform the same action with multiple frames in queue 3008 , so that if multiple sources are contributing to the congestion, these sources can all be notified.
  • FIG. 31 illustrates an exemplary hybrid trunk in accordance with one embodiment of the present invention.
  • two link trunk groups 3106 and 3108 are formed between VCS member switches 3102 and 3104 .
  • trunk groups 3106 and 3108 can be identified by both switches without using a link aggregation group (LAG) ID
  • LAG link aggregation group
  • trunks 3106 and 3108 can be regarded as logical links.
  • LAG 3110 can also include other individual, physical links (not shown).
  • FIG. 32 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • the VCS member switch is a TRILL RBridge 3200 running special VCS software.
  • RBridge 3200 includes a number of Ethernet communication ports 3201 , which can transmit and receive Ethernet frames and/or TRILL encapsulated frames.
  • Also included in RBridge 3200 is a packet processor 3202 , a virtual FC switch management module 3204 , a logical FC switch 3205 , a VCS configuration database 3206 , a traffic management module 3207 , and a TRILL header generation module 3208 .
  • packet processor 3202 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames.
  • Virtual FC switch management module 3204 maintains the state of logical FC switch 3205 , which is used to join other VCS switches using the FC switch fabric protocols.
  • VCS configuration database 3206 maintains the configuration state of every switch within the VCS.
  • TRILL header generation module 3208 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • Traffic management module 3207 facilitates the aforementioned traffic management functions, such as multi-pathing, flow control, congestion notification, and hybrid trunking.
  • a converged network extension (CNE) device can be implemented in a two-box solution, wherein one box is capable of L2/L3/FCoE switching and is part of the VCS, and the other facilitates the WAN tunneling to transport Ethernet and/or FC traffic over WAN.
  • the CNE device can also be implemented in a one-box solution, wherein a single piece of network equipment combines the functions of L2/L3/FCoE switching and WAN tunneling.
  • VCS as a layer-2 switch uses TRILL as its inter-switch connectivity and delivers a notion of single logical layer-2 switch.
  • This single logical layer-2 switch delivers a transparent LAN service.
  • All the edge ports of VCS support standard protocols and features like Link Aggregation Control Protocol (LACP), Link Layer Discovery Protocol (LLDP), VLANs, MAC learning, etc.
  • LACP Link Aggregation Control Protocol
  • LLDP Link Layer Discovery Protocol
  • VCS achieves a distributed MAC address database using Ethernet Name Service (eNS) and attempts to avoid flooding as much as possible.
  • eNS Ethernet Name Service
  • VCS also provides various intelligent services, such as virtual link aggregation group (vLAG), advance port profile management (APPM), End-to-End FCoE, Edge-Loop-Detection, etc.
  • vLAG virtual link aggregation group
  • APM advance port profile management
  • End-to-End FCoE End-to-End FCoE
  • Edge-Loop-Detection
  • each data center is represented as a single logical RBridge.
  • This logical RBridge can be assigned a virtual RBridge ID or use the RBridge ID of the CNE device that performs the WAN tunneling.
  • FIG. 33 illustrates an exemplary network architecture which includes CNE devices for facilitate cross-data-center communications, in accordance with one embodiment of the present invention.
  • two data centers 3344 and 3346 are coupled to a WAN 3326 via gateway routers 3324 and 3328 , respectively.
  • Data center 3344 includes a VCS 3316 , which couples to a number of hosts, such as host 3301 , via its member switches, such as switch 3310 .
  • Host 3301 includes two VMs 3302 and 3304 , which are coupled to virtual switches 3306 and 3308 in a dual-homing configuration.
  • virtual switches 3306 and 3308 reside on two network interface cards on host 3301 .
  • Virtual switches 3306 and 3308 are coupled to VCS member switch 3310 .
  • CNE device 3318 is configured to receive both Ethernet (or TRILL) traffic from member switch 3310 via an Ethernet (or TRILL) link 3312 , and FC traffic via FC link 3314 . Also coupled to CNE device 3318 is a target storage device 3320 , and a cloned target storage device 3322 (denoted by dotted lines). CNE device 3318 maintains an FCIP tunnel to data center 3346 across WAN 3326 via gateway routers 3324 and 3328 .
  • data center 3346 includes a VCS 3342 , which in turn includes a member switch 3332 .
  • Member switch 3332 is coupled to a host 3341 , which includes VMs 3334 and 3336 , both of which are coupled to virtual switches 3338 and 3340 .
  • VCS 3342 Also included in VCS 3342 is a CNE device 3330 .
  • CNE device is coupled to member switch 3332 via an Ethernet (TRILL) link and an FC link.
  • TRILL Ethernet
  • FC link FC link
  • CNE device 3330 is also coupled to target storage device 3322 and a clone of target storage device 3320 .
  • VM 3302 needs to move from host 3301 to host 3341 . Note that this movement is previously not possible, because virtual machines are visible only within the same layer-2 network domain.
  • a layer-3 device such as gateway router 3324 , all the identifying information for a particular virtual machine (which is carried in layer-2 headers) is lost.
  • CNE device extends the layer-2 domain from VCS 3316 to VCS 3342 , the movement of VM 3302 from data center 3344 to data center 3346 is now possible.
  • CNE device 3318 When forwarding TRILL frames from data center 3344 to data center 3346 , CNE device 3318 modifies the egress TRILL frames' header so that the destination RBridge identifier is the RBridge identifier assigned to data center 3346 . CNE device 3318 then uses the FCIP tunnel to deliver these TRILL frames to CNE device 3330 , which in turn forwards these TRILL frames to their respective layer-2 destinations.
  • VCS uses FC control plane to automatically form a fabric and assign RBridge identifiers to each member switch.
  • the CNE architecture keeps the TRILL and SAN fabrics separate between data centers. From a TRILL point of view, each VCS (which corresponds to a respective data center) is represented as a single virtual RBrdige.
  • the CNE device can be coupled to a VCS member switch with both a TRILL link and an FC link. The CNE device can join the VCS via a TRILL link.
  • the CNE devices keeps the TRILL VCS fabric and SAN (FC) fabric separate, the FC link between the CNE device and the member switch is configured for FC multi-fabric.
  • a data center 3408 is coupled to a WAN via a gateway router 3410
  • a data center 3420 is coupled to the WAN via a gateway router 3412
  • Data center 3408 includes a VCS 3406 , which includes a member switch 3404 .
  • CNE device 3402 is coupled to VCS member switch 3404 via a TRILL link and an FC link.
  • CNE device 3402 can join the VCS via the TRILL link.
  • the FC link allows CNE device 3402 to maintain a separate FC fabric with VCS member switch 3404 to carry FC traffic.
  • the FC port on CNE device 3402 is an FC EX_port.
  • the corresponding port on member switch 3404 is an FC E_port.
  • the port on CNE device 3402 on the WAN side (coupling to gateway router 3410 ) is an FCIP VE_port.
  • Data center 3420 has a similar configuration.
  • each data center's VCS includes a node designated as the ROOT RBridge for multicast purposes.
  • the CNE devices in the VCSs exchange each VCS's ROOT RBridge identifier.
  • the CNE devices also exchange each data center's RBridge identifier. Note that this RBridge identifier represents the entire data center. Information related to data-center RBridge identifiers is distributed as a static route to all the nodes in the local VCS.
  • FIG. 35A presents a diagram illustrating how CNE devices handle broadcast, unknown unicast, and multicast (BUM) traffic across data centers, in accordance with one embodiment of the present invention.
  • two data centers, DC-1 and DC-2 are coupled to an IP WAN via core IP routers.
  • the CNE device in DC-1 has an RBridge identifier of RB4, and the CNE device in DC-2 has an RBridge identifier of RB6.
  • a member switch RB1 is coupled to host A.
  • a member switch RB5 is coupled to host Z.
  • host A assembles an Ethernet frame 3502 , which has host Z's MAC address (denoted as MAC-Z) as its destination address (DA), and host A's MAC address (denoted as MAC-A) as its source address (SA).
  • member switch RB1 assembles a TRILL frame 3503 , whose TRILL header 3506 includes the RBridge identifier of data center DC-1's root RBridge (denoted as “DC1-ROOT”) as the destination RBridge, and RB1 as the source RBridge.
  • the outer Ethernet header 3504 of frame 3503 has CNE device RB4's MAC address (denoted as MAC-RB4) as the DA, and member switch RB1's MAC address (denoted as MAC-RB1) as the SA.
  • CNE device RB4 When frame 3503 reaches CNE device RB4, it further modifies the frame's TRILL header to produce frame 3505 .
  • CNE device RB4 replaces the destination RBridge identifier in the TRILL header 3510 with data center DC-2's root RBridge identifier DC2-ROOT.
  • the source RBridge identifier is changed to data center DC-1's virtual RBridge identifier, DC1-RB (which allows data center DC-2 to learn data center DC-1's RBridge identifier).
  • Outer Ethernet header 3508 has the core router's MAC address (MAC-RTR) as its DA, and CNE device RB4's MAC address (MAC-DC-1) as its SA.
  • Frame 3505 is subsequently transported across the IP WAN in an FCIP tunnel and reaches CNE device RB6.
  • CNE device RB6 updates the header to produce frame 3507 .
  • Frame 3507 's TRILL header 3514 remains the same as frame 3505 .
  • the outer Ethernet header 3512 now has member switch RB5's MAC address, MAC-RB5, as its DA, and CNE device RB6's MAC address, MAC-RB6, as its SA.
  • a CNE device can be configured to allow or disallow unknown unicast, broadcast (e.g., ARP), or multicast (e.g., IGMP snooped) traffic to cross data center boundaries.
  • ARP broadcast
  • multicast e.g., IGMP snooped
  • TRILL encapsulated BUM traffic between data centers can be sent with the remote data center's root RBrdige identifier. This translation is done at the terminating point of the FOP tunnel.
  • the TRILL ports between the CNE device and any VCS member switch can be configured to not participate in any of the VLAN MGIDs.
  • the eNS on both VCSs can be configured to synchronize their learned MAC address database to minimize traffic with unknown MAC DA. (Note that in one embodiment, before the learned MAC address databases are synchronized in different VCSs, frames with unknown MAC DAs are flooded within the local data center only.)
  • broadcast traffic such as ARP traffic can be reduced by snooping ARP responses to build ARP databases on VCS member switches.
  • the learned ARP databases are then exchanged and synchronized across different data centers using eNS.
  • Proxy-based ARP is used to respond to all know ARP requests in a VCS.
  • multicast traffic across data centers can be reduced by distributing the multicast group membership across data canters by sharing the IGMP snooping information via eNS.
  • the process of forwarding unicast traffic between data centers is described as follows.
  • the logical RBridge identifiers representing data centers are exchanged.
  • the source RBridge in the TRILL header is translated to the logical RBridge identifier assigned to the local data center.
  • the destination RBridge field in the TRILL header is set as the local (i.e., the destination) data center's virtual RBridge identifier.
  • the MAC DA and VLAN ID in the inner Ethernet header is then used to look up the corresponding destination RBridge (i.e., the RBridge identifier of the member switch to which the destination host is attached, and the destination RBridge field in the TRILL header is updated accordingly.
  • the VCS member switches learn the mapping between the MAC SA (in the inner Ethernet header of the frame) and the TRILL source RBridge (which is the virtual RBridge identifier assigned to the source data center). This allows future egress frames destined to that MAC address to be sent to the right remote data center. Note that since the RBridge identifier assigned to a given data center does not correspond to a physical RBridge, in one embodiment, a static route is used to map a remote data-center RBridge identifier to the local CNE device.
  • FIG. 35B presents a diagram illustrating how CNE devices handle unicast traffic across data centers, in accordance with one embodiment of the present invention.
  • host A needs to send unicast traffic to host Z, and that host A already has the knowledge of host Z's MAC address.
  • host A assembles an Ethernet frame 3502 , which has host Z's MAC address (MAC-Z) as its DA, and host A's MAC address (MAC-A) as its SA.
  • MAC-Z host Z's MAC address
  • MAC-A host A's MAC address
  • member switch RB1 Based on frame 3502 , member switch RB1 assembles a TRILL frame 3503 , whose TRILL header 3509 includes the RBridge identifier of data center DC-2's virtual Rbridge (denoted as “DC2-RB”) as the destination RBridge, and RB1 as the source RBridge.
  • the outer Ethernet header 3504 of frame 3503 has CNE device RB4's MAC address (MAC-RB4) as the DA, and member switch RB1's MAC address (MAC-RB1) as the SA.
  • CNE device RB4 When frame 3503 reaches CNE device RB4, it further modifies the frame's TRILL header to produce frame 3505 .
  • CNE device RB4 replaces the source RBridge identifier in the TRILL header 3511 with data center DC-1's virtual RBridge identifier DC1-RB (which allows data center DC-2 to learn data center DC-1's RBridge identifier).
  • Outer Ethernet header 3508 has the core router's MAC address (MAC-RTR) as its DA, and CNE device RB4's MAC address (MAC-DC-1) as its SA.
  • Frame 3505 is subsequently transported across the IP WAN in an FCIP tunnel and reaches CNE device RB6.
  • CNE device RB6 updates the header to produce frame 3507 .
  • Frame 3507 's TRILL header 3515 has an updated destination RBridge identifier, which is RB5, the VCS member switch in DC-2 that couples to host Z.
  • the outer Ethernet header 3512 now has member switch RB5's MAC address, MAC-RB5, as its DA, and CNE device RB6's MAC address, MAC-RB6, as its SA.
  • eNS can be used to distribute learned MAC address database, which reduces the amount of flooding across data centers.
  • VCSs in different data-centers do not join each other; hence the distributed configurations are kept separate.
  • a special module in either software or hardware is created for CNE purposes. This module is configured to retrieve the configuration information needed to facilitate moving of virtual machines across data centers and it is synchronized between two or more VCSs.
  • the learned MAC address databases are distributed across data centers. Also, edge port state change notifications (SCNs) are also distributed across data centers. When a physical RBridge is going down, the SCN is converted to multiple port SCNs on the inter-data-center FCIP link.
  • SCNs edge port state change notifications
  • a VCS can form a vLAG between two or more CNE devices.
  • the vLAG RBridge identifier is used as the data-center RBridge identifier.
  • the FCIP control plane is configured to be aware of this arrangement and exchange the vLAG RBridge identifers in such cases.
  • FIG. 14 illustrates an example where two CNE devices are used to construct a vLAG, in accordance with an embodiment of the present invention.
  • a VCS 1400 includes two CNE devices 1406 and 1408 . Both CNE devices 1406 and 1408 forms a vLAG 1410 which is coupled to a core IP router.
  • vLAG 1410 is assigned a virtual RBridge identifier, which is also used as the data-center RBridge identifier for VCS 1400 .
  • vLAG 1410 can facilitate both ingress and egress load balancing (e.g., based on equal-cost multi-pathing (ECMP)) for any member switch within VCS 1400 .
  • ECMP equal-cost multi-pathing
  • FIG. 37 illustrates an exemplary CNE device, in accordance with one embodiment of the present invention.
  • a CNE device 3700 includes a number of TRILL ports 3702 , FC ports 3704 , and FCIP ports 3705 .
  • TRILL ports 3702 and FC ports 3704 are used to handle traffic within the local data center.
  • FCIP ports 3705 are used to couple to WAN network equipment, such as IP routers. These ports are internally coupled to a packet processor 3706 , which is responsible for updating a received frame's header(s).
  • an address management module 3708 which is responsible for maintaining the virtual data-center RBridge identifier, learning address information from remote data centers, and updating the source RBrdige field for outgoing frames to a remote data center.
  • an FCIP management module 3710 which maintains the FCIP tunnel and is responsible for exchanging configuration information via the FCIP tunnel, and a TRILL management module 3712 , which manages the TRILL connection with the local VCS.
  • the methods and processes described herein can be embodied as code and/or data, which can be stored in a computer-readable non-transitory storage medium.
  • a computer system reads and executes the code and/or data stored on the computer-readable non-transitory storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the medium.
  • the methods and processes described herein can be executed by and/or included in hardware modules or apparatus.
  • These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed.
  • ASIC application-specific integrated circuit
  • FPGA field-programmable gate array
  • a dedicated or shared processor that executes a particular software module or a piece of code at a particular time
  • other programmable-logic devices now known or later developed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

One embodiment of the present invention provides a switch system. The switch includes one or more ports on the switch configured to transmit packets encapsulated based on a first protocol. The switch further includes a control mechanism. During operation, the control mechanism forms a logical switch based on a second protocol, receives an automatically assigned identifier for the logical switch without requiring manual configuration of the identifier, and joins a virtual cluster switch.

Description

    RELATED APPLICATIONS
  • This application is national-phase application based on PCT international application PCT/US2011/034917, which claims the benefit of U.S. Provisional Patent Applications:
  • 61/330,678, filed 3 May 2010;
  • 61/333,666, filed 11 May 2010;
  • 61/334,945, filed 14 May 2010;
  • 61/345,953, filed 18 May 2010;
  • 61/349,603, filed 28 May 2010;
  • 61/350,838, filed 2 Jun. 2010;
  • 61/352,264, filed 7 Jun. 2010;
  • 61/352,255, filed 7 Jun. 2010;
  • 61/352,819, filed 8 Jun. 2010;
  • 61/352,776, filed 8 Jun. 2010;
  • 61/380,807; filed 8 Sep. 2010;
  • 61/380,803, filed 8 Sep. 2010;
  • 61/380,819, filed 8 Sep. 2010;
  • 61/380,818, filed 8 Sep. 2010;
  • and U.S. Non-provisional Patent Applications;
  • Ser. No. 13/042,259, filed 7 Mar. 2011; Ser. No.
  • Ser. No. 13/087,239, filed 14 Apr. 2011; Ser. No.
  • Ser. No. 13/092,752, filed 22 Apr. 2011; Ser. No.
  • Ser. No. 13/092,460, filed 22 Apr. 2011; Ser. No.
  • Ser. No. 13/092,724, filed 22 Apr. 2011;
  • Ser. No. 13/092,877, filed 22 Apr. 2011;
  • Ser. No. 13/092,701, filed 22 Apr. 2011; Ser. No.
  • Ser. No. 13/092,580, filed 22 Apr. 2011; Ser. No.
  • Ser. No. 13/098,360, filed 29 Apr. 2011; and Ser. No.
  • Ser. No. 13/098,490, filed 2 May 2011;
  • the disclosures of which are incorporated by reference in entirety herein.
  • BACKGROUND
  • Field
  • The present disclosure relates to network design. More specifically, the present disclosure relates to a method for a constructing a scalable switching system that facilitates automatic configuration.
  • Related Art
  • The relentless growth of the Internet has brought with it an insatiable demand for bandwidth. As a result, equipment vendors race to build larger, faster, and more versatile switches to move traffic. However, the size of a switch cannot grow infinitely. It is limited by physical space, power consumption, and design complexity, to name a few factors. More importantly, because an overly large system often does not provide economy of scale due to its complexity, simply increasing the size and throughput of a switch may prove economically unavailable due to the increased per-port cost.
  • One way to increase the throughput of a switch system is to use switch stacking. In switch stacking, multiple smaller-scale, identical switches are interconnected in a special pattern to form a larger logical switch. However, switch stacking requires careful configuration of the ports and inter-switch links. The amount of required manual configuration becomes prohibitively complex and tedious when the stack reaches a certain size, which precludes switch stacking from being a practical option in building a large-scale switching system. Furthermore, a system based on stacked switches often has topology limitations which restrict the scalability of the system due to fabric bandwidth considerations.
  • SUMMARY
  • One embodiment of the present invention provides a switch system. The switch includes one or more ports on the switch configured to transmit packets encapsulated based on a first protocol. The switch further includes a control mechanism. During operation, the control mechanism forms a logical switch based on a second protocol, receives an automatically assigned identifier for the logical switch without requiring manual configuration of the identifier, and joins a virtual cluster switch.
  • In a variation on this embodiment, the virtual cluster switch comprises one or more physical switches which are allowed to be coupled in an arbitrary topology. Furthermore, the virtual cluster switch appears to be one single switch.
  • In a further variation, the first protocol is a Transparent Interconnection of Lots of Links (TRILL) protocol, and the packets are encapsulated in TRILL headers.
  • In a variation on this embodiment, the logical switch formed by the control mechanism is a logical Fibre Channel (FC) switch.
  • In a further variation, the identifier assigned to the logical switch is an FC switch domain ID.
  • In a variation on this embodiment, the control mechanism is further configured to maintain a copy of configuration information for the virtual cluster switch.
  • In a further variation on this embodiment, the configuration information for the virtual cluster switch comprises a number of logical switch identifiers assigned to the physical switches in the virtual cluster switch.
  • In a variation on this embodiment, the switch includes a media access control (MAC) learning mechanism which is configured to learn a source MAC address and a corresponding VLAN identifier of an ingress packet associated with a port and communicate a learned MAC address, a corresponding VLAN identifier, and the corresponding port information to a name service.
  • One embodiment of the present invention provides a switching system that includes a plurality of switches configured to transport packets using a first protocol. Each switch includes a control mechanism. The plurality switches are allowed to be coupled in an arbitrary topology. Furthermore, the control mechanism automatically configures the respective switch within the switching system based on a second protocol without requiring manual configuration, and the switching system appears externally as a single switch.
  • In a variation on this embodiment, a respective switch in the switching system receives an automatically configured identifier associated with a logical switch formed on the respective switch.
  • In a further variation, the logical switch is a logical FC switch. In addition, the identifier is an FC switch domain ID.
  • In a further variation, the packets are transported between switches based on a TRILL protocol. The respective switch is assigned a TRILL RBridge identifier that corresponds to the FC switch domain ID.
  • In a variation on this embodiment, a respective switch maintains a copy of configuration information of all the switches in the switching system.
  • In a variation on this embodiment, the switching system includes a name service which maintains records of MAC addresses and VLAN information learned by a respective switch.
  • BRIEF DESCRIPTION OF THE FIGURES
  • FIG. 1A illustrates an exemplary virtual cluster switch (VCS) system, in accordance with an embodiment of the present invention.
  • FIG. 1B illustrates an exemplary VCS system where the member switches are configured in a CLOS network, in accordance with an embodiment of the present invention.
  • FIG. 2 illustrates the protocol stack within a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 3 illustrates an exemplary configuration of a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 4 illustrates an exemplary configuration of how a virtual cluster switch can be connected to different edge networks, in accordance with an embodiment of the present invention.
  • FIG. 5A illustrates how a logical Fibre Channel switch fabric is formed in a virtual cluster switch in conjunction with the example in FIG. 4, in accordance with an embodiment of the present invention.
  • FIG. 5B illustrates an example of how a logical FC switch can be created within a physical Ethernet switch, in accordance with one embodiment of the present invention.
  • FIG. 6 illustrates an exemplary VCS configuration database, in accordance with an embodiment of the present invention.
  • FIG. 7A illustrates an exemplary process of a switch joining a virtual cluster switch, in accordance with an embodiment of the present invention.
  • FIG. 7B presents a flowchart illustrating the process of a default switch joining a VCS, in accordance with one embodiment of the present invention.
  • FIG. 7C presents a flowchart illustrating the process of a switch re-joining a previously joined VCS, in accordance with one embodiment of the present invention.
  • FIG. 7D presents a flowchart illustrating the general operation of a switch during a join process, in accordance with one embodiment of the present invention.
  • FIG. 8 presents a flowchart illustrating the process of looking up an ingress frame's destination MAC address and forwarding the frame in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 9 illustrates how data frames and control frames are transported through a VCS, in accordance with one embodiment of the present invention.
  • FIG. 10 illustrates a logical VCS access layer (VAL) which includes an automatic port profile manager, in accordance with one embodiment of the present invention.
  • FIG. 11 illustrates an example of the operation of automatic migration of port profiles (AMPP), in accordance with one embodiment of the present invention
  • FIG. 12A illustrates exemplary port profile contents, in accordance with one embodiment of the present invention.
  • FIG. 12B illustrates three exemplary port profiles, in accordance with on embodiment of the present invention.
  • FIGS. 13A and 13B illustrates how forwarding is achieved between VMs based on port profile membership, in accordance with one embodiment of the present invention.
  • FIG. 14 presents a flowchart illustrating the process of creating and applying a port profile, in accordance with one embodiment of the present invention.
  • FIG. 15 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • FIG. 16 illustrates an exemplary configuration of advanced link tracking in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 17 illustrates an example where advanced link tracking allows virtual machines to re-route egress traffic when a link fails, in accordance with one embodiment of the present invention.
  • FIG. 18 presents a flowchart illustrating the process of advance link tracking, in accordance with one embodiment of the present invention.
  • FIG. 19 illustrates an exemplary switch that facilitates virtual cluster switching and advanced link tracking, in accordance with one embodiment of the present invention.
  • FIG. 20 illustrates an exemplary VCS network environment with virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 21A illustrates an exemplary data structure for storing virtual port grouping information, in accordance with one embodiment of the present invention.
  • FIG. 21B illustrates logical virtual networks within a VCS as a result of virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 22 presents a flowchart illustrating the process of enforcing virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 23 illustrates an exemplary switch that facilitates virtual cluster switching with virtual port grouping, in accordance with one embodiment of the present invention.
  • FIG. 24 illustrates an example of name service operation in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 25 presents a flowchart illustrating the process of distributing learned MAC information by the Ethernet name service in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 26 presents a flowchart illustrating the process of distributing information of a learned MAC address via an MCT, in accordance with one embodiment of the present invention.
  • FIG. 27 presents a flowchart illustrating the process of updating the link state in an MCT group, in accordance with one embodiment of the present invention.
  • FIG. 28 illustrates an exemplary switch that facilitates formation of a virtual cluster switch with Ethernet and MCT name services, in accordance with an embodiment of the present invention.
  • FIG. 29 illustrates an exemplary equal cost multi-pathing configuration in a VCS, in accordance with one embodiment of the present invention.
  • FIG. 30 illustrates an example of VCS congestion notification, in accordance with one embodiment of the present invention.
  • FIG. 31 illustrates an exemplary hybrid trunk in accordance with one embodiment of the present invention.
  • FIG. 32 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention.
  • FIG. 33 illustrates an exemplary network architecture which includes CNE devices for facilitate cross-data-center communications, in accordance with one embodiment of the present invention.
  • FIG. 34 illustrates an exemplary implementation of CNE-enabled VCSs, in accordance with one embodiment of the present invention.
  • FIG. 35A presents a diagram illustrating how CNE devices handle broadcast, unknown unicast, and multicast (BUM) traffic across data centers, in accordance with one embodiment of the present invention.
  • FIG. 35B presents a diagram illustrating how CNE devices handle unicast traffic across data centers, in accordance with one embodiment of the present invention.
  • FIG. 36 illustrates an example where two CNE devices are used to construct a vLAG, in accordance with an embodiment of the present invention.
  • FIG. 37 illustrates an exemplary CNE device, in accordance with one embodiment of the present invention.
  • DETAILED DESCRIPTION
  • The following description is presented to enable any person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the claims.
  • Overview
  • In embodiments of the present invention, the problem of building a versatile, cost-effective, and scalable switching system is solved by running a control plane with automatic configuration capabilities (such as the Fibre Channel control plane) over a conventional transport protocol, thereby allowing a number of switches to be inter-connected to form a single, scalable logical switch without requiring burdensome manual configuration. As a result, one can form a large-scale logical switch (referred to as a “virtual cluster switch” or VCS herein) using a number of smaller physical switches. The automatic configuration capability provided by the control plane running on each physical switch allows any number of switches to be connected in an arbitrary topology without requiring tedious manual configuration of the ports and links. This feature makes it possible to use many smaller, inexpensive switches to construct a large cluster switch, which can be viewed as a single logical switch externally.
  • It should be noted that a virtual cluster switch is not the same as conventional switch stacking. In switch stacking, multiple switches are interconnected at a common location (often within the same rack), based on a particular topology, and manually configured in a particular way. These stacked switches typically share a common address, e.g., IP address, so they can be addressed as a single switch externally. Furthermore, switch stacking requires a significant amount of manual configuration of the ports and inter-switch links. The need for manual configuration prohibits switch stacking from being a viable option in building a large-scale switching system. The topology restriction imposed by switch stacking also limits the number of switches that can be stacked. This is because it is very difficult, if not impossible, to design a stack topology that allows the overall switch bandwidth to scale adequately with the number of switch units.
  • In contrast, a VCS can include an arbitrary number of switches with individual addresses, can be based on an arbitrary topology, and does not require extensive manual configuration. The switches can reside in the same location, or be distributed over different locations. These features overcome the inherent limitations of switch stacking and make it possible to build a large “switch farm” which can be treated as a single, logical switch. Due to the automatic configuration capabilities of the VCS, an individual physical switch can dynamically join or leave the VCS without disrupting services to the rest of the network.
  • Furthermore, the automatic and dynamic configurability of VCS allows a network operator to build its switching system in a distributed and “pay-as-you-grow” fashion without sacrificing scalability. The VCS's ability to respond to changing network conditions makes it an ideal solution in a virtual computing environment, where network loads often change with time.
  • Although this disclosure is presented using examples based on the Transparent Interconnection of Lots of Links (TRILL) as the transport protocol and the Fibre Channel (FC) fabric protocol as the control-plane protocol, embodiments of the present invention are not limited to TRILL networks, or networks defined in a particular Open System Interconnection Reference Model (OSI reference model) layer. For example, a VCS can also be implemented with switches running multi-protocol label switching (MPLS) protocols for the transport. In addition, the terms “RBridge” and “switch” are used interchangeably in this disclosure. The use of the term “RBridge” does not limit embodiments of the present invention to TRILL networks only. The TRILL protocol is described in IETF draft “RBridges: Base Protocol Specification,” available at http://tools.ietf.org/html/draft-ietf-trill-rbridge-protocol, which is incorporated by reference herein
  • The terms “virtual cluster switch,” “virtual cluster switching,” and “VCS” refer to a group of interconnected physical switches operating as a single logical switch. The control plane for these physical switches provides the ability to automatically configure a given physical switch, so that when it joins the VCS, little or no manual configuration is required. VCS is not limited to a specific product family from a particular vendor. In addition, “VCS” is not the only term that can be used to name the switching system described herein. Other terms, such as “Ethernet fabric,” “Ethernet fabric switch,” “fabric switch,” “cluster switch,” “Ethernet mesh switch,” and “mesh switch” can also be used to describe the same switching system. Hence, in some embodiments, these terms and “VCS” can be used interchangeably.
  • The term “RBridge” refers to routing bridges, which are bridges implementing the TRILL protocol as described in IETF draft “RBridges: Base Protocol Specification.” Embodiments of the present invention are not limited to the application among RBridges. Other types of switches, routers, and forwarders can also be used.
  • The terms “frame” or “packet” refer to a group of bits that can be transported together across a network. “Frame” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. “Packet” should not be interpreted as limiting embodiments of the present invention to layer-3 networks. “Frame” or “packet” can be replaced by other terminologies referring to a group of bits, such as “cell” or “datagram.”
  • VCS Architecture
  • FIG. 1A illustrates an exemplary virtual cluster switch system, in accordance with an embodiment of the present invention. In this example, a VCS 100 includes physical switches 101, 102, 103, 104, 105, 106, and 107. A given physical switch runs an Ethernet-based transport protocol on its ports (e.g., TRILL on its inter-switch ports, and Ethernet transport on its external ports), while its control plane runs an FC switch fabric protocol stack. The TRILL protocol facilitates transport of Ethernet frames within and across VCS 100 in a routed fashion (since TRILL provides routing functions to Ethernet frames). The FC switch fabric protocol stack facilitates the automatic configuration of individual physical switches, in a way similar to how a conventional FC switch fabric is formed and automatically configured. In one embodiment, VCS 100 can appear externally as an ultra-high-capacity Ethernet switch. More details on FC network architecture, protocols, naming/address conventions, and various standards are available in the documentation available from the NCITS/ANSI T11 committee (www.t11.org) and publicly available literature, such as “Designing Storage Area Networks,” by Tom Clark, 2nd Ed., Addison Wesley, 2003, the disclosures of which are incorporated by reference in their entirety herein.
  • A physical switch may dedicate a number of ports for external use (i.e., to be coupled to end hosts or other switches external to the VCS) and other ports for inter-switch connection. Viewed externally, VCS 100 appears to be one switch to a device from the outside, and any port from any of the physical switches is considered one port on the VCS. For example, port groups 110 and 112 are both VCS external ports and can be treated equally as if they were ports on a common physical switch, although switches 105 and 107 may reside in two different locations.
  • The physical switches can reside at a common location, such as a data center or central office, or be distributed in different locations. Hence, it is possible to construct a large-scale centralized switching system using many smaller, inexpensive switches housed in one or more chassis at the same location. It is also possible to have the physical switches placed at different locations, thus creating a logical switch that can be accessed from multiple locations. The topology used to interconnect the physical switches can also be versatile. VCS 100 is based on a mesh topology. In further embodiments, a VCS can be based on a ring, tree, or other types of topologies.
  • In one embodiment, the protocol architecture of a VCS is based on elements from the standard IEEE 802.1Q Ethernet bridge, which is emulated over a transport based on the Fibre Channel Framing and Signaling-2 (FC-FS-2) standard. The resulting switch is capable of transparently switching frames from an ingress Ethernet port from one of the edge switches to an egress Ethernet port on a different edge switch through the VCS.
  • Because of its automatic configuration capability, a VCS can be dynamically expanded as the network demand increases. In addition, one can build a large-scale switch using many smaller physical switches without the burden of manual configuration. For example, it is possible to build a high-throughput fully non-blocking switch using a number of smaller switches. This ability to use small switches to build a large non-blocking switch significantly reduces the cost associated switch complexity. FIG. 1B presents an exemplary VCS with its member switches connected in a CLOS network, in accordance with one embodiment of the present invention. In this example, a VCS 120 forms a fully non-blocking 8×8 switch, using eight 4×4 switches and four 2×2 switches connected in a three-stage CLOS network. A large-scale switch with a higher port count can be built in a similar way.
  • FIG. 2 illustrates the protocol stack within a virtual cluster switch, in accordance with an embodiment of the present invention. In this example, two physical switches 202 and 204 are illustrated within a VCS 200. Switch 202 includes an ingress Ethernet port 206 and an inter-switch port 208. Switch 204 includes an egress Ethernet port 212 and an inter-switch port 210. Ingress Ethernet port 206 receives Ethernet frames from an external device. The Ethernet header is processed by a medium access control (MAC) layer protocol. On top of the MAC layer is a MAC client layer, which hands off the information extracted from the frame's Ethernet header to a forwarding database (FDB) 214. Typically, in a conventional IEEE 802.1Q Ethernet switch, FDB 214 is maintained locally in a switch, which would perform a lookup based on the destination MAC address and the VLAN indicated in the Ethernet frame. The lookup result would provide the corresponding output port. However, since VCS 200 is not one single physical switch, FDB 214 would return the egress switch's identifier (i.e., switch 204's identifier). In one embodiment, FDB 214 is a data structure replicated and distributed among all the physical switches. That is, every physical switch maintains its own copy of FDB 214. When a given physical switch learns the source MAC address and VLAN of an Ethernet frame (similar to what a conventional IEEE 802.1Q Ethernet switch does) as being reachable via the ingress port, the learned MAC and VLAN information, together with the ingress Ethernet port and switch information, is propagated to all the physical switches so every physical switch's copy of FDB 214 can remain synchronized. This prevents forwarding based on stale or incorrect information when there are changes to the connectivity of end stations or edge networks to the VCS.
  • The forwarding of the Ethernet frame between ingress switch 202 and egress switch 204 is performed via inter-switch ports 208 and 210. The frame transported between the two inter-switch ports is encapsulated in an outer MAC header and a TRILL header, in accordance with the TRILL standard. The protocol stack associated with a given inter-switch port includes the following (from bottom up): MAC layer, TRILL layer, FC-FS-2 layer, FC E-Port layer, and FC link services (FC-LS) layer. The FC-LS layer is responsible for maintaining the connectivity information of a physical switch's neighbor, and populating an FC routing information base (RIB) 222. This operation is similar to what is done in an FC switch fabric. The FC-LS protocol is also responsible for handling joining and departure of a physical switch in VCS 200. The operation of the FC-LS layer is specified in the FC-LS standard, which is available at http://www.t11.org/ftp/t11/member/fc/ls/06-393v5.pdf, the disclosure of which is incorporated herein in its entirety.
  • During operation, when FDB 214 returns the egress switch 204 corresponding to the destination MAC address of the ingress Ethernet frame, the destination egress switch's identifier is passed to a path selector 218. Path selector 218 performs a fabric shortest-path first (FSPF)-based route lookup in conjunction with RIB 222, and identifies the next-hop switch within VCS 200. In other words, the routing is performed by the FC portion of the protocol stack, similar to what is done in an FC switch fabric.
  • Also included in each physical switch are an address manager 216 and a fabric controller 220. Address manager 216 is responsible for configuring the address of a physical switch when the switch first joins the VCS. For example, when switch 202 first joins VCS 200, address manager 216 can negotiate a new FC switch domain ID, which is subsequently used to identify the switch within VCS 200. Fabric controller 220 is responsible for managing and configuring the logical FC switch fabric formed on the control plane of VCS 200.
  • One way to understand the protocol architecture of VCS is to view the VCS as an FC switch fabric with an Ethernet/TRILL transport. Each physical switch, from an external point of view, appears to be a TRILL RBridge. However, the switch's control plane implements the FC switch fabric software. In other words, embodiments of the present invention facilitate the construction of an “Ethernet switch fabric” running on FC control software. This unique combination provides the VCS with automatic configuration capability and allows it to provide the ubiquitous Ethernet services in a very scalable fashion.
  • FIG. 3 illustrates an exemplary configuration of a virtual cluster switch, in accordance with an embodiment of the present invention. In this example, a VCS 300 includes four physical switches 302, 304, 306, and 308. VCS 300 constitutes an access layer which is coupled to two aggregation switches 310 and 312. Note that the physical switches within VCS 300 are connected in a ring topology. Aggregation switch 310 or 312 can connect to any of the physical switches within VCS 300. For example, aggregation switch 310 is coupled to physical switches 302 and 308. These two links are viewed as a trunked link to VCS 300, since the corresponding ports on switches 302 and 308 are considered to be from the same logical switch, VCS 300. Note that, without VCS, such topology would not have been possible, because the FDB needs to remain synchronized, which is facilitated by the VCS.
  • FIG. 4 illustrates an exemplary configuration of how a virtual cluster switch can be connected to different edge networks, in accordance with an embodiment of the present invention. In this example, a VCS 400 includes a number of TRILL RBridges 402, 404, 406, 408, and 410, which are controlled by the FC switch-fabric control plane. Also included in VCS 400 are RBridges 412, 414, and 416. Each RBridge has a number of edge ports which can be connected to external edge networks.
  • For example, RBridge 412 is coupled with hosts 420 and 422 via 10GE ports. RBridge 414 is coupled to a host 426 via a 10GE port. These RBridges have TRILL-based inter-switch ports for connection with other TRILL RBridges in VCS 400. Similarly, RBridge 416 is coupled to host 428 and an external Ethernet switch 430, which is coupled to an external network that includes a host 424. In addition, network equipment can also be coupled directly to any of the physical switches in VCS 400. As illustrated here, TRILL RBridge 408 is coupled to a data storage 417, and TRILL RBridge 410 is coupled to a data storage 418.
  • Although the physical switches within VCS 400 are labeled as “TRILL RBridges,” they are different from the conventional TRILL RBridge in the sense that they are controlled by the FC switch fabric control plane. In other words, the assignment of switch addresses, link discovery and maintenance, topology convergence, routing, and forwarding can be handled by the corresponding FC protocols. Particularly, each TRILL RBridge's switch ID or nickname is mapped from the corresponding FC switch domain ID, which can be automatically assigned when a switch joins VCS 400 (which is logically similar to an FC switch fabric).
  • Note that TRILL is only used as a transport between the switches within VCS 400. This is because TRILL can readily accommodate native Ethernet frames. Also, the TRILL standards provide a ready-to-use forwarding mechanism that can be used in any routed network with arbitrary topology (although the actual routing in VCS is done by the FC switch fabric protocols). Embodiments of the present invention should be not limited to using only TRILL as the transport. Other protocols (such as multi-protocol label switching (MPLS) or Internet Protocol (IP)), either public or proprietary, can also be used for the transport.
  • VCS Formation
  • In one embodiment, a VCS is created by instantiating a logical FC switch in the control plane of each switch. After the logical FC switch is created, a virtual generic port (denoted as G_Port) is created for each Ethernet port on the RBridge. A G_Port assumes the normal G_Port behavior from the FC switch perspective. However, in this case, since the physical links are based on Ethernet, the specific transition from a G_Port to either an FC F_Port or E_Port is determined by the underlying link and physical layer protocols. For example, if the physical Ethernet port is connected to an external device which lacks VCS capabilities, the corresponding G_Port will be turned into an F_Port. On the other hand, if the physical Ethernet port is connected to a switch with VCS capabilities and it is confirmed that the switch on the other side is part of a VCS, then the G_Port will be turned into an E_port.
  • FIG. 5A illustrates how a logical Fibre Channel switch fabric is formed in a virtual cluster switch in conjunction with the example in FIG. 4, in accordance with an embodiment of the present invention. RBridge 412 contains a virtual, logical FC switch 502. Corresponding to the physical Ethernet ports coupled to hosts 420 and 422, logical FC switch 502 has two logical F_Ports, which are logically coupled to hosts 420 and 422. In addition, two logical N_Ports, 506 and 504, are created for hosts 420 and 422, respectively. On the VCS side, logical FC switch 502 has three logical E_Ports, which are to be coupled with other logical FC switches in the logical FC switch fabric in the VCS.
  • Similarly, RBridge 416 contains a virtual, logical FC switch 512. Corresponding to the physical Ethernet ports coupled to host 428 and external switch 430, logical FC switch 512 has a logical F_Port coupled to host 428, and a logical FL_Port coupled to switch 430. In addition, a logical N_Port 510 is created for host 428, and a logical NL_Port 508 is created for switch 430. Note that the logical FL_Port is created because that port is coupled to a switch (switch 430), instead of a regular host, and therefore logical FC switch 512 assumes an arbitrated loop topology leading to switch 430. Logical NL_Port 508 is created based on the same reasoning to represent a corresponding NL_Port on switch 430. On the VCS side, logical FC switch 512 has two logical E_Ports, which to be coupled with other logical FC switches in the logical FC switch fabric in the VCS.
  • FIG. 5B illustrates an example of how a logical FC switch can be created within a physical Ethernet switch, in accordance with one embodiment of the present invention. The term “fabric port” refers to a port used to couple multiple switches in a VCS. The clustering protocols control the forwarding between fabric ports. The term “edge port” refers to a port that is not currently coupled to another switch unit in the VCS. Standard IEEE 802.1Q and layer-3 protocols control forwarding on edge ports.
  • In the example illustrated in FIG. 5B, a logical FC switch 521 is created within a physical switch (RBridge) 520. Logical FC switch 521 participates in the FC switch fabric protocol via logical inter-switch links (ISLs) to other switch units and has an FC switch domain ID assigned to it just as a physical FC switch does. In other words, the domain allocation, principal switch selection, and conflict resolution work just as they would on a physical FC ISL.
  • The physical edge ports 522 and 524 are mapped to logical F_Ports 532 and 534, respectively. In addition, physical fabric ports 526 and 528 are mapped to logical E_Ports 536 and 538, respectively. Initially, when logical FC switch 521 is created (for example, during the boot-up sequence), logical FC switch 521 only has four G_Ports which correspond to the four physical ports. These G_Ports are subsequently mapped to F_Ports or E_Ports, depending on the devices coupled to the physical ports.
  • Neighbor discovery is the first step in VCS formation between two VCS-capable switches. It is assumed that the verification of VCS capability can be carried out by a handshake process between two neighbor switches when the link is first brought up.
  • Distributed Configuration Management
  • In general, a VCS presents itself as one unified switch composed of multiple member switches. Hence, the creation and configuration of VCS is of critical importance. The VCS configuration is based on a distributed database, which is replicated and distributed over all switches. In other words, each VCS member switch maintains a copy of the VCS configuration database, and any change to the database is propagated to all the member switches. As a result, a network administrator can configure any part of the VCS from anywhere by, for example, executing a command line interface (CLI) command from any member switch.
  • In one embodiment, a VCS configuration database includes a global configuration table (GT) of the VCS and a list of switch description tables (STs), each of which describes a VCS member switch. In its simplest form, a member switch can have a VCS configuration database that includes a global table and one switch description table, e.g., [<GT><ST>]. A VCS with multiple switches will have a configuration database that has a single global table and multiple switch description tables, e.g., [<GT><ST0><ST1> . . . <STn−1>]. The number n corresponds to the number of member switches in the VCS. In one embodiment, the GT can include at least the following information: the VCS ID, number of nodes in the VCS, a list of VLANs supported by the VCS, a list of all the switches (e.g., list of FC switch domain IDs for all active switches) in the VCS, and the FC switch domain ID of the principal switch (as in a logical FC switch fabric). A switch description table can include at least the following information: the IN_VCS flag, indication whether the switch is a principal switch in the logical FC switch fabric, the FC switch domain ID for the switch, the FC world-wide name (WWN) for the corresponding logical FC switch; the mapped ID of the switch, and optionally the IP address of the switch. Note that, as explained below, a switch can have an IP address dynamically assigned to it when it joins the VCS. The IP address can be derived from and have a one-to-one mapping to the FC switch domain ID assigned to the switch.
  • In addition, each switch's global configuration database is associated with a transaction ID. The transaction ID specifies the latest transaction (e.g., update or change) incurred to the global configuration database. The transaction IDs of the global configuration databases in two switches can be compared to determine which database has the most current information (i.e., the database with the more current transaction ID is more up-to-date). In one embodiment, the transaction ID is the switch's serial number plus a sequential transaction number. This configuration can unambiguously resolve which switch has the latest configuration.
  • As illustrated in FIG. 6, a VCS member switch typically maintains two configuration tables that describe its instance: a VCS configuration database 600, and a default switch configuration table 604. VCS configuration database 600 describes the VCS configuration when the switch is part of a VCS. Default switch configuration table 604 describes the switch's default configuration. VCS configuration database 600 includes a GT 602, which includes a VCS identifier (denoted as VCS_ID) and a VLAN list within the VCS. Also included in VCS configuration database 600 are a number of STs, such as ST0, ST1, and STn. Each ST includes the corresponding member switch's MAC address and FC switch domain ID, as well as the switch's interface details. Note that each switch also has a VCS-mapped ID which is a switch index within the VCS.
  • In one embodiment, each switch also has a VCS-mapped ID (denoted as “mappedID”), which is a switch index within the VCS. This mapped ID is unique and persistent within the VCS. That is, when a switch joins the VCS for the first time, the VCS assigns a mapped ID to the switch. This mapped ID persists with the switch, even if the switch leaves the VCS. When the switch joins the VCS again at a later time, the same mapped ID is used by the VCS to retrieve previous configuration information for the switch. This feature can reduce the amount of configuration overhead in VCS. Also, the persistent mapped ID allows the VCS to “recognize” a previously configured member switch when it re-joins the VCS, since a dynamically assigned FC fabric domain ID would change each time the member switch joins and is configured by the VCS.
  • Default switch configuration table 604 has an entry for the mappedID that points to the corresponding ST in VCS configuration database 600. Note that only VCS configuration database 600 is replicated and distributed to all switches in the VCS. Default switch configuration table 604 is local to a particular member switch.
  • The “IN_VCS” value in default switch configuration table 604 indicates whether the member switch is part of a VCS. A switch is considered to be “in a VCS” when it is assigned one of the FC switch domains by the FC switch fabric with two or more switch domains. If a switch is part of an FC switch fabric that has only one switch domain, i.e., its own switch domain, then the switch is considered to be “not in a VCS.”
  • When a switch is first connected to a VCS, the logical FC switch fabric formation process allocates a new switch domain ID to the joining switch. In one embodiment, only the switches directly connected to the new switch participate in the VCS join operation.
  • Note that in the case where the global configuration database of a joining switch is current and in sync with the global configuration database of the VCS based on a comparison of the transaction IDs of the two databases (e.g., when a member switch is temporarily disconnected from the VCS and re-connected shortly afterward), a trivial merge is performed. That is, the joining switch can be connected to the VCS, and no change or update to the global VCS configuration database is required.
  • When a switch joins a VCS (see the description below), the VCS assigns an FC switch domain ID to the logical FC switch formed within the joining switch. The joining switch can be automatically assigned an IP address (which in one embodiment can be internal to the VCS) corresponding to the FC switch domain ID. For example, the joining switch's IP address can be 127.10.10.FC_domain_ID. In addition, each port on the joining switch can be automatically assigned a MAC address which is unique to that port, on that switch. For instance, a port on the joining switch can be assigned a MAC address of OUI:FC_domain_ID:0:0, where OUI is the Organizationally Unique Identifier for the port, and FC_domain_ID is the switch domain ID assigned to the logical FC switch. The IP and MAC address assignment process can be part of the software within the switch which controls the FC switch formation and the join/merge process.
  • The automatically assigned switch IP address and port MAC address for a given VCS member switch allow any port on the switch to be remotely configured from anywhere within the VCS. For example, a configuration command of a given member switch can be issued from a host connected to any member switch in the VCS and identifies the switch to be configured by its IP address and optionally one or more port MAC addresses. Such configuration command might include information on VLAN configuration, quality-of-service (QoS) configuration, and/or access-control configuration, each of which can be port-specific. In one embodiment, the change to a switch's configuration is tentatively transmitted to the switch. After the switch confirms and validates the change, a commit-change command is transmitted to all the member switches in the VCS, so the global configuration database can be updated throughout the VCS. In a further embodiment, the change is tentatively transmitted to all the member switches in the VCS, and the commit-change command is only sent out after all the switches confirm and validate the tentatively change.
  • VCS Join and Merge
  • FIG. 7A illustrates an exemplary process of a switch joining a virtual cluster switch, in accordance with an embodiment of the present invention. In this example, it is assumed that a switch 702 is within an existing VCS, and a switch 704 is joining the VCS. During operation, both switches 702 and 704 trigger an FC State Change Notification (SCN) process. Subsequently, both switches 702 and 704 perform a PRE-INVITE operation. The pre-invite operation involves the following process.
  • When a switch joins the VCS via a link, both neighbors on each end of the link present to the other switch a VCS four-tuple of <Prior VCS_ID, SWITCH_MAC, mappedID, IN_VCS> from a prior incarnation, if any. Otherwise, the switch presents to the counterpart a default tuple. If the VCS_ID value was not set from a prior join operation, a VCS_ID value of −1 is used. In addition, if a switch's IN_VCS flag is set to 0, it sends out its interface configuration to the neighboring switch. In the example in FIG. 7, both switches 702 and 704 send the above information to the other switch.
  • After the above PRE-INVITE operation, a driver switch for the join process is selected. By default, if a switch's IN_VCS value is 1 and the other switch's IN_VCS value is 0, the switch with IN_VCS=1 is selected as the driver switch. If both switches have their IN_VCS values as 1, then nothing happens, i.e., the PRE-INVITE operation would not lead to an INVITE operation. If both switches have their IN_VCS values as 0, then one of the switches is elected to be the driving switch (for example, the switch with a lower FC switch domain ID value). The driving switch's IN_VCS value is then set to 1 and drives the join process.
  • After switch 702 is selected as the driver switch, switch 702 then attempts to reserve a slot in the VCS configuration database corresponding to the mappedID value in switch 704's PRE-INVITE information. Next, switch 702 searches the VCS configuration database for switch 704's MAC address in any mappedID slot. If such a slot is found, switch 702 copies all information from the identified slot into the reserved slot. Otherwise, switch 702 copies the information received during the PRE-INVITE from switch 704 into the VCS configuration database. The updated VCS configuration database is then propagated to all the switches in the VCS as a prepare operation in the database (note that the update is not committed to the database yet).
  • Subsequently, the prepare operation may or may not result in configuration conflicts, which may be flagged as warnings or fatal errors. Such conflicts can include inconsistencies between the joining switch's local configuration or policy setting and the VCS configuration. For example, a conflict arises when the joining switch is manually configured to allow packets with a particular VLAN value to pass through, whereas the VCS does not allow this VLAN value to enter the switch fabric from this particular RBridge (for example, when this VLAN value is reserved for other purposes). In one embodiment, the prepare operation is handled locally and/or remotely in concert with other VCS member switches. If there is an un-resolvable conflict, switch 702 sends out a PRE-INVITE-FAILED message to switch 704. Otherwise, switch 702 generates an INVITE message with the VCS's merged view of the switch (i.e., the updated VCS configuration database).
  • Upon receiving the INVITE message, switch 704 either accepts or rejects the INVITE. The INVITE can be rejected if the configuration in the INVITE is in conflict with what switch 704 can accept. If the INVITE is acceptable, switch 704 sends back an INVITE-ACCEPT message in response. The INVITE-ACCEPT message then triggers a final database commit throughout all member switches in the VCS. In other words, the updated VCS configuration database is updated, replicated, and distributed to all the switches in the VCS.
  • If more than one switch in a VCS has connectivity to the new joining switch, all these neighboring member switches may send PRE-INVITE to the new joining switch. The joining switch can send out only one PRE-INVITE to a randomly selected neighboring member switch to complete the join process. Various use cases of the join process are described below. In the following description, a “joining switch” refers to a switch attempting to join a VCS. A “neighboring VCS member switch” or “neighboring member switch” refers to a VCS member switch to which the joining switch is connected.
  • VCS Pre-Provisioned to Accept a Switch.
  • A VCS is pre-configured (e.g., the global configuration database) with the MAC address of a joining switch with an optionally pre-allocated mapped ID for the joining switch. The joining switch may be allowed to carry any value in the VCS_ID field of its existing configuration. The neighboring VCS member switch can assign an FC switch domain ID and the proper VCS_ID to the joining switch in the INVITE message. In one embodiment, the joining switch may be pre-provisioned to join an existing VCS (e.g., with the parameters in the default switch configuration table, such as mappedID, VCS_ID, and IN_VCS, populated with values corresponding to the VCS). If the pre-provisioned parameters do not guarantee a slot with the same mappedID in the global configuration database when the switch joins the VCS, the switch can revert to the default joining procedure described below.
  • Default Switch Joins a VCS.
  • A default switch is one that has no records of any previous joining with a VCS. A switch can become a default switch if it is forced into a factory default state. A joining default switch can present its initial configuration information (for example, its interface configuration details) to a neighboring VCS member switch. In one embodiment, a slot in the VCS configuration database is selected based on a monotonically incrementing number, which is used as the mapped ID for the joining switch. The corresponding FC switch domain ID which is allocated to the joining switch and the joining switch's MAC is updated accordingly in this slot. The neighboring VCS member switch then initiates a prepare transaction, which propagates to all VCS member switches and requires an explicit validation of the joining switch's configuration information from each VCS member switch. If the prepare transaction fails, a PRE-INVITE-FAILED message is sent to the joining switch and the joining process is aborted.
  • FIG. 7B presents a flowchart illustrating the process of a default switch joining a VCS, in accordance with one embodiment of the present invention. During operation, after a joining switch with default configuration is connected to a neighboring VCS member switch, the VCS member switch reserves a slot in the global configuration database (operation 722). The reserved slot corresponds to a mappedID value which is also assigned to the joining switch. Subsequently, the logical FC switch within the neighboring VCS member switch allocates an FC switch domain ID for the joining switch (operation 724). This operation resembles a similar fabric login operation in the FC switch fabric join process.
  • The neighboring VCS member switch then tentatively updates the reserved slot in the global configuration database with the allocated FC switch domain ID and the joining switch's MAC address (operation 726). Next, the neighboring VCS member switch transmits the joining switch's tentative configuration to all member switches in the VCS (operation 728), and determines whether the joining switch's configuration information is confirmed and validated by all VCS member switches (operation 730). If the joining switch's configuration is confirmed, the neighboring member switch then commits the changes to the global configuration database and completes the join process (operation 732). Otherwise, the join process is aborted and the tentative changes to the global configuration database are discarded (operation 734).
  • Switch Re-Joins a Previously Joined VCS.
  • If for some reason a switch is joining a VCS to which the switch previously belongs (for example, due to a link failure), the FC switch domain ID that is re-allocated to the joining switch will most likely be the same. When such a switch joins the VCS, the neighboring VCS member switch first checks whether the joining switch's VCS_ID is the same as the existing VCS_ID on the member switch. If the two VCS_ID values are the same, the neighboring member switch attempts to find a slot in the global configuration database with the same mappedID value which was received from the joining switching during the tuple-exchange process. If such a slot in the global database is available, the slot is reserved for the joining switch. In addition, the global configuration database is searched for a match to the joining switch's MAC address. If a match is found in another slot, the configuration information from that slot is copied to the reserved slot. Subsequently, the join process continues as described in FIG. 7A. If the two VCS_ID values are different, the system performs a join process as described blow for the use case “switch joins another VCS.”
  • FIG. 7C presents a flowchart illustrating the process of a switch re-joining a previously joined VCS, in accordance with one embodiment of the present invention. During operation, a neighboring member switch first determines whether the joining switch's existing VCS_ID is the same as the neighboring member switch's VCS_ID (operation 740). If the two VCS_IDs are different, the joining process proceeds to the operations illustrated in FIG. 7D (operation 741). If the two VCS_IDs are the same, which means that the joining switch has previously been part of the same VCS, the neighboring member switch further determines whether there is a slot in the global configuration database with the same mappedID as the joining switch (operation 742). If so, the neighboring member switch reserves a slot in the global configuration database with the same mappedID (operation 744). Otherwise, the neighboring member switch reserves a slot in the global configuration database with a new mappedID, which is also communicated to the joining switch (operation 746).
  • Subsequently, the neighboring member switch determines whether the global configuration database contains a slot with the same MAC address as the joining switch (operation 748). If there is such a slot, which means that the global configuration database contains a slot which has been used previously for the same joining switch's configuration information, such information is copied from the identified slot to the reserved slot (operation 750). Otherwise, the neighboring member switch proceeds to complete the join process as illustrated in FIG. 7A.
  • Switch Joins Another VCS.
  • This use case occurs when a switch is disconnected from one VCS and then connected to a different VCS without being reset to the default state. This scenario can also occur when a switch is connected to a VCS while it is participating in another VCS. In such cases, there will be a VCS_ID mismatch in the join process. In addition, the IN_VCS field in the joining switch's configuration table might or might not be set. If the IN_VCS field is not set, which means that the joining switch is not currently participating in a VCS, the join process can assign the switch a new VCS_ID corresponding to the VCS the switch is joining. In one embodiment, if the IN_VCS filed is set in the joining switch's configuration, which means that the joining switch is currently participating in a different VCS, the join process is disallowed. Optionally, the joining switch can complete the joining process after being set to the default state.
  • Initial Joining of Two Switches which are Both not in a VCS.
  • When two switches are connected together and both of them are not in a VCS, an election process can be used to let one of them be the driving switch in the VCS formation process. In one embodiment, the switch with a lower FC switch domain ID would have its IN_VCS field set to “1” and drives the join process.
  • Joining of Two VCSs.
  • In one embodiment, two VCSs are allowed to merge together. Similar to the FC switch fabric formation process, the logical FC switches in both VCSs would select a new principal FC switch. This newly selected principal FC switch then re-assigns FC switch domain IDs to all the member switches. After the FC switch domain IDs are assigned, a “fabric up” message which is broadcast to all the member switches starts the VCS join process.
  • During the join process, the principal FC switch's IN_VCS field is set to “1,” whereas all other member switches' IN_VCS fields are set to “0.” Subsequently, each member switch can join the VCS (which initially only contains the switch with the principal FC switch) using the “switch joins another VCS” procedure described above.
  • Removal of a Switch from VCS.
  • When a switch is removed from a VCS, its neighboring member switch typically receives a “domain-unreachable” notification at its logical FC switch. Upon receiving this notification, the neighboring member switch disables this switch from the global VCS configuration database and propagates this change to all other member switches. Optionally, the neighboring member switch does not clear the slot previously used by the removed switch in the global configuration database. This way, if the departure of the switch is only temporary, the same slot in the configuration database can still be used when the switch re-joins the VCS.
  • If the VCS is temporarily disjoint due to a link failure, the logical FC infrastructure in the member switches can detect the disconnection of the switch(es) and issues a number of “domain-unreachable” notifications. When the disjoint switch is reconnected to the VCS, a comparison between the switch's configuration information and the corresponding slot information in the global VCS configuration database allows the switch to be added to the VCS using the same slot (i.e., the slot with the same mappedID) in the global configuration database.
  • General Operation.
  • FIG. 7D presents a flowchart illustrating the general operation of a switch during a join process, in accordance with one embodiment of the present invention. This operation applies to both a joining switch and a neighboring VCS member switch. At the beginning of a join process, the logical FC switch within a switch receives an SCN notification on a newly detected switch domain ID on a link (operation 760). The system then exchanges the VCS four-tuple with the neighbor switch (operation 762). Subsequently, the system determines whether itself is part of a VCS (i.e., whether its IN_VCS=1) (operation 764). If the system is not part of a VCS (which means the local switch is attempting to join a VCS), the system then waits for an INVITE from the neighbor member switch (operation 766). After the INVITE is received, the system determines whether the configuration information in the INVITE causes any un-resolved conflict with the local switch (operation 768). If there is an un-resolved conflict, the system rejects the INVITE (operation 770). Otherwise, the system accepts the INVITE (operation 772).
  • If the system determines that itself is already part of a VCS (i.e., its IN_VCS=1) (operation 764), the system then further determines whether there is an existing slot in the global configuration database with the same mappedID as the joining switch (operation 774). If such a slot exists, the system then sends the INVITE to the joining switch (operation 775) and determines whether there is any un-resolved conflict between the configuration information stored in this slot and the information provided by the joining switch (operation 780). If so, the system revokes the INVITE (operation 782). Otherwise, the system updates the global configuration database with the joining switch's configuration information and propagates the update to all other member switches (operation 784).
  • If there is no slot in the global configuration database with the same mappedID as the joining switch (operation 774), the system allocates an interim slot in the global configuration database (operation 776), and sends an INVITE to the joining switch (operation 778). After receiving an INVITE acceptance from the joining switch (operation 779), the system then updates the global configuration database (operation 784) and completes the join process.
  • Layer-2 Services in VCS
  • In one embodiment, each VCS switch unit performs source MAC address learning, similar to what an Ethernet bridge does. Each {MAC address, VLAN} tuple learned on a physical port on a VCS switch unit is registered into the local Fibre Channel Name Server (FC-NS) via a logical Nx_Port interface corresponding to that physical port. This registration binds the address learned to the specific interface identified by the Nx_Port. Each FC-NS instance on each VCS switch unit coordinates and distributes all locally learned {MAC address, VLAN} tuples with every other FC-NS instance in the fabric. This feature allows the dissemination of locally learned {MAC addresses, VLAN} information to every switch in the VCS. In one embodiment, the learned MAC addresses are aged locally by individual switches.
  • FIG. 8 presents a flowchart illustrating the process of looking up an ingress frame's destination MAC address and forwarding the frame in a VCS, in accordance with one embodiment of the present invention. During operation, a VCS switch receives an Ethernet frame at one of its Ethernet ports (operation 802). The switch then extracts the frame's destination MAC address and queries the local FC Name Server (operation 804). Next, the switch determines whether the FC-NS returns an N_Port or an NL_Port identifier that corresponds to an egress Ethernet port (operation 806).
  • If the FC-NS returns a valid result, the switch forwards the frame to the identified N_Port or NL_Port (operation 808). Otherwise, the switch floods the frame on the TRILL multicast tree as well as on all the N_Ports and NL_Ports that participate in that VLAN (operation 810). This flood/broadcast operation is similar to the broadcast process in a conventional TRILL RBridge, wherein all the physical switches in the VCS will receive and process this frame, and learn the source address corresponding to the ingress RBridge. In addition, each receiving switch floods the frame to its local ports that participate in the frame's VLAN (operation 812). Note that the above operations are based on the presumption that there is a one-to-one mapping between a switch's TRILL identifier (or nickname) and its FC switch domain ID. There is also a one-to-one mapping between a physical Ethernet port on a switch and the corresponding logical FC port.
  • End-to-End Frame Delivery
  • FIG. 9 illustrates how data frames and control frames are transported in a VCS, in accordance with an embodiment of the present invention. In this example, a VCS 930 includes member switches 934, 936, 938, 944, 946, and 948. An end host 932 is communicating with an end host 940. Switch 934 is the ingress VCS member switch corresponding to host 932, and switch 938 is the egress VCS member switch corresponding to host 938. During operation, host 932 sends an Ethernet frame 933 to host 940. Ethernet frame 933 is first encountered by ingress switch 934. Upon receiving frame 933, switch 934 first extracts frame 933's destination MAC address. Switch 934 then performs a MAC address lookup using the Ethernet name service, which provides the egress switch identifier (i.e., the RBridge identifier of egress switch 938). Based on the egress switch identifier, the logical FC switch in switch 934 performs a routing table lookup to determine the next-hop switch, which is switch 936, and the corresponding output port for forwarding frame 933. The egress switch identifier is then used to generate a TRILL header (which specifies the destination switch's RBridge identifier), and the next-hop switch information is used to generate an outer Ethernet header. Subsequently, switch 934 encapsulates frame 933 with the proper TRILL header and outer Ethernet header, and sends the encapsulated frame 935 to switch 936. Based on the destination RBridge identifier in the TRILL header of frame 935, switch 936 performs a routing table lookup and determines the next hop. Based on the next-hop information, switch 936 updates frame 935's outer Ethernet header and forwards frame 935 to egress switch 938.
  • Upon receiving frame 935, switch 938 determines that it is the destination RBridge based on frame 935's TRILL header. Correspondingly, switch 938 strips frame 935 of its outer Ethernet header and TRILL header, and inspects the destination MAC address of its inner Ethernet header. Switch 938 then performs a MAC address lookup and determines the correct output port leading to host 940. Subsequently, the original Ethernet frame 933 is transmitted to host 940.
  • As described above, the logical FC switches within the physical VCS member switches may send control frames to one another (for example, to update the VCS global configuration database or to notify other switches of the learned MAC addresses). In one embodiment, such control frames can be FC control frames encapsulated in a TRILL header and an outer Ethernet header. For example, if the logical FC switch in switch 944 is in communication with the logical FC switch in switch 938, switch 944 can sends a TRILL-encapsulated FC control frame 942 to switch 946. Switch 946 can forward frame 942 just like a regular data frame, since switch 946 is not concerned with the payload in frame 942.
  • Automatic Port Profile Management
  • Today's server virtualization infrastructure (e.g. a Hypervisor, also called virtual machine monitor) associates a server side (e.g. Hypervisor or adapter) Virtual Ethernet Bridge (VEB) port profile to each Ethernet MAC address used by a virtual machine (VM) to access the network through a VEB port. Examples of the VEB's port profile attributes includes: the types of frames allowed on the port (e.g. all frames, only frames tagged with certain VLAN values, or untagged frames), the VLAN identifiers that are allowed to be used, and rate limiting attributes (e.g. port or access-control based rate limits). In today's server virtualization infrastructure, if the VM migrates from one physical server to another, the VEB's port profile migrates with it. In other words, today's server virtualization infrastructure provides automated port profile migration of the server's VEB port(s) that are associated with a VM.
  • However, in existing technologies, there remains a gap between the access and Quality of Service (QoS) controls supported in external layer 2 switches and server virtualization infrastructure. That is, external layer 2 switches have more advanced controls compared to server VEB implementations. Although server virtualization infrastructure is continually adding these controls, this gap is expected to remain. Some environments prefer the more advanced controls provided by external network switches. An example of such an environment is a multi-tier data center that has several types of applications, each with differing advanced network controls, running over the same layer-2 network. In this type of environment the network administrator often prefers the use of advanced access controls available in external switches.
  • Today's layer-2 networks do not provide a mechanism for automatically migrating switch access and traffic controls associated with an end-point device (e.g., a VM), when that device migrates from one switch to another. The migration may be physical, such as an Operating System image (application, middleware, Operating System and associated state) that is running on one physical system and is migrated to another system. The migration may be also be virtual, such as an Operating System image (OS image) that is running over a Hypervisor on one system and is migrated to run over a Hypervisor on another system.
  • Embodiments of the present invention provides a mechanism for automatically migrating port profiles resident in a switch and associated with an OS image to a port on a second switch, when that OS image migrates from one physical end-host system to another end-host system, which is attached to the second switch.
  • FIG. 10 illustrates a logical VCS access layer (VAL) which includes an automatic port profile manager, in accordance with one embodiment of the present invention. In this example, a VCS 1000 is coupled with a number of physical server systems, such as system 1002. Each physical server system runs a number of virtual machines (VMs, also called virtual servers). For example, system 1002 includes four VMs, one of which is VM 1004. A VM may be dedicated to a certain application (e.g., instant messaging services, directory services, data base applications, etc.) and may have its own requirement on the network. A VM running mission-critical applications may require a separate VLAN within VCS 1000 and may have more strict QoS requirement (such as guaranteed port bandwidth, low latency, and guaranteed packet delivery). A VM running non-critical applications may have much lower requirements.
  • The switches within VCS 100 which are coupled externally to the physical end-host systems form a logical VCS access layer (VAL) 1010. The automatic migration of port profiles (AMPP) is implemented in VAL 1010. During operation, various port profiles, which are often tailored to different requirements of the VMs, are created and distributed to all the member switches in VCS 1000. As described in detail below, when the packets generated by a VM are detected by an ingress member switch of VCS 1000, the VM's source MAC address is recognized and used to identify the corresponding port profile, which is then applied to the appropriate ingress switch port. When a VM moves from one physical server to another, the MAC-address detection mechanism can quickly identify the new physical switch port to which the VM is coupled to, and apply the same port profile to the new port.
  • FIG. 11 illustrates an example of the operation of AMPP, in accordance with one embodiment of the present invention. In this example, a VCS 1100 includes two switches 1120 and 1122, which are coupled to two physical servers, 1116 and 1118, respectively. Physical server 1116 hosts four VMs, 1102, 1104, 1106, and 1008. Each VM has a virtual port (VP, or virtual network interface card, VNIC). For example, VM 1102 has a VP 1110. A respective VP is assigned a virtual MAC address. The four VPs are logically coupled to a virtual switch 1112 which is provided by a hypervisor 1114. Virtual switch 1112 is responsible for dispatching outgoing and incoming traffic through a physical NIC 1117. Note that an Ethernet frame generated by a respective VM has the virtual MAC of the corresponding VP as its source address. Logically, virtual switch 1112 functions as an aggregation point that provides a link to the ingress member switch in VCS 1100. Physical server 1118 has a similar architecture. During operation, a VM can migrate from one physical server to another (e.g., “VMotion” function provided by VMware). This migration can be event-driven or pre-scheduled. Such migration is often used to cope with changing dynamics in a number of parameters, such as server load, power consumption, resource utilization, etc.
  • During operation, one or more port profiles can be created to specify a number of requirements/restrictions/limitations that should be enforced at a VCS switch port corresponding to one or more VMs. For example, a port profile for VM 1102 (which can be identified by the virtual MAC address of VP 1110) can be created and distributed to every member switch of VCS 1100. When VM 1102 sends its first Ethernet frame to the network, switch 1120 would learn this source MAC address. Upon learning VP 1110's MAC address, switch 1120 then searches its port profile database and identifies the matching port profile. Subsequently, the identified port profile is applied to the port on switch 1120 which is coupled to system 1116. In addition, the same port profile is applied to the port where the matching MAC address is the destination MAC address of a frame. This way, the same network parameters are enforced at both ingress and egress ports of the VCS. Note that the port profile might include “soft” parameters. In other words, the requirements and limitations in the port profile may be specific to certain MAC addresses, and may not be “hard” limitations on the physical parameters of the switch port, since traffic from/to multiple VMs is handled by the same physical switch port.
  • In one embodiment, VCS 1100 provides a mechanism that distributes all the port profiles and the port-profile-to-MAC mapping information to all the member switches. The port profiles can be created using a command line interface (CLI) or other network management software. In addition, upon migration of a VM (such as a VMware VMotion), the target switch port in the VCS can automatically activate the correct port profile configuration.
  • FIG. 12A illustrates exemplary port profile contents, in accordance with one embodiment of the present invention. As shown in FIG. 12A, a port profile can contain the entire configuration needed for a VM to gain access to a LAN or WAN, which can include: Fibre Channel over Ethernet (FCoE) configuration, VLAN configuration, QoS related configuration, and security related configuration (such as access control lists, ACLs). The list above is by no means complete or exhaustive. Furthermore, it is not necessary that a port profile contains every type of configuration information.
  • In one embodiment, a port profile can be capable of operating as a self contained configuration container. In other words, if a port profile is applied to a new switch without any additional configuration, the port profile should be sufficient to set the switch's global and local (interface level) configuration and allow the switch to start carrying traffic.
  • A VLAN configuration profile within a port profile can define:
      • a VLAN membership which includes tagged VLANs and an untagged VLAN; and
      • ingress/egress VLAN filtering rules based on the VLAN membership.
  • A QoS configuration profile within a port profile can define:
      • mapping from an incoming frame's 802.1p priority to internal queue priority; (if the port is in QoS untrusted mode, all incoming frame's priorities would be mapped to the default best-effort priority)
      • mapping from an incoming frame's priority to outgoing priority;
      • scheduling profile, such as weighted Round-Robin or strict-priority based queuing;
      • mapping of an incoming frame's priority to strict-priority based or weighted Round-Robin traffic classes;
      • flow control mechanisms on a strict-priority based or weight Round-Robin traffic class; and
      • limitations on multicast datarate.
  • An FCoE configuration profile within a port profile defines the attributes needed for the port to support FCoE, which can include:
      • FCoE VLAN;
      • FCMAP;
      • FCoE Priority; and
      • virtual Fabric ID.
  • A security configuration profile within a port profile defines the security rules needed for the server port. However, the security rules can be different at different ports, so some of the locally configured ACLs can be allowed to override conflicting rules from a port profile. A typical security profile can contain the following attributes:
      • Enable 802.1x with EAP TLV extensions for VM mobility; and
      • MAC based standard and extended ACLs.
  • In one embodiment, each port profile can have one or more MAC addresses associated with it. FIG. 12B illustrates three exemplary port profiles, in accordance with on embodiment of the present invention. In this example, port profile PP-1 is associated with 5 MAC addresses. These MAC address can be virtual MAC addresses assigned to different VMs. The port-profile-to-MAC mapping information is distributed throughout the VCS. A port profile can be activated on a server port in three ways: (1) when a hypervisor binds a MAC address to a port profile ID; (2) through regular MAC learning; and (3) through a manual configuration process via a management interface.
  • It is possible to group a set of VMs in the network by associating them with one port profile. This group can be used to dictate forwarding between the VMs. FIG. 13 illustrates how forwarding is achieved between VMs based on port profile membership, in accordance with one embodiment of the present invention. In this example, port profile 1 has two members: MAC-1 and MAC-3. Port profile 2 has four members: MAC-2, MAC-4, MAC-5, and MAC-6. All the VMs belong to the same VLAN X. Based on the tuple <MAC, VLAN ID>, a policy group ID (GID) can be determined. All the MAC addresses mapped to the same port profile should belong to the same policy group which dictates the forwarding boundary. This configuration allows enforcing different forwarding domains within a VLAN, as illustrated in FIG. 13B. The system then ensures that both the source MAC address and destination MAC address are part of the same port profile.
  • FIG. 14 presents a flowchart illustrating the process of creating and applying a port profile, in accordance with one embodiment of the present invention. During operation, the system receives a user created port profile with the corresponding VM's MAC address (operation 1402). This MAC address is then associated with the port profile and can be later used to identify the profile. The system then determines whether the new profile creates dependencies on other existing profiles or conflicts (operation 1404). If so, the system allows the user to resolve the conflicting configuration and/or dependencies (operation 1406).
  • Subsequently, the system distributes the port profile and the corresponding VM MAC address to every member switch throughout the VCS fabric (operation 1408). When a VM is initiated or migrated, the system then detects a matching virtual MAC address from the received ingress packets (operation 1410). Based on the learned MAC address, the system then activates the corresponding port profile on the switch port (operation 1412).
  • Exemplary VCS Member Switch with AMPP
  • FIG. 15 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention. In this example, the VCS member switch is a TRILL RBridge 1500 running special VCS software. RBridge 1500 includes a number of Ethernet communication ports 1501, which can transmit and receive Ethernet frames and/or TRILL encapsulated frames. Also included in RBridge 1500 is a packet processor 1502, a virtual FC switch management module 1504, a logical FC switch 1505, a VCS configuration database 1506, an AMPP management module 1507, and a TRILL header generation module 1508.
  • During operation, packet processor 1502 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames. Virtual FC switch management module 1504 maintains the state of logical FC switch 1505, which is used to join other VCS switches using the FC switch fabric protocols. VCS configuration database 1506 maintains the configuration state of every switch within the VCS. TRILL header generation module 1508 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • Upon learning a new MAC address from a VM, AMPP management module 1507 identifies the port profile corresponding to the learned MAC, and applies the identified port profile. This dynamic port profile application process can respond to VM migration within a very short period of time, thereby facilitating automatic port configuration in a virtual computing environment.
  • Advanced Link Tracking
  • Today's server virtualization infrastructure (e.g. a Hypervisor, also called virtual machine monitor) typically provides one or more virtual switches (also called virtual Ethernet bridges, VEBs) within a physical server. Each virtual switch serves a number of virtual machines. When a number of such servers connect to a VCS, the number of communication sessions among the virtual machines can be quite large. In such a network environment, when a network link or port fails, the failure would typically disrupt the reachability to one or more virtual machines. This disruption can affect the communication sessions of some of the virtual machines. In conventional networks, such reachability disruption only triggers a topology change and/or MAC address learning update in the network, and the source virtual machines are not notified about these updates. Correspondingly, with conventional technologies, there is no way for a Hypervisor to re-configure the connectivity of the virtual machines absent of some signaling from the network via proprietary protocols.
  • Embodiments of the present invention facilitate advanced link tracking by monitoring any reachability disruption in the network and notifying the affected hypervisor. In response, the hypervisor can re-configure the connectivity of the virtual machines under its control to bypass the failed link or port. In one embodiment, this advanced link tracking function can be carried out in a logical VCS access layer.
  • FIG. 16 illustrates an exemplary configuration of advanced link tracking in a VCS, in accordance with one embodiment of the present invention. In this example, a VCS 1600 includes four switches (which can be RBridges), 1620, 1622, 1624, and 1626. A physical server 1618 is coupled to both switches 1622 and 1624 via two network interface cards (NICs), 1603 and 1605, respectively. Physical server 1618 hosts four VMs, 1622, 1624, 1626, and 1628, which are managed by a hypervisor 1601. Hypervisor 1601 provides two virtual switches, 1602 and 1604. Each VM has two virtual ports (VPs), and is coupled to both virtual switches 1602 and 1604 via the VPs. In other words, each VM within physical server 1618 is dual-homed with virtual switches 1602 and 1604. This configuration provides redundancy to each VM, so that when one of the physical NICs (i.e., NIC 1603 or 1605) fails, hypervisor 1601 can instruct the VMs to use the other working NIC. During normal operation, for load-balancing purposes, VMs 1622 and 1624 are configured to communicate via virtual switch 1602, and VMs 1626 and 1628 are configured to communicate via virtual switch 1604.
  • Also coupled to VCS 1600 is physical servers 1617, which has a similar configuration as server 1618. Server 1617 includes four VMs, 1632, 1634, 1636, and 1638. These four VMs are each dual-homed with virtual switches 1642 and 1644, which are provided by hypervisor 1641. Virtual switch 1642 is coupled to VCS member switch 1620 via a NIC 1643, and virtual switch 1644 is coupled to VCS member switch 1626 via a NIC 1645. During normal operation, VMs 1632 and 1634 communicate with VCS 1600 via virtual switch 1642 and NIC 1643, and VMs 1636 and 1638 communicate with VCS 1600 via virtual switch 1644 and NIC 1645.
  • Assume that VMs 1622 and 1624 are in communication with VMs 1636 and 1638. Since VMs 1636 and 1638 are configured by hypervisor 1641 to use virtual switch 1644 and NIC 1645, the traffic between VMs 1622 and 1624 and VMs 1636 and 1638 is normally carried by VCS member switch 1626. Now, assume the link between switches 1620 and 1626 fails. As a result, VMs 1636 and 1638 can no longer be reached via NIC 1645. In embodiments of the present invention, this reachability update information is not only reflected in the VCS topology update (which is handled by the routing protocol within VCS 1600), but also communicated to hypervisor 1601 via NIC 1603. This update can allow hypervisor 1601 to quickly re-configure VMs 1622 and 1624, so that these two VMs use virtual switch 1604 and NIC 1605 to access VCS 1600. This way, the traffic from VMs 1622 and 1624 can still reach VMs 1636 and 1638 via switch 1624, switch 1620, NIC 1643, and virtual switch 1642. The new data path bypasses the failed link between switches 1620 and 1626. This re-configuration can take place shortly after the link failure is detected, thereby facilitating fast recovery at the source VMs.
  • FIG. 17 illustrates an example where advanced link tracking allows virtual machines to re-route egress traffic when a link fails, in accordance with one embodiment of the present invention. In this example, two servers 1702 and 1704 are coupled to a VCS 1700. Server 1702 hosts four VMs, 1706, 1708, 1710, and 1712, all of which are dual-homed with virtual switches 1714 and 1716. During operation, VMs 1706 and 1708 access VCS 1700 via VS 1714, and VMs 1710 and 1712 access VCS 1700 via VS 1716. Server 1704 have a similar configuration as server 1702. Assume that throughout VCS 1700 there is only one path leading from VS 1714 to VS 1718 in server 1704. Assume further that during operation the egress port coupling to VS 1718 in server 1704 fails. As a result, VS 1718 is no longer reachable from VS 1714. The advanced link tracking mechanism can notify VS 1714 of the lost reachability to VS 1718. In one embodiment, VCS 1700 can communicate with a third entity which maintains the connectivity-pattern information among all the VMs (such as the vCenter by VMware) to obtain information on the affected VMs. In further embodiments, VCS 1700 can notify every external port of the lost reachability, and let the individual hypervisor to determine whether re-configuration of the VM-to-VS connectivity is necessary.
  • FIG. 18 presents a flowchart illustrating the process of advance link tracking, in accordance with one embodiment of the present invention. During operation, the system first detects a link (or port) failure in the VCS (operation 1802). The system then determines whether the failure affects reachability of an end host (operation 1804). If the failure does not affect reachability of any end host, it is assumed that VCS can recover from the failure after its topology converges and the routing protocol updates every switch's forwarding table. If the reachability of an end host is affected, the system then optionally identifies ingress port(s) which are in communication with the affected end host(s) (operation 1806). Subsequently, the system notifies the end hosts via the ingress ports of the reachability disruption (operation 1808).
  • Exemplary VCS Member Switch with Advanced Link Tracking
  • FIG. 19 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention. In this example, the VCS member switch is a TRILL RBridge 1900 running special VCS software. RBridge 1900 includes a number of Ethernet communication ports 1901, which can be coupled to one or more servers hosting virtual machines and which can transmit and receive Ethernet frames and/or TRILL encapsulated frames. Also included in RBridge 1900 is a packet processor 1902, a virtual FC switch management module 1904, a logical FC switch 1905, a VCS configuration database 1906, an advanced link tracking module 1907, and a TRILL header generation module 1908.
  • During operation, packet processor 1902 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames. Virtual FC switch management module 1904 maintains the state of logical FC switch 1905, which is used to join other VCS switches using the FC switch fabric protocols. VCS configuration database 1906 maintains the configuration state of every switch within the VCS. TRILL header generation module 1908 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches.
  • Upon learning about disrupted reachability in the VCS, advanced link tracking module 1907 identifies the port(s) which are affected by the disruption, and notifies the hypervisor of the disruption. This notification can allow the hypervisor to expedite the re-configuration of the affected VMs and minimize service disruption. Furthermore, advanced link tracking module 1907 also monitors the health of all the links corresponding to ports 1901. Upon detection of any link or port failure, advanced link tracking module 1907 can notify other switches in the VCS of the link state change and any reachability disruption.
  • Virtual Port Grouping
  • Today's server virtualization infrastructure (e.g. a Hypervisor, also called virtual machine monitor) typically provides one or more virtual switches (also called virtual Ethernet bridges, VEBs) within a physical server. Each virtual switch serves a number of virtual machines. When a number of such servers connect to a VCS, the number of communication sessions among the virtual machines can be quite large. Often, different virtual machines belong to different customers, and different customers usually require dedicated and isolated network resources for security and quality of service purposes.
  • In one embodiment, each customer's virtual machines are placed in the same group based on the MAC address and VLAN tag of their associated virtual ports. The grouping information is replicated and stored at every VCS member switch. During operation, when a frame arrives at a VCS member switch, the member switch can enforce the grouping policy by looking up the group ID corresponding to the frame's source MAC address and VLAN tag. If the frame's destination MAC address combined with the VLAN tag belong to the same group, then the frame is forwarded to the proper next-hop switch. Otherwise, the frame is dropped. This configuration ensures that each customer's virtual machines only communicate within themselves, thereby creating virtual partitions within a VCS.
  • FIG. 20 illustrates an exemplary VCS network environment with virtual port grouping, in accordance with one embodiment of the present invention. In this example, a VCS 2000 includes four switches (which can be RBridges), 2020, 2022, 2024, and 2026. A physical server 2018 is coupled to both switches 2022 and 2024 via two network interface cards (NICs), 2003 and 2005, respectively. Physical server 2018 hosts four VMs, 2022, 2024, 2026, and 2028, which are managed by a hypervisor 2001. Hypervisor 2001 provides two virtual switches, 2002 and 2004. Each VM has two virtual ports (VPs), and is coupled to both virtual switches 2002 and 2004 via the VPs. In other words, each VM within physical server 2018 is dual-homed with virtual switches 2002 and 2004. This configuration provides redundancy to each VM, so that when one of the physical NICs (i.e., NIC 2003 or 2005) fails, hypervisor 2001 can instruct the VMs to use the other working NIC. During normal operation, for load-balancing purposes, VMs 2022 and 2024 are configured to communicate via virtual switch 2002, and VMs 2026 and 2028 are configured to communicate via virtual switch 2004.
  • Also coupled to VCS 2000 is physical servers 2017, which has a similar configuration as server 2018. Server 2017 includes four VMs, 2032, 2034, 2036, and 2038. These four VMs are each dual-homed with virtual switches 2042 and 2044, which are provided by hypervisor 2041. Virtual switch 2042 is coupled to VCS member switch 2020 via a NIC 2043, and virtual switch 2044 is coupled to VCS member switch 2026 via a NIC 2045. During normal operation, VMs 2032 and 2034 communicate with VCS 2000 via virtual switch 2042 and NIC 2043, and VMs 2036 and 2038 communicate with VCS 2000 via virtual switch 2044 and NIC 2045.
  • Assume that VMs 2024, 2028, 2032, and 2036 belong to the same customer (as indicated by the shading in each drawing element in FIG. 20), and that these VMs are restricted to communicate only within themselves. Typically, the customer would set a particular VLAN value to these VMs. However, the customer VLAN value might not work within VCS 2000, because the customer VLAN value might conflict with existing VLAN settings within VCS 2000. One way to solve this problem is to place the MAC addresses corresponding to the virtual ports of a particular customer's virtual machines, together with the customer's VLAN value, in a customer-specific group, and assign a unique group ID to this group. This grouping information can be a data structure that maps a {MAC, VLAN TAG} tuple to a group ID. This data structure is replicated and distributed to every member switch in VCS 2000.
  • FIG. 21A illustrates an exemplary data structure for storing virtual port grouping information, in accordance with one embodiment of the present invention. In this example, each {MAC, VLAN TAG} tuple maps to a group ID, which is used to restrict forwarding of frames between different groups.
  • FIG. 21B illustrates logical virtual networks within a VCS as a result of virtual port grouping, in accordance with one embodiment of the present invention. In this example, two servers 2102 and 2104 are coupled to a VCS 2100. Server 2102 hosts four VMs, 2106, 2108, 2110, and 2112, all of which are dual-homed with virtual switches 2114 and 2116. Server 2104 hosts four VMs, 2122, 2124, 2126, and 2128, all of which are dual-homed with virtual switches 2118 and 2120. VMs 2108, 2112, 2124, and 2128 belong to the same customer and are expected to communicate only within themselves.
  • During operation, a VM administrator 2124 configures VMs 2108, 2112, 2124, and 2128 with the same VLAN tag corresponding to their common customer. Correspondingly, the MAC address of the virtual ports for these machines, together with the designated VLAN tag, are assigned with the same group ID. VMs 2106, 2110, 2122, and 2126 and their respective VLAN tags are assigned a different group ID. This grouping information is then communicated to VCS 2100 and stored at every member switch.
  • When the frames are processed and forwarded within VCS 2100, frames with one group ID are prohibited from being forwarded to a MAC address-VLAN combination that corresponds to a different group ID. Thus, effectively, the virtual port grouping creates two virtual networks 2120 and 2121 within VCS 2100.
  • FIG. 22 presents a flowchart illustrating the process of enforcing virtual port grouping, in accordance with one embodiment of the present invention. During operation, a VCS member switch receives in ingress frame from an end host (operation 2202). The switch then identifies the group ID based on the source MAC address and VLAN tag of the received frame (operation 2204). In addition, the switch further determines whether the destination MAC address and VLAN tag of the received frame point to the same group ID (operation 2206). If so, the switch identifies the output port based on the destination MAC address and VLAN tag (operation 2208), and forwards the frame to the identified output port (operation 2210). Otherwise, the frame is discarded (operation 2212).
  • Exemplary VCS Member Switch with Virtual Port Grouping
  • FIG. 23 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention. In this example, the VCS member switch is a TRILL RBridge 2300 running special VCS software. RBridge 2300 includes a number of Ethernet communication ports 2301, which can be coupled to one or more servers hosting virtual machines and which can transmit and receive Ethernet frames and/or TRILL encapsulated frames. Also included in RBridge 2300 is a packet processor 2302, a virtual FC switch management module 2304, a logical FC switch 2305, a VCS configuration database 2306, virtual port grouping module 2307, and a TRILL header generation module 2308.
  • During operation, packet processor 2302 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames. Virtual FC switch management module 2304 maintains the state of logical FC switch 2305, which is used to join other VCS switches using the FC switch fabric protocols. VCS configuration database 2306 maintains the configuration state of every switch within the VCS. TRILL header generation module 2308 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches. Virtual port grouping module 2307 is responsible for maintaining a copy of the grouping information and enforcing the grouping policies. As described in conjunction with FIG. 22, when a frame's {source MAC, VLAN TAG} and {destination MAC, VLAN TAG} do not map to the same group ID, the frame is prevented from being forwarded to any of the output ports. In one embodiment, virtual port grouping module 2307 can communicate with a VM administrator (such as the VMware Vcenter) to obtain the virtual port grouping information (e.g., the complete data structure illustrated in FIG. 21A).
  • VCS Name Services
  • VCS allows an interconnected fabric of RBridges to function as a single logical switch. The VCS name services facilitate fast distribution of run-time network state changes, including newly learned MAC addresses (which is referred to as “Ethernet name service” or “Ethernet NS” in this disclosure) and multi-chassis trunk (MCT) port state updates (which is referred to as “MCT name service” or “MCT NS” in this disclosure). More details on MCT are provided in U.S. patent application Ser. No. 12/725,249, (attorney docket number BRCD-112-0439US), entitled “REDUNDANT HOST CONNECTION IN A ROUTED NETWORK,” by inventors Somesh Gupta, Anoop Ghanwani, Phanidhar Koganti, and Shunjia Yu, filed 16 Mar. 2010, the disclosure of which is incorporated by reference herein.
  • The Ethernet NS provides the ability to distribute various information across the VCS. The MAC information learned at one member switch is distributed to all other member switches, which facilitates fast MAC moves (for example, during migration of virtual machines) and global MAC learning. In some embodiments, layer-2 multicast information, which can be a multicast MAC address with corresponding switch/port identifiers and VLAN tag, can be distributed to facilitate efficient VCS-wide multicast. Optionally, Ethernet NS provides a distribution mechanism and does not maintain a central storage of the MAC-related knowledge base. In other words, the Ethernet NS knowledge database is replicated and stored distributively among all the VCS member switches.
  • Each member switch maintains a database of all the MAC addresses learned throughout the VCS. This database can be used to minimize the amount of flooding (a default behavior of Ethernet switch when a frame's destination MAC address is not recognized). Ethernet NS also provides VCS-wide distribution of multicast MAC-to-RBridge/Port mapping information which can be obtained by Internet Group Management Protocol (IGMP) snooping. (Details about IGMP and IGMP snooping can be found at IETF RFC 3376 available at http://tools.ietf.org/html/rfc3376 and IETF RFC 4541 available at http://tools.ietf.org/html/rfc4541.) Ethernet NS distributes this information to all RBridges, thereby allowing the VCS to behave as a single switch. By tracking and forwarding IGMP join and leave information, the Ethernet NS can efficiently track the multicast MAC information and maintain an accurate layer-2 multicast group.
  • One of the requirements of presenting a VCS as a single switch is to support connection of trunked links from external hosts to different RBridges within the VCS fabric. Such trunking which involves connection to different RBridges is referred to as multi-chassis trunking (MCT). Conceptually, support within the VCS fabric for routing to a MCT destination is achieved by presenting each MCT group (i.e., each trunk) as a virtual RBridge. In some embodiments, the virtual RBridge is not assigned a domain ID and thus does not utilize FSPF for routing setup. Instead, the a primary RBridge hosting the MCT distributes the virtual RBridge ID and the corresponding link state updates to the VCS fabric. The primary RBridge is responsible for learning a new MAC via an MCT and distributing the new MAC information to the VCS.
  • When an RBridge joins the VCS it will request a dump of the local NS database from the remote RBridge. It will not respond to individual updates from the remote RBridge until the DB dump has been received. After the database is in sync between two RBridges, individual changes are detected locally and pushed remotely. If a local database receives domain unreachable it is responsible for removing all records for that remote domain and doing any local notification that this removal implies.
  • FIG. 24 illustrates an example of name service operation in a VCS, in accordance with one embodiment of the present invention. In this example, a VCS 2400 includes four member switches (Rbridges), 2402, 2404, 2406, and 2408. Assume that an end host 2414 is coupled to switch 2402 during operation. When end host 2414 sends its first Ethernet frame, switch 2402 would not recognize the source MAC address of this ingress frame. Upon receiving this ingress frame, switch 2402 then determines the port (or interface) on which the frame arrives and the frame's VLAG tag. Subsequently, switch 2402 assembles an Ethernet NS update frame which indicates the learned MAC address (which corresponds to end host 2414), its switch identifier (which in one embodiment is the RBridge ID of switch 2402), the port identifier, and the VLAG tag for the frame. In one embodiment, this frame is an FC registered state change notification (RSCN) encapsulated in a TRILL header. Note that switch 2402 can obtain the information of all other member switches in the VCS by looking up the global configuration database. Subsequently, switch 2402 can send the Ethernet NS update frame to switches 2404, 2408, and 2406, respectively. Upon receiving the Ethernet NS update frame, each member switch updates its own MAC database accordingly. In this way, when one of the member switches receives an Ethernet frame destined to end-host 2414, it can forward that frame to switch 2402 (instead of flooding the frame to all of its ports).
  • Also shown in the example in FIG. 24 is an MCT group 2416. MCT group 2416 is formed by an end host 2412 which is dual-homed with switches 2406 and 2408. Assume that switch 2406 is the primary RBridge in MCT group 2416. When end host 2412 and MCT group 2410 is first configured, switch 2406 assigns a virtual RBridge 2410 to MCT group 2410. In addition, switch 2406 notifies the rest of VCS 2400 about the MAC address of end host 2412. Note that the NS update associated the MAC address of end host 2412 indicates the identifier of virtual RBridge 2410 (instead of the identifier of either switch 2406 or switch 2408). In this way, the rest of VCS 2400 can associate end host 2412 with virtual RBridge 2410. When forwarding a frame destined to end host 2412, a member switch in VCS 2400 would forward the frame toward virtual RBridge 2410 (i.e., by setting RBridge 2410 as the destination RBridge in the TRILL header). Note that switch 2406 is also responsible for distributing the link state information with respect to the virtual connectivity between virtual RBridge 2410 and switches 2406 and 2408 (indicated by the dotted lines).
  • In case when one of the links (i.e., either the link between switch 2406 and end host 2412, or the link between switch 2408 and end host 2412) fails, as part of the MCT NS, in one embodiment, primary RBridge 2406 is responsible for updating the rest of the VCS 2400 that host 2412's MAC address is no longer associated with virtual RBidge 2410. Instead, the MAC address of host 2412 is now associated with the switch to which host 2412 remains connected. In a further embodiment, it can be the responsibility of the switch that remains connected to host 2412 to distribute the updated MAC address association to the rest of VCS 2400.
  • FIG. 25 presents a flowchart illustrating the process of distributing learned MAC information by the Ethernet name service in a VCS, in accordance with one embodiment of the present invention. During operation, a VCS member switch detects an ingress frame with a new source MAC address (operation 2502). The switch then identifies the port on which the ingress frame is received (operation 2504). Subsequently, the switch assembles an Ethernet NS update frame with the learned MAC address, the switch identifier, port identifier, and VLAN tag (operation 2506). The switch then distributes the Ethernet NS update frames to all member switches in the VCS (operation 2508).
  • FIG. 26 presents a flowchart illustrating the process of distributing information of a learned MAC address via an MCT, in accordance with one embodiment of the present invention. During operation, assume that one of the switches in a MCT group detects an ingress frame with a new source MAC address (operation 2602). The switch then determines whether the end host which generates the frame is dual-homed with the MCT group (operation 2604). In one embodiment, the switch can make this determination by communicating with the other switch of the MCT group. In a further embodiment, the switch can inspect the link aggregation group (LAG) ID of the ingress frame to determine whether the end host is transmitting using a LAG. If the frame is an MCT frame, the switch then assembles an Ethernet NS update frame with the MAC address, the virtual RBridge identifier corresponding to the MCT, a port identifier, and the VLAG tag of the frame (operation 2606).
  • If the frame is determined to be from a regular end host (i.e., not a dual-homed host), the switch assembles an Ethernet NS updated frame with the MAC address, the local physical switch identifier (as opposed to the virtual RBridge ID), the identifier of the port on which the frame is received, and the frame's VLAN tag (operation 2607). The switch then distributes the Ethernet NS update frames to all the member switches in the VCS (operation 2608).
  • FIG. 27 presents a flowchart illustrating the process of updating the link state in an MCT group, in accordance with one embodiment of the present invention. During operation, assume one of the MCT partner switches detects a link or port failure which is part of the MCT group (operation 2702). Note that this failure can be detected locally (which means a port on the local switch or a link coupled to a local port has failed), or be detected remotely (which means that the failure occurs on the partner switch and the local switch is notified of the failure by the partner switch). The switch then determines whether the MCT end host is still connected to the local switch (operation 2704). If the end host is no longer connected to the local switch, the local switch optionally notifies the other partner switch in the MCT of the failure (operation 2710) and takes no further actions, assuming that the partner switch will assume responsibility of updating the link state (using, for example, the same procedure illustrated in FIG. 27).
  • If the MCT end host is still connected to the local switch, the switch then assembles an NS update frame with the end host's MAC address, the local switch's identifier (e.g., the physical RBridge ID of the local switch), the identifier of the port thought which the end host is connected, and the proper VLAN tag (operation 2706). The switch then distributes the NS update frames to all member switches in the VCS (operation 2708).
  • FIG. 28 illustrates an exemplary switch that facilitates formation of a virtual cluster switch with Ethernet and MCT name services, in accordance with an embodiment of the present invention. The VCS member switch is a TRILL RBridge 2800 running special VCS software. RBridge 2800 includes a number of Ethernet communication ports 2801, which can transmit and receive Ethernet frames and/or TRILL encapsulated frames. Also included in RBridge 2800 is a packet processor 2802, a virtual FC switch management module 2804, a logical FC switch 2805, a VCS configuration database 2806, a name services management module 2807, and a TRILL header generation module 2808.
  • During operation, packet processor 2802 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames. Virtual FC switch management module 2804 maintains the state of logical FC switch 2805, which is used to join other VCS switches using the FC switch fabric protocols. VCS configuration database 2806 maintains the configuration state of every switch within the VCS. TRILL header generation module 2808 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches. Based on the extracted MAC addresses of incoming frames, NS management module 2807 distributes the NS update frames to the rest of the VCS. NS management module 2807 also maintains a copy of NS database 2809. NS database 2809 stores all the learned MAC address information from every member switch in the VCS.
  • Equal Cost Multi-Pathing
  • Because VCS uses the Fabric Shortest Path First (FSPF) routing protocol and the TRILL transport, embodiments of the present invention can facilitate equal-cost or nearly-equal-cost multi-pathing for load balancing purposes. FIG. 29 illustrates an exemplary equal cost multi-pathing configuration in a VCS, in accordance with one embodiment of the present invention. In this example, a VCS 2930 includes switches 2934, 2936, 2938, 2944, 2946, and 2948. Assume that a traffic flow enters VCS 2930 via ingress switch 2934 and exits VCS 2930 via egress switch 2948. Based on the hop distance, there are two equal-cost data paths, 2950 and 2952. In one embodiment, when determining the output interface, switch 2934's forwarding engine can return two possible output ports, corresponding to data paths 2950 and 2952, respectively. If load balancing is desired, traffic from switch 2934 to switch 2948 can be split between the two data paths.
  • In one embodiment, the traffic splitting can be based on any field in a frame header. For example, the traffic can be split based on VLAN tags. The traffic can also be split based on layer-4 port numbers or application types.
  • Flow Control and Congestion Management
  • In some embodiments of the present invention, the VCS can achieve loss-less packet transport by implementing a number of flow control mechanisms. In one embodiment, the VCS member switches implement a priority-based flow control (PFC)-like mechanism to ensure a loss-less packet transport between neighboring switches. With priority-based flow control, traffic class is identified by the VLAN tag priority values. Priority-based flow control is intended to eliminate frame loss due to congestion. This is achieved by a mechanism similar to the IEEE 802.3x PAUSE, but operating on individual priorities. This mechanism enables support for higher layer protocols that are highly loss sensitive while not affecting the operation of traditional LAN protocols utilizing other priorities. Details of PFC can be found in the IEEE 802.1Qbb standard, available at http://www.ieee802.org/1/pages/802.1bb.html, which is incorporated by reference herein.
  • In further embodiments, the VCS member switch also employ enhanced transmission selection mechanisms, which support allocation of bandwidth amongst different traffic classes. When the offered load in a traffic class does not use its allocated bandwidth, enhanced transmission selection will allow other traffic classes to use the available bandwidth. The bandwidth-allocation priorities can coexist with strict priorities. The VCS can prioritize traffic to provide different service characteristics to traffic classes. It is possible to share bandwidth between priorities carrying bursty loads rather than servicing them with strict priority while allowing strict priority for time-sensitive and management traffic requiring minimum latency. Also, when traffic at a given priority level does not use its allocation, it is possible to allow other priorities to use that bandwidth. Details of ETS can be found in the IEEE 802.1Qas standard, available at http://www.ieee802.org/1/pages/802.1az.html, which is incorporated by reference herein.
  • In some embodiments, the VCS member switches can also implement congestion notification mechanisms to facilitate source-oriented flow control. FIG. 30 illustrates an example of VCS congestion notification, in accordance with one embodiment of the present invention. In this example, assume a source host 3002 is transmitting frames to a destination host 3004 via a VCS 3000. Assume that congestion occurs at an intermediary switch 3006. As frames accumulate at switch 3006, its corresponding queue 3008 becomes full. A queue monitoring mechanism within switch 3006 is triggered when the content of queue 3008 passes a predetermined threshold. In response, switch 3006 can randomly select a frame in queue 3008, extract the source MAC address of the selected frame to construct a congestion notification frame, and send this notification frame to the source device corresponding to the extracted source MAC address (which is host 3002). Switch 3006 can perform the same action with multiple frames in queue 3008, so that if multiple sources are contributing to the congestion, these sources can all be notified.
  • Hybrid Trunking
  • In one embodiment, the trunked links between two neighboring VCS member switches can be further trunked to form a hybrid trunk. This way, the amount of link state maintenance traffic between the two switches can be minimized. FIG. 31 illustrates an exemplary hybrid trunk in accordance with one embodiment of the present invention. In this example, two link trunk groups 3106 and 3108 are formed between VCS member switches 3102 and 3104. Assuming that trunk groups 3106 and 3108 can be identified by both switches without using a link aggregation group (LAG) ID, it is possible to form a LAG 3110 based on trunks 3106 and 3108. In this case, trunks 3106 and 3108 can be regarded as logical links. LAG 3110 can also include other individual, physical links (not shown).
  • FIG. 32 illustrates an exemplary VCS member switch, in accordance with one embodiment of the present invention. In this example, the VCS member switch is a TRILL RBridge 3200 running special VCS software. RBridge 3200 includes a number of Ethernet communication ports 3201, which can transmit and receive Ethernet frames and/or TRILL encapsulated frames. Also included in RBridge 3200 is a packet processor 3202, a virtual FC switch management module 3204, a logical FC switch 3205, a VCS configuration database 3206, a traffic management module 3207, and a TRILL header generation module 3208.
  • During operation, packet processor 3202 extracts the source and destination MAC addresses of incoming frames, and attaches proper Ethernet or TRILL headers to outgoing frames. Virtual FC switch management module 3204 maintains the state of logical FC switch 3205, which is used to join other VCS switches using the FC switch fabric protocols. VCS configuration database 3206 maintains the configuration state of every switch within the VCS. TRILL header generation module 3208 is responsible for generating property TRILL headers for frames that are to be transmitted to other VCS member switches. Traffic management module 3207 facilitates the aforementioned traffic management functions, such as multi-pathing, flow control, congestion notification, and hybrid trunking.
  • VCS with Converged Network Extension
  • One goal of the embodiments of the present invention is to extend a VCS and TRILL network across data centers and meet the scalability requirements needed by the deployments. A converged network extension (CNE) device can be implemented in a two-box solution, wherein one box is capable of L2/L3/FCoE switching and is part of the VCS, and the other facilitates the WAN tunneling to transport Ethernet and/or FC traffic over WAN. The CNE device can also be implemented in a one-box solution, wherein a single piece of network equipment combines the functions of L2/L3/FCoE switching and WAN tunneling.
  • As described above, VCS as a layer-2 switch uses TRILL as its inter-switch connectivity and delivers a notion of single logical layer-2 switch. This single logical layer-2 switch delivers a transparent LAN service. All the edge ports of VCS support standard protocols and features like Link Aggregation Control Protocol (LACP), Link Layer Discovery Protocol (LLDP), VLANs, MAC learning, etc. VCS achieves a distributed MAC address database using Ethernet Name Service (eNS) and attempts to avoid flooding as much as possible. VCS also provides various intelligent services, such as virtual link aggregation group (vLAG), advance port profile management (APPM), End-to-End FCoE, Edge-Loop-Detection, etc.
  • In embodiments of the present invention, for the purpose of cross-data-center communication, each data center is represented as a single logical RBridge. This logical RBridge can be assigned a virtual RBridge ID or use the RBridge ID of the CNE device that performs the WAN tunneling.
  • FIG. 33 illustrates an exemplary network architecture which includes CNE devices for facilitate cross-data-center communications, in accordance with one embodiment of the present invention. In this example, two data centers 3344 and 3346 are coupled to a WAN 3326 via gateway routers 3324 and 3328, respectively. Data center 3344 includes a VCS 3316, which couples to a number of hosts, such as host 3301, via its member switches, such as switch 3310. Host 3301 includes two VMs 3302 and 3304, which are coupled to virtual switches 3306 and 3308 in a dual-homing configuration. In one embodiment, virtual switches 3306 and 3308 reside on two network interface cards on host 3301. Virtual switches 3306 and 3308 are coupled to VCS member switch 3310. Also included in VCS 3316 is a CNE device 3318. CNE device 3318 is configured to receive both Ethernet (or TRILL) traffic from member switch 3310 via an Ethernet (or TRILL) link 3312, and FC traffic via FC link 3314. Also coupled to CNE device 3318 is a target storage device 3320, and a cloned target storage device 3322 (denoted by dotted lines). CNE device 3318 maintains an FCIP tunnel to data center 3346 across WAN 3326 via gateway routers 3324 and 3328.
  • Similarly, data center 3346 includes a VCS 3342, which in turn includes a member switch 3332. Member switch 3332 is coupled to a host 3341, which includes VMs 3334 and 3336, both of which are coupled to virtual switches 3338 and 3340. Also included in VCS 3342 is a CNE device 3330. CNE device is coupled to member switch 3332 via an Ethernet (TRILL) link and an FC link. CNE device 3330 is also coupled to target storage device 3322 and a clone of target storage device 3320.
  • During operation, assume that VM 3302 needs to move from host 3301 to host 3341. Note that this movement is previously not possible, because virtual machines are visible only within the same layer-2 network domain. Once the layer-2 network domain is terminated by a layer-3 device, such as gateway router 3324, all the identifying information for a particular virtual machine (which is carried in layer-2 headers) is lost. However, in embodiments of the present invention, because CNE device extends the layer-2 domain from VCS 3316 to VCS 3342, the movement of VM 3302 from data center 3344 to data center 3346 is now possible.
  • When forwarding TRILL frames from data center 3344 to data center 3346, CNE device 3318 modifies the egress TRILL frames' header so that the destination RBridge identifier is the RBridge identifier assigned to data center 3346. CNE device 3318 then uses the FCIP tunnel to deliver these TRILL frames to CNE device 3330, which in turn forwards these TRILL frames to their respective layer-2 destinations.
  • TRILL Fabric Formation Across Data Centers
  • VCS uses FC control plane to automatically form a fabric and assign RBridge identifiers to each member switch. In one embodiment, the CNE architecture keeps the TRILL and SAN fabrics separate between data centers. From a TRILL point of view, each VCS (which corresponds to a respective data center) is represented as a single virtual RBrdige. In addition, the CNE device can be coupled to a VCS member switch with both a TRILL link and an FC link. The CNE device can join the VCS via a TRILL link. However, since the CNE devices keeps the TRILL VCS fabric and SAN (FC) fabric separate, the FC link between the CNE device and the member switch is configured for FC multi-fabric.
  • As illustrated in FIG. 34, a data center 3408 is coupled to a WAN via a gateway router 3410, and a data center 3420 is coupled to the WAN via a gateway router 3412. Data center 3408 includes a VCS 3406, which includes a member switch 3404. Also included in data center 3408 is a CNE device 3402. CNE device 3402 is coupled to VCS member switch 3404 via a TRILL link and an FC link. CNE device 3402 can join the VCS via the TRILL link. However, the FC link allows CNE device 3402 to maintain a separate FC fabric with VCS member switch 3404 to carry FC traffic. In one embodiment, the FC port on CNE device 3402 is an FC EX_port. The corresponding port on member switch 3404 is an FC E_port. The port on CNE device 3402 on the WAN side (coupling to gateway router 3410) is an FCIP VE_port. Data center 3420 has a similar configuration.
  • In one embodiment, each data center's VCS includes a node designated as the ROOT RBridge for multicast purposes. During the initial setup, the CNE devices in the VCSs exchange each VCS's ROOT RBridge identifier. In addition, the CNE devices also exchange each data center's RBridge identifier. Note that this RBridge identifier represents the entire data center. Information related to data-center RBridge identifiers is distributed as a static route to all the nodes in the local VCS.
  • Forwarding BUM Traffic Across Data Centers
  • FIG. 35A presents a diagram illustrating how CNE devices handle broadcast, unknown unicast, and multicast (BUM) traffic across data centers, in accordance with one embodiment of the present invention. In this example, two data centers, DC-1 and DC-2, are coupled to an IP WAN via core IP routers. The CNE device in DC-1 has an RBridge identifier of RB4, and the CNE device in DC-2 has an RBridge identifier of RB6. Furthermore, in the VCS in DC-1, a member switch RB1 is coupled to host A. In the VCS in DC-2, a member switch RB5 is coupled to host Z.
  • Assume that host A needs to send multicast traffic to host Z, and that host A already has the knowledge of host Z's MAC address. During operation, host A assembles an Ethernet frame 3502, which has host Z's MAC address (denoted as MAC-Z) as its destination address (DA), and host A's MAC address (denoted as MAC-A) as its source address (SA). Based on frame 3502, member switch RB1 assembles a TRILL frame 3503, whose TRILL header 3506 includes the RBridge identifier of data center DC-1's root RBridge (denoted as “DC1-ROOT”) as the destination RBridge, and RB1 as the source RBridge. (That is, within DC-1, the multicast traffic is distributed on the local multicast tree.) The outer Ethernet header 3504 of frame 3503 has CNE device RB4's MAC address (denoted as MAC-RB4) as the DA, and member switch RB1's MAC address (denoted as MAC-RB1) as the SA.
  • When frame 3503 reaches CNE device RB4, it further modifies the frame's TRILL header to produce frame 3505. CNE device RB4 replaces the destination RBridge identifier in the TRILL header 3510 with data center DC-2's root RBridge identifier DC2-ROOT. The source RBridge identifier is changed to data center DC-1's virtual RBridge identifier, DC1-RB (which allows data center DC-2 to learn data center DC-1's RBridge identifier). Outer Ethernet header 3508 has the core router's MAC address (MAC-RTR) as its DA, and CNE device RB4's MAC address (MAC-DC-1) as its SA.
  • Frame 3505 is subsequently transported across the IP WAN in an FCIP tunnel and reaches CNE device RB6. Correspondingly, CNE device RB6 updates the header to produce frame 3507. Frame 3507's TRILL header 3514 remains the same as frame 3505. The outer Ethernet header 3512 now has member switch RB5's MAC address, MAC-RB5, as its DA, and CNE device RB6's MAC address, MAC-RB6, as its SA. Once frame 3507 reaches member switch RB5, the TRILL header is removed, and the inner Ethernet frame is delivered to host Z.
  • In various embodiments, a CNE device can be configured to allow or disallow unknown unicast, broadcast (e.g., ARP), or multicast (e.g., IGMP snooped) traffic to cross data center boundaries. By having these options, one can limit the amount of BUM traffic across data centers. Note that all TRILL encapsulated BUM traffic between data centers can be sent with the remote data center's root RBrdige identifier. This translation is done at the terminating point of the FOP tunnel.
  • Additional mechanisms can be implemented to minimize BUM traffic across data centers. For instance, the TRILL ports between the CNE device and any VCS member switch can be configured to not participate in any of the VLAN MGIDs. In addition, the eNS on both VCSs can be configured to synchronize their learned MAC address database to minimize traffic with unknown MAC DA. (Note that in one embodiment, before the learned MAC address databases are synchronized in different VCSs, frames with unknown MAC DAs are flooded within the local data center only.)
  • To further minimize BUM traffic, broadcast traffic such as ARP traffic can be reduced by snooping ARP responses to build ARP databases on VCS member switches. The learned ARP databases are then exchanged and synchronized across different data centers using eNS. Proxy-based ARP is used to respond to all know ARP requests in a VCS. Furthermore, multicast traffic across data centers can be reduced by distributing the multicast group membership across data canters by sharing the IGMP snooping information via eNS.
  • Forwarding Unicast Traffic Across Data Centers
  • The process of forwarding unicast traffic between data centers is described as follows. During the FCIP tunnel formation, the logical RBridge identifiers representing data centers are exchanged. When a TRILL frame arrives at the entry node of the FOP tunnel, wherein the TRILL destination RBridge is set as the RBridge identifier of the remote data center, the source RBridge in the TRILL header is translated to the logical RBridge identifier assigned to the local data center. When the frame exits the FOP tunnel, the destination RBridge field in the TRILL header is set as the local (i.e., the destination) data center's virtual RBridge identifier. The MAC DA and VLAN ID in the inner Ethernet header is then used to look up the corresponding destination RBridge (i.e., the RBridge identifier of the member switch to which the destination host is attached, and the destination RBridge field in the TRILL header is updated accordingly.
  • In the destination data center, based on an ingress frame, all the VCS member switches learn the mapping between the MAC SA (in the inner Ethernet header of the frame) and the TRILL source RBridge (which is the virtual RBridge identifier assigned to the source data center). This allows future egress frames destined to that MAC address to be sent to the right remote data center. Note that since the RBridge identifier assigned to a given data center does not correspond to a physical RBridge, in one embodiment, a static route is used to map a remote data-center RBridge identifier to the local CNE device.
  • FIG. 35B presents a diagram illustrating how CNE devices handle unicast traffic across data centers, in accordance with one embodiment of the present invention. Assume that host A needs to send unicast traffic to host Z, and that host A already has the knowledge of host Z's MAC address. During operation, host A assembles an Ethernet frame 3502, which has host Z's MAC address (MAC-Z) as its DA, and host A's MAC address (MAC-A) as its SA. Based on frame 3502, member switch RB1 assembles a TRILL frame 3503, whose TRILL header 3509 includes the RBridge identifier of data center DC-2's virtual Rbridge (denoted as “DC2-RB”) as the destination RBridge, and RB1 as the source RBridge. The outer Ethernet header 3504 of frame 3503 has CNE device RB4's MAC address (MAC-RB4) as the DA, and member switch RB1's MAC address (MAC-RB1) as the SA.
  • When frame 3503 reaches CNE device RB4, it further modifies the frame's TRILL header to produce frame 3505. CNE device RB4 replaces the source RBridge identifier in the TRILL header 3511 with data center DC-1's virtual RBridge identifier DC1-RB (which allows data center DC-2 to learn data center DC-1's RBridge identifier). Outer Ethernet header 3508 has the core router's MAC address (MAC-RTR) as its DA, and CNE device RB4's MAC address (MAC-DC-1) as its SA.
  • Frame 3505 is subsequently transported across the IP WAN in an FCIP tunnel and reaches CNE device RB6. Correspondingly, CNE device RB6 updates the header to produce frame 3507. Frame 3507's TRILL header 3515 has an updated destination RBridge identifier, which is RB5, the VCS member switch in DC-2 that couples to host Z. The outer Ethernet header 3512 now has member switch RB5's MAC address, MAC-RB5, as its DA, and CNE device RB6's MAC address, MAC-RB6, as its SA. Once frame 3507 reaches member switch RB5, the TRILL header is removed, and the inner Ethernet frame is delivered to host Z.
  • Learning MAC Addresses and Distributing Configuration Information Across Data Centers
  • Flooding across data centers of frames with unknown MAC Das is one way for the data centers to learn the MAC address in another data center. All unknown SAs are learned as MACs behind an RBridge and it is no exception for the CNE device. In one embodiment, eNS can be used to distribute learned MAC address database, which reduces the amount of flooding across data centers.
  • In order to optimize flushes, even though MAC addresses are learned behind RBridges, the actual VCS edge port associated with a MAC address is present in the eNS MAC updates. However, the edge port IDs might no longer be unique across data-centers. To resolve this problem, all eNS updates across data centers will qualify the MAC entry with the data-center's RBridge identifier. This configuration allows propagation of port flushes across data centers.
  • In the architecture described herein, VCSs in different data-centers do not join each other; hence the distributed configurations are kept separate. However, in order to allow virtual machines to move across data-centers, there will be some configuration data that needs to be synchronized across data-centers. In one embodiment, a special module (in either software or hardware) is created for CNE purposes. This module is configured to retrieve the configuration information needed to facilitate moving of virtual machines across data centers and it is synchronized between two or more VCSs.
  • In one embodiment, the learned MAC address databases are distributed across data centers. Also, edge port state change notifications (SCNs) are also distributed across data centers. When a physical RBridge is going down, the SCN is converted to multiple port SCNs on the inter-data-center FCIP link.
  • Redundancy in CNE-Enabled VCSs
  • In order to protect the inter-data-center connectivity, a VCS can form a vLAG between two or more CNE devices. In this model, the vLAG RBridge identifier is used as the data-center RBridge identifier. The FCIP control plane is configured to be aware of this arrangement and exchange the vLAG RBridge identifers in such cases.
  • FIG. 14 illustrates an example where two CNE devices are used to construct a vLAG, in accordance with an embodiment of the present invention. In this example, a VCS 1400 includes two CNE devices 1406 and 1408. Both CNE devices 1406 and 1408 forms a vLAG 1410 which is coupled to a core IP router. vLAG 1410 is assigned a virtual RBridge identifier, which is also used as the data-center RBridge identifier for VCS 1400. Furthermore, vLAG 1410 can facilitate both ingress and egress load balancing (e.g., based on equal-cost multi-pathing (ECMP)) for any member switch within VCS 1400.
  • Routing Among FC and Ethernet Inter-Switch Links
  • Existing routing models assume every inter-switch link (ISL) can be a path to a reachable domain. However, if an FCIP-enabled VCS node joins the VCS via both FC and Ethernet ISLs, extra care needs to be taken to ensure that only FCoE storage traffic uses the FC links while the Ethernet ISLs are used for the LAN traffic. This can be achieved by allowing only FCoE VLAN traffic to use the FC links, while all non-FCoE traffic are allowed to use only the Ethernet ISLs.
  • CNE Device Structure
  • FIG. 37 illustrates an exemplary CNE device, in accordance with one embodiment of the present invention. In this example, a CNE device 3700 includes a number of TRILL ports 3702, FC ports 3704, and FCIP ports 3705. TRILL ports 3702 and FC ports 3704 are used to handle traffic within the local data center. FCIP ports 3705 are used to couple to WAN network equipment, such as IP routers. These ports are internally coupled to a packet processor 3706, which is responsible for updating a received frame's header(s).
  • Coupled to packet processor 3706 is an address management module 3708, which is responsible for maintaining the virtual data-center RBridge identifier, learning address information from remote data centers, and updating the source RBrdige field for outgoing frames to a remote data center. Also included in CNE device 3700 is an FCIP management module 3710, which maintains the FCIP tunnel and is responsible for exchanging configuration information via the FCIP tunnel, and a TRILL management module 3712, which manages the TRILL connection with the local VCS.
  • The methods and processes described herein can be embodied as code and/or data, which can be stored in a computer-readable non-transitory storage medium. When a computer system reads and executes the code and/or data stored on the computer-readable non-transitory storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the medium.
  • The methods and processes described herein can be executed by and/or included in hardware modules or apparatus. These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.
  • The foregoing descriptions of embodiments of the present invention have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit this disclosure. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. The scope of the present invention is defined by the appended claims.

Claims (18)

1-20. (canceled)
21. A switch, comprising:
a control module configured to:
provide a control plane with configuration capabilities;
join a network of interconnected switches via the control plane to become a member switch of the network of interconnected switches; and
control the network of interconnected switches as a single logical switch based on the control plane; and
a traffic management module configured to:
in response to determining congestion in a local queue, extract a source media access control (MAC) address of a packet in the local queue, wherein the source MAC address is encapsulated in an encapsulation header, which is used to transport packets among member switches within the network of interconnected; and
construct a congestion notification frame for a device corresponding to the extracted source MAC address.
22. The switch of claim 21,
wherein the traffic management module is further configured to associate incoming traffic with two outgoing ports corresponding to two respective equal-cost data paths to a common destination, thereby facilitating load balancing.
23. The switch of claim 21,
wherein the traffic management module is further configured to perform priority-based flow control on a respective link.
24. The switch of claim 21,
wherein the traffic management module is further configured to perform enhanced transmission selection for ingress traffic; and
wherein the enhanced transmission selection comprises enforcement of bandwidth allocation to a first class of traffic.
25. The switch of claim 21, wherein the traffic management module is further configured to, in response to a first class of traffic not using entire allocated bandwidth, allocate the unused bandwidth to a second class of traffic.
26. The switch of claim 21, further comprising a number of links grouped into a hybrid trunk, wherein the hybrid trunk includes one or more trunk groups.
27. The switch of claim 21,
wherein the one or more physical switches are allowed to be coupled in an arbitrary topology.
28. The switch of claim 21, wherein a packet received at the switch is encapsulated based on a protocol other than Ethernet.
29. The switch of claim 28, wherein the protocol is one or more of:
a Transparent Interconnection of Lots of Links (TRILL) protocol, wherein the packets are encapsulated in TRILL headers;
an Internet Protocol (IP), wherein the packets are encapsulated in IP headers; and
a multi-protocol label switching (MPLS) protocol, wherein the packets are encapsulated in MPLS headers.
30. The switch of claim 21, wherein a common identifier is associated with the network of interconnected switches.
31. A method, comprising:
providing a control plane with automatic configuration capabilities;
joining, by a switch, a network of interconnected switches via the control plane to become a member switch of the network of interconnected switches;
controlling the network of interconnected switches as a single logical switch based on the control plane;
in response to determining congestion in a local queue of the switch, extracting a source media access control (MAC) address of a packet in the local queue, wherein the source MAC address is encapsulated in an encapsulation header, which is used to transport packets among member switches within the network of interconnected switches; and
constructing a congestion notification frame for a device corresponding to the extracted source MAC address.
32. The method of claim 31, further comprising associating incoming traffic with two outgoing ports corresponding to two respective equal-cost data paths to a common destination, thereby facilitating load balancing.
33. The method of claim 31, further comprising performing enhanced transmission selection for ingress traffic;
wherein the enhanced transmission selection comprises enforcement of bandwidth allocation to a first class of traffic.
34. The method of claim 31, further comprising performing priority-based flow control on a respective link.
35. The method of claim 31, further comprising, in response to a first class of traffic not using entire allocated bandwidth, allocating the unused bandwidth to a second class of traffic.
36. The method of claim 31, further comprising grouping a number of links into a hybrid trunk, wherein the hybrid trunk includes one or more trunk groups.
37. A switch comprising:
a control module comprising:
a control plane means for providing a control plane with configuration capabilities;
a joining means for joining a network of interconnected switches via the control plane to become a member switch of the network of interconnected switches;
an control means for controlling the network of interconnected switches as a single logical switch based on the control plane; and
a traffic management means for, in response to determining congestion in a local queue of the switch, extracting a source media access control (MAC) address of a packet in the local queue, wherein the source MAC address is encapsulated in an encapsulation header, which is used to transport packets among member within the network of interconnected switches; and
a notification means for constructing a congestion notification frame for a device corresponding to the extracted source MAC address.
US14/912,585 2010-05-03 2011-05-03 Virtual cluster switching Abandoned US20170155599A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/912,585 US20170155599A1 (en) 2010-05-03 2011-05-03 Virtual cluster switching

Applications Claiming Priority (26)

Application Number Priority Date Filing Date Title
US33067810P 2010-05-03 2010-05-03
US33366610P 2010-05-11 2010-05-11
US33494510P 2010-05-14 2010-05-14
US34595310P 2010-05-18 2010-05-18
US34960310P 2010-05-28 2010-05-28
US35083810P 2010-06-02 2010-06-02
US35226410P 2010-06-07 2010-06-07
US35225510P 2010-06-07 2010-06-07
US35277610P 2010-06-08 2010-06-08
US35281910P 2010-06-08 2010-06-08
US38081810P 2010-09-08 2010-09-08
US38080710P 2010-09-08 2010-09-08
US38081910P 2010-09-08 2010-09-08
US38080310P 2010-09-08 2010-09-08
US13/042,259 US9461840B2 (en) 2010-06-02 2011-03-07 Port profile management for virtual cluster switching
US13/087,239 US8867552B2 (en) 2010-05-03 2011-04-14 Virtual cluster switching
US13/092,580 US9716672B2 (en) 2010-05-28 2011-04-22 Distributed configuration management for virtual cluster switching
US13/092,752 US9270486B2 (en) 2010-06-07 2011-04-22 Name services for virtual cluster switching
US13/092,877 US9231890B2 (en) 2010-06-08 2011-04-22 Traffic management for virtual cluster switching
US13/092,724 US9001824B2 (en) 2010-05-18 2011-04-22 Fabric formation for virtual cluster switching
US13/092,460 US9769016B2 (en) 2010-06-07 2011-04-22 Advanced link tracking for virtual cluster switching
US13/092,701 US8989186B2 (en) 2010-06-08 2011-04-22 Virtual port grouping for virtual cluster switching
US13/098,360 US8625616B2 (en) 2010-05-11 2011-04-29 Converged network extension
US13/098,490 US10673703B2 (en) 2010-05-03 2011-05-02 Fabric switching
PCT/US2011/034917 WO2011140028A1 (en) 2010-05-03 2011-05-03 Virtual cluster switching
US14/912,585 US20170155599A1 (en) 2010-05-03 2011-05-03 Virtual cluster switching

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US13/042,259 Continuation US9461840B2 (en) 2010-05-03 2011-03-07 Port profile management for virtual cluster switching

Publications (1)

Publication Number Publication Date
US20170155599A1 true US20170155599A1 (en) 2017-06-01

Family

ID=45064390

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/042,259 Active 2031-11-22 US9461840B2 (en) 2010-05-03 2011-03-07 Port profile management for virtual cluster switching
US14/912,585 Abandoned US20170155599A1 (en) 2010-05-03 2011-05-03 Virtual cluster switching

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US13/042,259 Active 2031-11-22 US9461840B2 (en) 2010-05-03 2011-03-07 Port profile management for virtual cluster switching

Country Status (1)

Country Link
US (2) US9461840B2 (en)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170034047A1 (en) * 2015-07-31 2017-02-02 Brocade Communications Systems, Inc. Graceful recovery of a multicast-enabled switch
US20180167111A1 (en) * 2016-03-03 2018-06-14 Andrew Wireless Systems Gmbh Hybrid ran/digital das repeater system with ethernet transport
US20190089637A1 (en) * 2017-09-21 2019-03-21 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
WO2019060003A1 (en) * 2017-09-21 2019-03-28 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
WO2019060004A1 (en) * 2017-09-21 2019-03-28 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
US10284457B2 (en) * 2016-07-12 2019-05-07 Dell Products, L.P. System and method for virtual link trunking
US10361912B2 (en) * 2014-06-30 2019-07-23 Huawei Technologies Co., Ltd. Traffic switching method and apparatus
US10594565B2 (en) 2014-12-19 2020-03-17 Hewlett Packard Enterprise Development Lp Multicast advertisement message for a network switch in a storage area network
US10630554B1 (en) 2018-10-29 2020-04-21 International Business Machines Corporation Input/output (I/O) performance of hosts through bi-directional bandwidth feedback optimization
US20200136997A1 (en) * 2018-10-26 2020-04-30 Cisco Technology, Inc. Managed midlay layers on a routed network
US10644935B2 (en) 2014-07-22 2020-05-05 Huawei Technologies Co., Ltd. Method for configuring fibre channel storage area network, and apparatus
US10693832B1 (en) * 2015-12-30 2020-06-23 Avago Technologies International Sales Pte. Limited Address resolution protocol operation in a fibre channel fabric
US10841375B2 (en) 2013-11-01 2020-11-17 Hewlett Packard Enterprise Development Lp Protocol agnostic storage access in a software defined network topology
US11240160B2 (en) 2018-12-28 2022-02-01 Alibaba Group Holding Limited Method, apparatus, and computer-readable storage medium for network control
US20220070625A1 (en) * 2018-12-19 2022-03-03 Telefonaktiebolaget Lm Ericsson (Publ) Reporting of multicast mac addresses
US11368395B2 (en) * 2019-03-13 2022-06-21 Suzhou Kuhan Information Technologies Co., Ltd. System, method and apparatus for storage controller having multiple heterogeneous network interface ports
US20220247631A1 (en) * 2019-05-28 2022-08-04 Nippon Telegraph And Telephone Corporation Network management apparatus and method
US11431652B2 (en) * 2020-07-10 2022-08-30 Dell Products L.P. Automated multi-fabric link aggregation system
US11552906B2 (en) 2016-10-21 2023-01-10 Avago Technologies International Sales Pte. Limited Hard zoning of virtual local area networks in a fibre channel fabric
US11582290B2 (en) * 2020-01-23 2023-02-14 Hewlett Packard Enterprise Development Lp Switch event ordering
US20230353506A1 (en) * 2022-04-29 2023-11-02 Hewlett Packard Enterprise Development Lp Remote management of a switch stack

Families Citing this family (150)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8566946B1 (en) * 2006-04-20 2013-10-22 Fireeye, Inc. Malware containment on connection
US9716672B2 (en) 2010-05-28 2017-07-25 Brocade Communications Systems, Inc. Distributed configuration management for virtual cluster switching
US8989186B2 (en) 2010-06-08 2015-03-24 Brocade Communication Systems, Inc. Virtual port grouping for virtual cluster switching
US9461840B2 (en) 2010-06-02 2016-10-04 Brocade Communications Systems, Inc. Port profile management for virtual cluster switching
US8867552B2 (en) 2010-05-03 2014-10-21 Brocade Communications Systems, Inc. Virtual cluster switching
US9001824B2 (en) 2010-05-18 2015-04-07 Brocade Communication Systems, Inc. Fabric formation for virtual cluster switching
US9270486B2 (en) 2010-06-07 2016-02-23 Brocade Communications Systems, Inc. Name services for virtual cluster switching
US9769016B2 (en) 2010-06-07 2017-09-19 Brocade Communications Systems, Inc. Advanced link tracking for virtual cluster switching
US9608833B2 (en) 2010-06-08 2017-03-28 Brocade Communications Systems, Inc. Supporting multiple multicast trees in trill networks
US8446914B2 (en) 2010-06-08 2013-05-21 Brocade Communications Systems, Inc. Method and system for link aggregation across multiple switches
US9806906B2 (en) 2010-06-08 2017-10-31 Brocade Communications Systems, Inc. Flooding packets on a per-virtual-network basis
US9628293B2 (en) 2010-06-08 2017-04-18 Brocade Communications Systems, Inc. Network layer multicasting in trill networks
US9246703B2 (en) 2010-06-08 2016-01-26 Brocade Communications Systems, Inc. Remote port mirroring
US9680750B2 (en) 2010-07-06 2017-06-13 Nicira, Inc. Use of tunnels to hide network addresses
US8958292B2 (en) 2010-07-06 2015-02-17 Nicira, Inc. Network control apparatus and method with port security controls
US8964528B2 (en) 2010-07-06 2015-02-24 Nicira, Inc. Method and apparatus for robust packet distribution among hierarchical managed switching elements
US9807031B2 (en) 2010-07-16 2017-10-31 Brocade Communications Systems, Inc. System and method for network configuration
US8954962B2 (en) * 2010-09-22 2015-02-10 Juniper Networks, Inc. Automatically reconfiguring physical switches to be in synchronization with changes made to associated virtual system
US9787608B2 (en) * 2010-10-19 2017-10-10 International Business Machines Corporation Unified fabric port
US9608939B2 (en) 2010-12-22 2017-03-28 Juniper Networks, Inc. Methods and apparatus to reduce forwarding state on an FCoE-to-FC gateway using port-specific MAC addresses
US9031072B2 (en) * 2010-12-22 2015-05-12 Juniper Networks, Inc. Methods and apparatus to route fibre channel frames using reduced forwarding state on an FCOE-to-FC gateway
JP5500270B2 (en) * 2011-01-07 2014-05-21 富士通株式会社 Profile processing program, data relay apparatus, and profile control method
US9270572B2 (en) 2011-05-02 2016-02-23 Brocade Communications Systems Inc. Layer-3 support in TRILL networks
US8526815B2 (en) * 2011-05-19 2013-09-03 Pmc-Sierra Israel Ltd. Dynamic bandwidth allocation for congestion management in PON channel aggregation
US9363207B2 (en) * 2011-06-24 2016-06-07 Cisco Technology, Inc. Private virtual local area network isolation
US9401861B2 (en) 2011-06-28 2016-07-26 Brocade Communications Systems, Inc. Scalable MAC address distribution in an Ethernet fabric switch
US8948056B2 (en) 2011-06-28 2015-02-03 Brocade Communication Systems, Inc. Spanning-tree based loop detection for an ethernet fabric switch
US20130024553A1 (en) * 2011-07-18 2013-01-24 Cisco Technology, Inc. Location independent dynamic IP address assignment
WO2012159339A1 (en) * 2011-07-21 2012-11-29 华为技术有限公司 Interface register method and device for network device to join cluster system
US10091028B2 (en) 2011-08-17 2018-10-02 Nicira, Inc. Hierarchical controller clusters for interconnecting two or more logical datapath sets
US9736085B2 (en) 2011-08-29 2017-08-15 Brocade Communications Systems, Inc. End-to end lossless Ethernet in Ethernet fabric
US20130074066A1 (en) * 2011-09-21 2013-03-21 Cisco Technology, Inc. Portable Port Profiles for Virtual Machines in a Virtualized Data Center
US9419842B1 (en) * 2011-10-04 2016-08-16 Amazon Technologies, Inc. Dynamic network device configuration
US9699117B2 (en) 2011-11-08 2017-07-04 Brocade Communications Systems, Inc. Integrated fibre channel support in an ethernet fabric switch
US9450870B2 (en) 2011-11-10 2016-09-20 Brocade Communications Systems, Inc. System and method for flow management in software-defined networks
CN103947160B (en) * 2011-12-07 2017-04-12 华为技术有限公司 Method to carry FCOE frames over a TRILL based network
US8995272B2 (en) 2012-01-26 2015-03-31 Brocade Communication Systems, Inc. Link aggregation in software-defined networks
US9742693B2 (en) 2012-02-27 2017-08-22 Brocade Communications Systems, Inc. Dynamic service insertion in a fabric switch
US9154416B2 (en) 2012-03-22 2015-10-06 Brocade Communications Systems, Inc. Overlay tunnel in a fabric switch
US9374301B2 (en) 2012-05-18 2016-06-21 Brocade Communications Systems, Inc. Network feedback in software-defined networks
US10277464B2 (en) 2012-05-22 2019-04-30 Arris Enterprises Llc Client auto-configuration in a multi-switch link aggregation
EP2853066B1 (en) 2012-05-23 2017-02-22 Brocade Communications Systems, Inc. Layer-3 overlay gateways
US9450859B2 (en) * 2012-06-15 2016-09-20 Citrix Systems, Inc. Systems and methods for deriving unique MAC address for a cluster
US9602430B2 (en) 2012-08-21 2017-03-21 Brocade Communications Systems, Inc. Global VLANs for fabric switches
US9306768B2 (en) * 2012-11-07 2016-04-05 Cisco Technology, Inc. System and method for propagating virtualization awareness in a network environment
US9401872B2 (en) 2012-11-16 2016-07-26 Brocade Communications Systems, Inc. Virtual link aggregations across multiple fabric switches
CN103856397B (en) 2012-12-07 2018-08-14 中兴通讯股份有限公司 Multicast forward method and device, routing bridge in transparent interconnection of lots of links interference networks
US8953624B2 (en) * 2012-12-20 2015-02-10 Cisco Technology, Inc. Intelligent host route distribution for low latency forwarding and ubiquitous virtual machine mobility in interconnected data centers
US9413691B2 (en) 2013-01-11 2016-08-09 Brocade Communications Systems, Inc. MAC address synchronization in a fabric switch
US9548926B2 (en) 2013-01-11 2017-01-17 Brocade Communications Systems, Inc. Multicast traffic load balancing over virtual link aggregation
US9350680B2 (en) 2013-01-11 2016-05-24 Brocade Communications Systems, Inc. Protection switching over a virtual link aggregation
CN103460675B (en) 2013-01-14 2016-09-28 华为技术有限公司 Cluster and retransmission method
US9565113B2 (en) 2013-01-15 2017-02-07 Brocade Communications Systems, Inc. Adaptive link aggregation and virtual link aggregation
US9565099B2 (en) 2013-03-01 2017-02-07 Brocade Communications Systems, Inc. Spanning tree in fabric switches
US9571338B2 (en) 2013-03-14 2017-02-14 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Scalable distributed control plane for network switching systems
WO2014145750A1 (en) 2013-03-15 2014-09-18 Brocade Communications Systems, Inc. Scalable gateways for a fabric switch
US9215129B2 (en) * 2013-04-11 2015-12-15 International Business Machines Corporation Automatically constructing protection scope in a virtual infrastructure
US9432215B2 (en) 2013-05-21 2016-08-30 Nicira, Inc. Hierarchical network managers
US9565028B2 (en) 2013-06-10 2017-02-07 Brocade Communications Systems, Inc. Ingress switch multicast distribution in a fabric switch
US9699001B2 (en) * 2013-06-10 2017-07-04 Brocade Communications Systems, Inc. Scalable and segregated network virtualization
US9860116B1 (en) * 2013-06-18 2018-01-02 Cisco Technology, Inc. Physical network orchestration for data centers
US9571386B2 (en) 2013-07-08 2017-02-14 Nicira, Inc. Hybrid packet processing
US9282019B2 (en) 2013-07-12 2016-03-08 Nicira, Inc. Tracing logical network packets through physical network
US9344349B2 (en) 2013-07-12 2016-05-17 Nicira, Inc. Tracing network packets by a cluster of network controllers
US9407580B2 (en) 2013-07-12 2016-08-02 Nicira, Inc. Maintaining data stored with a packet
US9215147B2 (en) * 2013-07-15 2015-12-15 Cisco Technology, Inc. Flexible and scalable monitoring in a TRILL network
US9806949B2 (en) 2013-09-06 2017-10-31 Brocade Communications Systems, Inc. Transparent interconnection of Ethernet fabric switches
US9674087B2 (en) 2013-09-15 2017-06-06 Nicira, Inc. Performing a multi-stage lookup to classify packets
US9602398B2 (en) 2013-09-15 2017-03-21 Nicira, Inc. Dynamically generating flows with wildcard fields
US9350607B2 (en) 2013-09-25 2016-05-24 International Business Machines Corporation Scalable network configuration with consistent updates in software defined networks
US9912612B2 (en) 2013-10-28 2018-03-06 Brocade Communications Systems LLC Extended ethernet fabric switches
US9112794B2 (en) 2013-11-05 2015-08-18 International Business Machines Corporation Dynamic multipath forwarding in software defined data center networks
US9634948B2 (en) 2013-11-07 2017-04-25 International Business Machines Corporation Management of addresses in virtual machines
US10193771B2 (en) 2013-12-09 2019-01-29 Nicira, Inc. Detecting and handling elephant flows
US9967199B2 (en) 2013-12-09 2018-05-08 Nicira, Inc. Inspecting operations of a machine to detect elephant flows
US9996467B2 (en) 2013-12-13 2018-06-12 Nicira, Inc. Dynamically adjusting the number of flows allowed in a flow table cache
US9569368B2 (en) 2013-12-13 2017-02-14 Nicira, Inc. Installing and managing flows in a flow table cache
US9548873B2 (en) 2014-02-10 2017-01-17 Brocade Communications Systems, Inc. Virtual extensible LAN tunnel keepalives
US10581758B2 (en) 2014-03-19 2020-03-03 Avago Technologies International Sales Pte. Limited Distributed hot standby links for vLAG
US10476698B2 (en) 2014-03-20 2019-11-12 Avago Technologies International Sales Pte. Limited Redundent virtual link aggregation group
US10193806B2 (en) 2014-03-31 2019-01-29 Nicira, Inc. Performing a finishing operation to improve the quality of a resulting hash
US9686200B2 (en) 2014-03-31 2017-06-20 Nicira, Inc. Flow cache hierarchy
US9385954B2 (en) 2014-03-31 2016-07-05 Nicira, Inc. Hashing techniques for use in a network environment
US10523595B2 (en) * 2014-04-02 2019-12-31 Comcast Cable Communications, Llc Application port management
US9524181B2 (en) * 2014-04-11 2016-12-20 Vmware, Inc. Virtual switch and virtual switch port management for VM availability
US10063473B2 (en) 2014-04-30 2018-08-28 Brocade Communications Systems LLC Method and system for facilitating switch virtualization in a network of interconnected switches
US9800471B2 (en) 2014-05-13 2017-10-24 Brocade Communications Systems, Inc. Network extension groups of global VLANs in a fabric switch
EP3143733B1 (en) * 2014-05-13 2018-12-05 Telefonaktiebolaget LM Ericsson (publ) Virtual flow network in a cloud environment
US9742881B2 (en) 2014-06-30 2017-08-22 Nicira, Inc. Network virtualization using just-in-time distributed capability for classification encoding
US10616108B2 (en) 2014-07-29 2020-04-07 Avago Technologies International Sales Pte. Limited Scalable MAC address virtualization
US9544219B2 (en) 2014-07-31 2017-01-10 Brocade Communications Systems, Inc. Global VLAN services
US9807007B2 (en) 2014-08-11 2017-10-31 Brocade Communications Systems, Inc. Progressive MAC address learning
US10091294B2 (en) 2014-08-13 2018-10-02 Vmware, Inc. Networking component management in host computing systems in a virtual computing environment
US9893995B1 (en) 2014-09-25 2018-02-13 Cisco Technology, Inc. Systems, methods, and apparatus for implementing agents in service appliances
US11178051B2 (en) 2014-09-30 2021-11-16 Vmware, Inc. Packet key parser for flow-based forwarding elements
US9524173B2 (en) 2014-10-09 2016-12-20 Brocade Communications Systems, Inc. Fast reboot for a switch
US9699029B2 (en) 2014-10-10 2017-07-04 Brocade Communications Systems, Inc. Distributed configuration management in a switch group
US10469342B2 (en) 2014-10-10 2019-11-05 Nicira, Inc. Logical network traffic analysis
US9626255B2 (en) 2014-12-31 2017-04-18 Brocade Communications Systems, Inc. Online restoration of a switch snapshot
US10601647B2 (en) * 2014-12-31 2020-03-24 Dell Products L.P. Network configuration system
US9628407B2 (en) 2014-12-31 2017-04-18 Brocade Communications Systems, Inc. Multiple software versions in a switch group
US10003552B2 (en) 2015-01-05 2018-06-19 Brocade Communications Systems, Llc. Distributed bidirectional forwarding detection protocol (D-BFD) for cluster of interconnected switches
US9942097B2 (en) 2015-01-05 2018-04-10 Brocade Communications Systems LLC Power management in a network of interconnected switches
US10038592B2 (en) 2015-03-17 2018-07-31 Brocade Communications Systems LLC Identifier assignment to a new switch in a switch group
US9807005B2 (en) 2015-03-17 2017-10-31 Brocade Communications Systems, Inc. Multi-fabric manager
US10579406B2 (en) 2015-04-08 2020-03-03 Avago Technologies International Sales Pte. Limited Dynamic orchestration of overlay tunnels
US10243848B2 (en) 2015-06-27 2019-03-26 Nicira, Inc. Provisioning logical entities in a multi-datacenter environment
US10395029B1 (en) 2015-06-30 2019-08-27 Fireeye, Inc. Virtual system and method with threat protection
US11113086B1 (en) 2015-06-30 2021-09-07 Fireeye, Inc. Virtual system and method for securing external network connectivity
US10642753B1 (en) 2015-06-30 2020-05-05 Fireeye, Inc. System and method for protecting a software component running in virtual machine using a virtualization layer
US10726127B1 (en) 2015-06-30 2020-07-28 Fireeye, Inc. System and method for protecting a software component running in a virtual machine through virtual interrupts by the virtualization layer
US10216927B1 (en) 2015-06-30 2019-02-26 Fireeye, Inc. System and method for protecting memory pages associated with a process using a virtualization layer
US10171303B2 (en) 2015-09-16 2019-01-01 Avago Technologies International Sales Pte. Limited IP-based interconnection of switches with a logical chassis
US10033759B1 (en) 2015-09-28 2018-07-24 Fireeye, Inc. System and method of threat detection under hypervisor control
US9912614B2 (en) 2015-12-07 2018-03-06 Brocade Communications Systems LLC Interconnection of switches based on hierarchical overlay tunneling
US10171507B2 (en) * 2016-05-19 2019-01-01 Cisco Technology, Inc. Microsegmentation in heterogeneous software defined networking environments
US10382335B2 (en) 2016-10-21 2019-08-13 International Business Machines Corporation MAC learning in a multiple virtual switch environment
US10237090B2 (en) 2016-10-28 2019-03-19 Avago Technologies International Sales Pte. Limited Rule-based network identifier mapping
US11089066B2 (en) 2016-12-09 2021-08-10 Nutanix, Inc. System and method for dynamic medium access control (MAC) relating to a virtualization environment
US10356131B2 (en) * 2016-12-09 2019-07-16 Nutanix, Inc. Service chaining for multiple uplink bridges
US10951523B2 (en) * 2017-01-09 2021-03-16 Marvell Asia Pte, Ltd. Port extender with local switching
US10785118B2 (en) * 2017-02-23 2020-09-22 Dell Products L.P. Systems and methods for network topology validation
US10425287B2 (en) 2017-02-23 2019-09-24 Dell Products L.P. Systems and methods for network topology discovery
US10805239B2 (en) 2017-03-07 2020-10-13 Nicira, Inc. Visualization of path between logical network endpoints
US10193812B2 (en) * 2017-03-31 2019-01-29 Juniper Networks, Inc. Multicast load balancing in multihoming EVPN networks
US10382332B2 (en) * 2017-05-12 2019-08-13 Juniper Networks, Inc. Route signaling and convergence in EVPN of port extenders
US10608887B2 (en) 2017-10-06 2020-03-31 Nicira, Inc. Using packet tracing tool to automatically execute packet capture operations
US10348683B2 (en) * 2017-11-02 2019-07-09 Nicira Inc. Network packet filtering via media access control (MAC) address learning
CN111953508B (en) * 2019-05-17 2023-05-26 阿里巴巴集团控股有限公司 Equipment control method and device, switch and electronic equipment
US11283699B2 (en) 2020-01-17 2022-03-22 Vmware, Inc. Practical overlay network latency measurement in datacenter
US11777793B2 (en) 2020-04-06 2023-10-03 Vmware, Inc. Location criteria for security groups
US11303557B2 (en) 2020-04-06 2022-04-12 Vmware, Inc. Tunnel endpoint group records for inter-datacenter traffic
US11088902B1 (en) 2020-04-06 2021-08-10 Vmware, Inc. Synchronization of logical network state between global and local managers
US11088919B1 (en) 2020-04-06 2021-08-10 Vmware, Inc. Data structure for defining multi-site logical network
US11799726B2 (en) 2020-04-06 2023-10-24 Vmware, Inc. Multi-site security groups
US11163709B1 (en) 2020-07-10 2021-11-02 Dell Products L.P. Port configuration migration system
US11196628B1 (en) 2020-07-29 2021-12-07 Vmware, Inc. Monitoring container clusters
US11570090B2 (en) 2020-07-29 2023-01-31 Vmware, Inc. Flow tracing operation in container cluster
US11558426B2 (en) 2020-07-29 2023-01-17 Vmware, Inc. Connection tracking for container cluster
US11558349B2 (en) * 2020-08-10 2023-01-17 Arista Networks, Inc. MAC mobility for 802.1x addresses for virtual machines
US11509627B2 (en) * 2020-08-10 2022-11-22 Arista Networks, Inc. MAC mobility for 802.1x addresses for physical machines
US11601474B2 (en) 2020-09-28 2023-03-07 Vmware, Inc. Network virtualization infrastructure with divided user responsibilities
US11736436B2 (en) 2020-12-31 2023-08-22 Vmware, Inc. Identifying routes with indirect addressing in a datacenter
US11336533B1 (en) 2021-01-08 2022-05-17 Vmware, Inc. Network visualization of correlations between logical elements and associated physical elements
US11687210B2 (en) 2021-07-05 2023-06-27 Vmware, Inc. Criteria-based expansion of group nodes in a network topology visualization
US11711278B2 (en) 2021-07-24 2023-07-25 Vmware, Inc. Visualization of flow trace operation across multiple sites
US11706109B2 (en) 2021-09-17 2023-07-18 Vmware, Inc. Performance of traffic monitoring actions
CA214944S (en) * 2021-10-22 2024-06-10 Banner Eng Communication hub
US12107722B2 (en) 2022-07-20 2024-10-01 VMware LLC Sharing network manager between multiple tenants
CN115733759A (en) * 2022-09-16 2023-03-03 中国航空无线电电子研究所 Simulation model system for TTFC message scheduling

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050195813A1 (en) * 2004-02-23 2005-09-08 Sinett Corporation Unified architecture for wired and wireless networks
US20080080517A1 (en) * 2006-09-28 2008-04-03 At & T Corp. System and method for forwarding traffic data in an MPLS VPN
US20090052326A1 (en) * 2007-08-21 2009-02-26 Cisco Technology, Inc., A Corporation Of California Backward congestion notification
US20090092043A1 (en) * 2007-10-03 2009-04-09 Nortel Networks Limited Providing an abstraction layer in a cluster switch that includes plural switches
US20090116381A1 (en) * 2007-11-07 2009-05-07 Brocade Communications Systems, Inc. Method and system for congestion management in a fibre channel network
US20100103939A1 (en) * 2008-10-24 2010-04-29 Carlson Scott M Determining the configuration of an ethernet fabric
US7787480B1 (en) * 2009-03-04 2010-08-31 Juniper Networks, Inc. Routing frames in a trill network using service VLAN identifiers

Family Cites Families (333)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5309433A (en) 1992-06-18 1994-05-03 International Business Machines Corp. Methods and apparatus for routing packets in packet transmission networks
US5390173A (en) 1992-10-22 1995-02-14 Digital Equipment Corporation Packet format in hub for packet data communications system
US5802278A (en) 1995-05-10 1998-09-01 3Com Corporation Bridge/router architecture for high performance scalable networking
US5983278A (en) 1996-04-19 1999-11-09 Lucent Technologies Inc. Low-loss, fair bandwidth allocation flow control in a packet switch
US6085238A (en) 1996-04-23 2000-07-04 Matsushita Electric Works, Ltd. Virtual LAN system
US5878232A (en) 1996-12-27 1999-03-02 Compaq Computer Corporation Dynamic reconfiguration of network device's virtual LANs using the root identifiers and root ports determined by a spanning tree procedure
US6041042A (en) 1997-05-27 2000-03-21 Cabletron Systems, Inc. Remote port mirroring system and method thereof
US5959968A (en) 1997-07-30 1999-09-28 Cisco Systems, Inc. Port aggregation protocol
US6185214B1 (en) * 1997-09-11 2001-02-06 3Com Corporation Use of code vectors for frame forwarding in a bridge/router
US7430164B2 (en) 1998-05-04 2008-09-30 Hewlett-Packard Development Company, L.P. Path recovery on failure in load balancing switch protocols
US5973278A (en) 1998-05-07 1999-10-26 Eaton Corporation Snap acting charge/discharge and open/closed indicators displaying states of electrical switching apparatus
WO2000003522A1 (en) 1998-07-08 2000-01-20 Broadcom Corporation A method of sending packets between trunk ports of network switches
IL125272A0 (en) 1998-07-08 1999-03-12 Galileo Technology Ltd Vlan protocol
US6185241B1 (en) 1998-10-29 2001-02-06 Xerox Corporation Metal spatial filter to enhance model reflectivity in a vertical cavity surface emitting laser
US6438106B1 (en) 1998-12-22 2002-08-20 Nortel Networks Limited Inter-class schedulers utilizing statistical priority guaranteed queuing and generic cell-rate algorithm priority guaranteed queuing
US6771610B1 (en) 1999-01-19 2004-08-03 3Com Corporation Spanning tree with protocol for bypassing port state transition timers
US6542266B1 (en) * 1999-06-24 2003-04-01 Qwest Communications International Inc. System and method for providing broadband data service
JP4148605B2 (en) * 1999-08-06 2008-09-10 富士通株式会社 Network system and server
US6498781B1 (en) 1999-08-13 2002-12-24 International Business Machines Corporation Self-tuning link aggregation system
ES2277854T5 (en) 1999-08-25 2011-02-04 Allergan, Inc. ACTIVABLE RECOMBINANT NEUROTOXINS.
US7061877B1 (en) 1999-09-10 2006-06-13 Georgia Tech Reseach Corporation System and method for providing high speed wireless media access
JP2003525000A (en) 2000-02-22 2003-08-19 トップ レイヤー ネットワークス,インク. Data flow mirror processing system and method in network switch
US7197564B1 (en) 2000-04-07 2007-03-27 Hewlett-Packard Development Company, L.P. Adaptive admission control system for a server application system
JP2001313670A (en) 2000-04-28 2001-11-09 Oki Electric Ind Co Ltd Method for managing network
WO2001086886A2 (en) 2000-05-11 2001-11-15 Firemedia Communications (Israel) Ltd. Three-dimensional switch providing packet routing between multiple multimedia buses
JP4168574B2 (en) 2000-06-02 2008-10-22 株式会社日立製作所 Packet transfer apparatus, packet transfer control method, and packet transfer apparatus setting method
AU7170301A (en) 2000-06-29 2002-01-14 Cachestream Corp Virtual multicasting
US7924837B1 (en) 2000-07-31 2011-04-12 Avaya Communication Israel Ltd. IP multicast in VLAN environment
US6633761B1 (en) 2000-08-11 2003-10-14 Reefedge, Inc. Enabling seamless user mobility in a short-range wireless networking environment
US8619793B2 (en) 2000-08-21 2013-12-31 Rockstar Consortium Us Lp Dynamic assignment of traffic classes to a priority queue in a packet forwarding device
CA2355473A1 (en) 2000-09-29 2002-03-29 Linghsiao Wang Buffer management for support of quality-of-service guarantees and data flow control in data switching
US6937576B1 (en) 2000-10-17 2005-08-30 Cisco Technology, Inc. Multiple instance spanning tree protocol
JP2002135410A (en) 2000-10-26 2002-05-10 Kddi Research & Development Laboratories Inc Access network system
US6957269B2 (en) 2001-01-03 2005-10-18 Advanced Micro Devices, Inc. Method and apparatus for performing priority-based flow control
US6912592B2 (en) * 2001-01-05 2005-06-28 Extreme Networks, Inc. Method and system of aggregate multiple VLANs in a metropolitan area network
WO2002061510A2 (en) 2001-01-31 2002-08-08 Lancope, Inc. Network port profiling
US7016352B1 (en) 2001-03-23 2006-03-21 Advanced Micro Devices, Inc. Address modification within a switching device in a packet-switched network
US7450595B1 (en) * 2001-05-01 2008-11-11 At&T Corp. Method and system for managing multiple networks over a set of ports
US7102996B1 (en) 2001-05-24 2006-09-05 F5 Networks, Inc. Method and system for scaling network traffic managers
US6956824B2 (en) 2001-06-14 2005-10-18 Tropic Networks Inc. Extension of link aggregation protocols over the network
US20040001433A1 (en) 2001-07-18 2004-01-01 Gram Charles Andrew Interactive control of network devices
US7382787B1 (en) 2001-07-30 2008-06-03 Cisco Technology, Inc. Packet routing and switching device
US20040165596A1 (en) 2001-08-01 2004-08-26 Garcia Javier Romero Apparatus and method for flow scheduling based on priorities in a mobile network
JP2003069573A (en) * 2001-08-23 2003-03-07 Allied Tereshisu Kk System and method for managing network equipment using information recording medium
US7173934B2 (en) 2001-09-10 2007-02-06 Nortel Networks Limited System, device, and method for improving communication network reliability using trunk splitting
US20030084219A1 (en) 2001-10-26 2003-05-01 Maxxan Systems, Inc. System, apparatus and method for address forwarding for a computer network
US20070094465A1 (en) 2001-12-26 2007-04-26 Cisco Technology, Inc., A Corporation Of California Mirroring mechanisms for storage area networks and network based virtualization
US20030123393A1 (en) 2002-01-03 2003-07-03 Feuerstraeter Mark T. Method and apparatus for priority based flow control in an ethernet architecture
JPWO2003071749A1 (en) 2002-02-20 2005-06-16 三菱電機株式会社 Mobile network
US7688960B1 (en) 2002-02-26 2010-03-30 Sprint Communications Company L.P. Method and system for separating business and device logic in a computing network system
US7092943B2 (en) 2002-03-01 2006-08-15 Enterasys Networks, Inc. Location based data
US20030174706A1 (en) 2002-03-15 2003-09-18 Broadcom Corporation Fastpath implementation for transparent local area network (LAN) services over multiprotocol label switching (MPLS)
US7315545B1 (en) 2002-03-29 2008-01-01 Nortel Networks Limited Method and apparatus to support differential internet data packet treatment in a base station controller
TW550902B (en) * 2002-04-03 2003-09-01 Accton Technology Corp Method of setting network configuration and device and system thereof
US7209435B1 (en) 2002-04-16 2007-04-24 Foundry Networks, Inc. System and method for providing network route redundancy across Layer 2 devices
US20090279558A1 (en) 2002-05-06 2009-11-12 Ian Edward Davis Network routing apparatus for enhanced efficiency and monitoring capability
US7206288B2 (en) 2002-06-12 2007-04-17 Cisco Technology, Inc. Methods and apparatus for characterizing a route in fibre channel fabric
US7330897B2 (en) 2002-07-09 2008-02-12 International Business Machines Corporation Methods and apparatus for storage area network component registration
US7453888B2 (en) 2002-08-27 2008-11-18 Alcatel Lucent Stackable virtual local area network provisioning in bridged networks
US7316031B2 (en) 2002-09-06 2008-01-01 Capital One Financial Corporation System and method for remotely monitoring wireless networks
JP4444834B2 (en) * 2002-10-04 2010-03-31 テレフオンアクチーボラゲット エル エム エリクソン(パブル) Isolating hosts connected to the access network
US7647427B1 (en) 2002-10-18 2010-01-12 Foundry Networks, Inc. Redundancy support for network address translation (NAT)
KR20050070119A (en) 2002-11-08 2005-07-05 코닌클리케 필립스 일렉트로닉스 엔.브이. Method and apparatus allowing remote access in data networks
US7424014B2 (en) 2002-11-12 2008-09-09 Cisco Technology, Inc. System and method for local packet transport services within distributed routers
US7397794B1 (en) * 2002-11-21 2008-07-08 Juniper Networks, Inc. Systems and methods for implementing virtual switch planes in a physical switch fabric
KR100480366B1 (en) * 2002-12-24 2005-03-31 한국전자통신연구원 A system for VLAN configuration of E-PON and method thereof, its program stored recording medium
US7417950B2 (en) 2003-02-03 2008-08-26 Ciena Corporation Method and apparatus for performing data flow ingress/egress admission control in a provider network
JP4256693B2 (en) * 2003-02-18 2009-04-22 株式会社日立製作所 Computer system, I / O device, and virtual sharing method of I / O device
US20040165595A1 (en) 2003-02-25 2004-08-26 At&T Corp. Discovery and integrity testing method in an ethernet domain
US7411973B2 (en) 2003-03-11 2008-08-12 Broadcom Corporation System and method for interfacing with a management system
US7486674B2 (en) 2003-04-28 2009-02-03 Alcatel-Lucent Usa Inc. Data mirroring in a service
US7370346B2 (en) 2003-04-29 2008-05-06 Hewlett-Packard Development Company, L.P. Method and apparatus for access security services
US7561590B1 (en) 2003-05-05 2009-07-14 Marvell International Ltd. Network switch having virtual input queues for flow control
US7516487B1 (en) * 2003-05-21 2009-04-07 Foundry Networks, Inc. System and method for source IP anti-spoofing security
WO2004112327A1 (en) 2003-06-11 2004-12-23 Nec Corporation Router and network connecting method
US7480258B1 (en) 2003-07-03 2009-01-20 Cisco Technology, Inc. Cross stack rapid transition protocol
US7463579B2 (en) 2003-07-11 2008-12-09 Nortel Networks Limited Routed split multilink trunking
JP4123088B2 (en) * 2003-08-06 2008-07-23 株式会社日立製作所 Storage network management apparatus and method
US7380025B1 (en) * 2003-10-07 2008-05-27 Cisco Technology, Inc. Method and apparatus providing role-based configuration of a port of a network element
US8179808B2 (en) 2003-10-31 2012-05-15 Brocade Communication Systems, Inc. Network path tracing method
US8050180B2 (en) 2003-10-31 2011-11-01 Brocade Communications Systems, Inc. Network path tracing method
US7561578B2 (en) 2003-11-13 2009-07-14 Cryptek, Inc. System and method for traversing metadata across multiple network domains at various layers of the protocol stack
US7558273B1 (en) 2003-12-23 2009-07-07 Extreme Networks, Inc. Methods and systems for associating and translating virtual local area network (VLAN) tags
US7333508B2 (en) 2004-01-20 2008-02-19 Nortel Networks Limited Method and system for Ethernet and frame relay network interworking
US7701948B2 (en) 2004-01-20 2010-04-20 Nortel Networks Limited Metro ethernet service enhancements
US7843925B2 (en) 2004-01-20 2010-11-30 Nortel Networks Limited Ethernet differentiated services architecture
US7849098B1 (en) 2004-02-06 2010-12-07 Vmware, Inc. Providing multiple concurrent access to a file system
US7310664B1 (en) * 2004-02-06 2007-12-18 Extreme Networks Unified, configurable, adaptive, network architecture
US7860097B1 (en) * 2004-02-13 2010-12-28 Habanero Holdings, Inc. Fabric-backplane enterprise servers with VNICs and VLANs
US7843906B1 (en) 2004-02-13 2010-11-30 Habanero Holdings, Inc. Storage gateway initiator for fabric-backplane enterprise servers
US7843907B1 (en) * 2004-02-13 2010-11-30 Habanero Holdings, Inc. Storage gateway target for fabric-backplane enterprise servers
US7477894B1 (en) 2004-02-23 2009-01-13 Foundry Networks, Inc. Methods and apparatus for handling wireless roaming among and across wireless area networks
US7690040B2 (en) 2004-03-10 2010-03-30 Enterasys Networks, Inc. Method for network traffic mirroring with data privacy
US20050220096A1 (en) 2004-04-06 2005-10-06 Robert Friskney Traffic engineering in frame-based carrier networks
WO2005109908A2 (en) 2004-04-30 2005-11-17 Vulcan Inc. Maintaining a graphical user interface state that is based on a selected piece of content
JP4373271B2 (en) * 2004-05-14 2009-11-25 富士通株式会社 Method and program for grasping network configuration of virtual LAN in node network
JP4397292B2 (en) 2004-07-09 2010-01-13 富士通株式会社 Control packet loop prevention method and bridge device using the same
FR2873524B1 (en) 2004-07-22 2006-10-27 Alcatel Sa LOCAL NETWORK WITH VIRTUAL GROUP (S) OF HEART EQUIPMENT WHICH IS CLEAR AT THE LEVEL TWO SWITCHING
US7466712B2 (en) 2004-07-30 2008-12-16 Brocade Communications Systems, Inc. System and method for providing proxy and translation domains in a fibre channel router
EP1782293A2 (en) 2004-08-20 2007-05-09 Enterasys Networks, Inc. System, method and apparatus for traffic mirror setup, service and security in communication networks
US7463597B1 (en) 2004-08-27 2008-12-09 Juniper Networks, Inc. Spanning tree protocol synchronization within virtual private networks
US7558263B1 (en) 2004-08-30 2009-07-07 Juniper Networks, Inc. Reliable exchange of control information for multicast virtual private networks
US8116307B1 (en) 2004-09-23 2012-02-14 Juniper Networks, Inc. Packet structure for mirrored traffic flow
US7764768B2 (en) 2004-10-06 2010-07-27 Alcatel-Lucent Usa Inc. Providing CALEA/legal intercept information to law enforcement agencies for internet protocol multimedia subsystems (IMS)
US7508757B2 (en) * 2004-10-15 2009-03-24 Alcatel Lucent Network with MAC table overflow protection
US7801125B2 (en) 2004-10-22 2010-09-21 Cisco Technology, Inc. Forwarding table reduction and multipath network forwarding
US8238347B2 (en) 2004-10-22 2012-08-07 Cisco Technology, Inc. Fibre channel over ethernet
GB2419701A (en) * 2004-10-29 2006-05-03 Hewlett Packard Development Co Virtual overlay infrastructure with dynamic control of mapping
EP1657853A1 (en) 2004-11-12 2006-05-17 STMicroelectronics (Research & Development) Limited Roaming network stations using a MAC address identifier to select a new access point
US8700799B2 (en) 2004-11-12 2014-04-15 Brocade Communications Systems, Inc. Methods, devices and systems with improved zone merge operation by operating on a switch basis
US7808992B2 (en) 2004-12-30 2010-10-05 Cisco Technology, Inc. Platform independent implementation of private VLANS
US20070036178A1 (en) 2005-02-02 2007-02-15 Susan Hares Layer 2 virtual switching environment
US20060184937A1 (en) * 2005-02-11 2006-08-17 Timothy Abels System and method for centralized software management in virtual machines
US7586895B2 (en) 2005-04-01 2009-09-08 Cisco Technology, Inc. Performing extended lookups on MAC-based tables including level 3 multicast group destination addresses
US7673068B2 (en) 2005-04-18 2010-03-02 Alcatel Lucent Method and system for implementing a high availability VLAN
GB2425681A (en) * 2005-04-27 2006-11-01 3Com Corporaton Access control by Dynamic Host Configuration Protocol snooping
US7835370B2 (en) 2005-04-28 2010-11-16 Cisco Technology, Inc. System and method for DSL subscriber identification over ethernet network
US8751649B2 (en) * 2005-06-07 2014-06-10 Extreme Networks Port management system
US20060285499A1 (en) 2005-06-17 2006-12-21 Broadcom Corporation Loop detection for a network device
US7571447B2 (en) 2005-06-20 2009-08-04 International Business Machines Corporation Loose coupling of web services
GB0516158D0 (en) 2005-08-05 2005-09-14 Univ Montfort An apparatus and method for `non-contact' electrical impedance imaging
US8095774B1 (en) 2007-07-05 2012-01-10 Silver Peak Systems, Inc. Pre-fetching data into a memory
US7937756B2 (en) 2005-08-19 2011-05-03 Cpacket Networks, Inc. Apparatus and method for facilitating network security
EP1924864B1 (en) 2005-09-12 2015-03-04 Rockstar Consortium US LP Forwarding plane data communications channel for ethernet transport networks
US9143841B2 (en) 2005-09-29 2015-09-22 Brocade Communications Systems, Inc. Federated management of intelligent service modules
DE102005048585A1 (en) 2005-10-06 2007-04-12 Robert Bosch Gmbh Subscriber and communication controller of a communication system and method for implementing a gateway functionality in a subscriber of a communication system
CN100442772C (en) 2005-10-19 2008-12-10 华为技术有限公司 Bridge-connection transmitting method
US9497600B2 (en) 2005-10-28 2016-11-15 Hewlett Packard Enterprise Development Lp Service chaining
US7697528B2 (en) 2005-11-01 2010-04-13 Nortel Networks Limited Multilink trunking for encapsulated traffic
EP1964426B1 (en) 2005-12-12 2010-07-28 Telefonaktiebolaget LM Ericsson (publ) Method and devices for specifying the quality of service in a transmission of data packets
US7716240B2 (en) 2005-12-29 2010-05-11 Nextlabs, Inc. Techniques and system to deploy policies intelligently
US7835378B2 (en) 2006-02-02 2010-11-16 Cisco Technology, Inc. Root node redundancy for multipoint-to-multipoint transport trees
US20070177597A1 (en) 2006-02-02 2007-08-02 Yu Ju Ethernet connection-based forwarding process
US7639605B2 (en) 2006-02-08 2009-12-29 Cisco Technology, Inc. System and method for detecting and recovering from virtual switch link failures
US8189575B2 (en) 2006-03-13 2012-05-29 Rockstar Bidco, L.P. Modular scalable switch architecture
US8160080B1 (en) 2006-05-08 2012-04-17 Marvell Israel (M.I.S.L.) Ltd. Implementation of reliable synchronization of distributed databases
JP2007318553A (en) * 2006-05-26 2007-12-06 Fujitsu Ltd Network managing method
US8018938B1 (en) 2006-06-02 2011-09-13 World Wide Packets, Inc. Translating between a switching format and a transport format
JP4834493B2 (en) * 2006-08-25 2011-12-14 アラクサラネットワークス株式会社 Network relay device and method for controlling network relay device
US20080072280A1 (en) * 2006-08-30 2008-03-20 Tardo Joseph J Method and system to control access to a secure asset via an electronic communications network
CN100583825C (en) 2006-08-30 2010-01-20 华为技术有限公司 Method of generating symmetrical tree in the shortest path bridge
US8396945B2 (en) 2006-09-11 2013-03-12 Alcatel Lucent Network management system with adaptive sampled proactive diagnostic capabilities
US7697556B2 (en) 2006-10-26 2010-04-13 Telefonaktiebolaget L M Ericsson (Publ) MAC (media access control) tunneling and control and method
US7720889B1 (en) * 2006-10-31 2010-05-18 Netapp, Inc. System and method for nearly in-band search indexing
WO2008056838A1 (en) 2006-11-08 2008-05-15 Chang Hwan Cho System and method for controlling network traffic
US20080112400A1 (en) 2006-11-15 2008-05-15 Futurewei Technologies, Inc. System for Providing Both Traditional and Traffic Engineering Enabled Services
US7599901B2 (en) 2006-12-06 2009-10-06 Microsoft Corporation Processing data-centric business models
US20080159277A1 (en) 2006-12-15 2008-07-03 Brocade Communications Systems, Inc. Ethernet over fibre channel
US20080181243A1 (en) 2006-12-15 2008-07-31 Brocade Communications Systems, Inc. Ethernet forwarding in high performance fabrics
US8973098B2 (en) * 2007-01-11 2015-03-03 International Business Machines Corporation System and method for virtualized resource configuration
US7706255B1 (en) 2007-01-29 2010-04-27 Solace Systems, Inc. Communications apparatus with redundant switching or backpressure mechanism
US20080181196A1 (en) 2007-01-31 2008-07-31 Alcatel Lucent Link aggregation across multiple chassis
JP4259581B2 (en) 2007-02-07 2009-04-30 日立電線株式会社 Switching hub and LAN system
IL189514A (en) 2007-02-14 2011-12-29 Marvell Israel Misl Ltd Logical bridging system and method
US9661112B2 (en) * 2007-02-22 2017-05-23 International Business Machines Corporation System and methods for providing server virtualization assistance
US8140696B2 (en) 2007-03-12 2012-03-20 International Business Machines Corporation Layering serial attached small computer system interface (SAS) over ethernet
US8077721B2 (en) 2007-03-15 2011-12-13 Cisco Technology, Inc. Methods and apparatus providing two stage tunneling
US7916741B2 (en) 2007-04-02 2011-03-29 William Marsh Rice University System and method for preventing count-to-infinity problems in ethernet networks
US8078704B2 (en) 2007-04-12 2011-12-13 Hewlett-Packard Development Company, L.P. Provisioning of a service environment using web services
US8301686B1 (en) * 2007-04-16 2012-10-30 Citrix Systems, Inc. Systems and methods for decentralized computing
US7873038B2 (en) 2007-04-30 2011-01-18 Hewlett-Packard Development Company, L.P. Packet processing
US7724674B2 (en) 2007-05-16 2010-05-25 Simula Innovations As Deadlock free network routing
JP4862743B2 (en) 2007-05-17 2012-01-25 日本電気株式会社 Node, communication method and node program
US20080298248A1 (en) 2007-05-28 2008-12-04 Guenter Roeck Method and Apparatus For Computer Network Bandwidth Control and Congestion Management
US7945941B2 (en) * 2007-06-01 2011-05-17 Cisco Technology, Inc. Flexible access control policy enforcement
US20080310342A1 (en) 2007-06-12 2008-12-18 Cisco Technology, Inc. Addressing Messages in a Two-Tier Network
US7898959B1 (en) 2007-06-28 2011-03-01 Marvell Israel (Misl) Ltd. Method for weighted load-balancing among network interfaces
US8615008B2 (en) 2007-07-11 2013-12-24 Foundry Networks Llc Duplicating network traffic through transparent VLAN flooding
GB0713785D0 (en) 2007-07-16 2007-08-22 Cellfire Security Technologies Voice over IP system
US20090044270A1 (en) 2007-08-07 2009-02-12 Asaf Shelly Network element and an infrastructure for a network risk management system
US7836332B2 (en) * 2007-07-18 2010-11-16 Hitachi, Ltd. Method and apparatus for managing virtual ports on storage systems
US7864712B2 (en) 2007-07-20 2011-01-04 Cisco Technology, Inc. Preventing loops in networks operating different protocols to provide loop-free topology
US8166205B2 (en) 2007-07-31 2012-04-24 Cisco Technology, Inc. Overlay transport virtualization
US7729296B1 (en) 2007-09-07 2010-06-01 Force 10 Networks, Inc. Distributed BPDU processing for spanning tree protocols
US20090080345A1 (en) 2007-09-21 2009-03-26 Ericsson, Inc. Efficient multipoint distribution tree construction for shortest path bridging
US8798056B2 (en) * 2007-09-24 2014-08-05 Intel Corporation Method and system for virtual port communications
US20090079560A1 (en) 2007-09-26 2009-03-26 General Electric Company Remotely monitoring railroad equipment using network protocols
JP5393686B2 (en) 2007-09-26 2014-01-22 ニシラ, インコーポレイテッド Network operating system for managing and securing a network
JP5030063B2 (en) 2007-10-05 2012-09-19 本田技研工業株式会社 Navigation device and navigation system
US7975033B2 (en) * 2007-10-23 2011-07-05 Virtudatacenter Holdings, L.L.C. System and method for initializing and maintaining a series of virtual local area networks contained in a clustered computer system
WO2009064407A1 (en) 2007-11-16 2009-05-22 Ericsson Ab Method and system for telecommunications including self-organizing scalable ethernet using is-is hierarchy
US8117495B2 (en) 2007-11-26 2012-02-14 Stratus Technologies Bermuda Ltd Systems and methods of high availability cluster environment failover protection
WO2009068045A1 (en) 2007-11-26 2009-06-04 Telefonaktiebolaget Lm Ericsson (Publ) Technique for address resolution in a data transmission network
US8194674B1 (en) * 2007-12-20 2012-06-05 Quest Software, Inc. System and method for aggregating communications and for translating between overlapping internal network addresses and unique external network addresses
US7796593B1 (en) 2007-12-21 2010-09-14 Juniper Networks, Inc. Router using internal flood groups for flooding VPLS traffic
US7860093B2 (en) 2007-12-24 2010-12-28 Cisco Technology, Inc. Fast multicast convergence at secondary designated router or designated forwarder
US8018841B2 (en) 2007-12-31 2011-09-13 Ciena Corporation Interworking an ethernet ring network and an ethernet network with traffic engineered trunks
JP2009187368A (en) * 2008-02-07 2009-08-20 Hitachi Ltd Method for controlling sharing of usb port
US20090245137A1 (en) 2008-03-03 2009-10-01 Green Hills Software, Inc. Highly available virtual stacking architecture
US20090222879A1 (en) 2008-03-03 2009-09-03 Microsoft Corporation Super policy in information protection systems
US8230069B2 (en) * 2008-03-04 2012-07-24 International Business Machines Corporation Server and storage-aware method for selecting virtual machine migration targets
WO2009115726A1 (en) 2008-03-04 2009-09-24 France Telecom Technique for determining a point-to-multipoint tree linking a root node to a plurality of leaf nodes
PT2615085E (en) 2008-03-31 2015-10-09 Vertex Pharma Pyridyl derivatives as cftr modulators
US7792148B2 (en) 2008-03-31 2010-09-07 International Business Machines Corporation Virtual fibre channel over Ethernet switch
US8743740B2 (en) 2008-04-08 2014-06-03 At&T Intellectual Property I, L.P. Methods and apparatus to implement a partial mesh virtual private local area network service
US7911982B1 (en) 2008-05-01 2011-03-22 Juniper Networks, Inc. Configuring networks including spanning trees
WO2009139489A1 (en) 2008-05-16 2009-11-19 日本電気株式会社 Pci express switch, pci express system, and network control method
US8195774B2 (en) * 2008-05-23 2012-06-05 Vmware, Inc. Distributed virtual switch for virtualized computer systems
US8160063B2 (en) 2008-06-09 2012-04-17 Microsoft Corporation Data center interconnect and traffic engineering
US7873711B2 (en) * 2008-06-27 2011-01-18 International Business Machines Corporation Method, system and program product for managing assignment of MAC addresses in a virtual machine environment
US7941539B2 (en) * 2008-06-30 2011-05-10 Oracle America, Inc. Method and system for creating a virtual router in a blade chassis to maintain connectivity
KR101508794B1 (en) 2008-07-09 2015-04-06 삼성전자주식회사 Method for selectively securing records in a ndef message
US8102791B2 (en) 2008-07-25 2012-01-24 Newport Media, Inc. Interleaver address generation in turbo decoders for mobile multimedia multicast system communication systems
US8102781B2 (en) 2008-07-31 2012-01-24 Cisco Technology, Inc. Dynamic distribution of virtual machines in a communication network
US9426095B2 (en) 2008-08-28 2016-08-23 International Business Machines Corporation Apparatus and method of switching packets between virtual ports
US8259569B2 (en) 2008-09-09 2012-09-04 Cisco Technology, Inc. Differentiated services for unicast and multicast frames in layer 2 topologies
US8134922B2 (en) * 2008-09-12 2012-03-13 Cisco Technology, Inc. Reducing flooding in a bridged network
US8392606B2 (en) * 2008-09-23 2013-03-05 Synapse Wireless, Inc. Wireless networks and methods using multiple valid network identifiers
US7944812B2 (en) 2008-10-20 2011-05-17 International Business Machines Corporation Redundant intermediary switch solution for detecting and managing fibre channel over ethernet FCoE switch failures
US9100269B2 (en) 2008-10-28 2015-08-04 Rpx Clearinghouse Llc Provisioned provider link state bridging (PLSB) with routed back-up
US7962647B2 (en) * 2008-11-24 2011-06-14 Vmware, Inc. Application delivery control module for virtual network switch
WO2010071888A2 (en) 2008-12-19 2010-06-24 Watchguard Technologies, Inc. Cluster architecture and configuration for network security devices
US7929554B2 (en) 2008-12-23 2011-04-19 Cisco Technology, Inc. Optimized forwarding for provider backbone bridges with both I and B components (IB-PBB)
US8509248B2 (en) 2008-12-29 2013-08-13 Juniper Networks, Inc. Routing frames in a computer network using bridge identifiers
US8054832B1 (en) * 2008-12-30 2011-11-08 Juniper Networks, Inc. Methods and apparatus for routing between virtual resources based on a routing location policy
US8255496B2 (en) * 2008-12-30 2012-08-28 Juniper Networks, Inc. Method and apparatus for determining a network topology during network provisioning
US8331362B2 (en) * 2008-12-30 2012-12-11 Juniper Networks, Inc. Methods and apparatus for distributed dynamic network provisioning
US7820853B2 (en) 2008-12-31 2010-10-26 Celanese International Corporation Integrated process for the production of vinyl acetate from acetic acid via ethyl acetate
US8336079B2 (en) 2008-12-31 2012-12-18 Hytrust, Inc. Intelligent security control system for virtualized ecosystems
JP5168166B2 (en) 2009-01-21 2013-03-21 富士通株式会社 Communication apparatus and communication control method
US9043621B2 (en) * 2009-01-21 2015-05-26 Hitachi, Ltd. Power-saving network management server, network system, and method of determining supply of power
CA2756265A1 (en) 2009-02-13 2010-08-19 Adc Telecommunications, Inc. Inter-networking devices for use with physical layer information
US8213336B2 (en) * 2009-02-23 2012-07-03 Cisco Technology, Inc. Distributed data center access switch
US8238340B2 (en) 2009-03-06 2012-08-07 Futurewei Technologies, Inc. Transport multiplexer—mechanisms to force ethernet traffic from one domain to be switched in a different (external) domain
US8155150B1 (en) 2009-03-11 2012-04-10 Juniper Networks, Inc. Cooperative MAC learning/aging in highly distributed forwarding system
US8665886B2 (en) 2009-03-26 2014-03-04 Brocade Communications Systems, Inc. Redundant host connection in a routed network
CA2756289C (en) 2009-04-01 2016-02-02 Nicira Networks, Inc. Method and apparatus for implementing and managing virtual switches
US8213313B1 (en) 2009-04-15 2012-07-03 Tellabs Operations, Inc. Methods and apparatus for shared layer 3 application card in multi-service router
US8116213B2 (en) 2009-04-24 2012-02-14 Verizon Patent And Licensing Inc. Tracing routes and protocols
US8027354B1 (en) * 2009-04-29 2011-09-27 Cisco Technology, Inc. Network consolidation for virtualized servers
US8874709B2 (en) 2009-05-01 2014-10-28 Futurewei Technologies, Inc. Automatic subnet creation in networks that support dynamic ethernet-local area network services for use by operation, administration, and maintenance
US8429647B2 (en) * 2009-05-06 2013-04-23 Vmware, Inc. Virtual machine migration across network by publishing routes to the associated virtual networks via virtual router after the start of migration of the virtual machine
US20100287262A1 (en) * 2009-05-08 2010-11-11 Uri Elzur Method and system for guaranteed end-to-end data flows in a local networking domain
US9282057B2 (en) 2009-05-11 2016-03-08 Brocade Communication Systems, Inc. Flexible stacking port
US8472443B2 (en) * 2009-05-15 2013-06-25 Cisco Technology Port grouping for association with virtual interfaces
US8165122B2 (en) 2009-05-26 2012-04-24 Alcatel Lucent System and method for converting unicast client requests into multicast client requests
US8170038B2 (en) 2009-05-27 2012-05-01 International Business Machines Corporation Two-layer switch apparatus to avoid first layer inter-switch link data traffic in steering packets through bump-in-the-wire service applications
US8174984B2 (en) 2009-05-29 2012-05-08 Oracle America, Inc. Managing traffic on virtualized lanes between a network switch and a virtual machine
US7944860B2 (en) 2009-06-04 2011-05-17 Cisco Technology, Inc. Preventing loss of network traffic due to inconsistent configurations within the network
US8199753B2 (en) 2009-06-05 2012-06-12 Juniper Networks, Inc. Forwarding frames in a computer network using shortest path bridging
US8102760B2 (en) 2009-06-30 2012-01-24 Alcatel Lucent Method for reconvergence after failure in a dual-homing network environment
US8351352B1 (en) 2009-07-15 2013-01-08 Eastlake Iii Donald E Methods and apparatus for RBridge hop-by-hop compression and frame aggregation
US8204061B1 (en) 2009-07-23 2012-06-19 Cisco Technology, Inc. Virtual port channel switches with distributed control planes
US8125928B2 (en) 2009-07-24 2012-02-28 Juniper Networks, Inc. Routing frames in a shortest path computer network for a multi-homed legacy bridge node
US8341725B2 (en) * 2009-07-30 2012-12-25 Calix, Inc. Secure DHCP processing for layer two access networks
US8503329B2 (en) 2009-08-05 2013-08-06 Cisco Technology, Inc. Signaling of attachment circuit status and automatic discovery of inter-chassis communication peers
US8504690B2 (en) * 2009-08-07 2013-08-06 Broadcom Corporation Method and system for managing network power policy and configuration of data center bridging
IL200504A0 (en) 2009-08-20 2011-08-01 Eci Telecom Ltd Technique for dual homing interconnection between communication networks
US8369332B2 (en) 2009-08-21 2013-02-05 Alcatel Lucent Server-side load balancing using parent-child link aggregation groups
US8706905B1 (en) 2009-08-24 2014-04-22 Qlogic, Corporation Method and system for routing information in a network
US8339994B2 (en) 2009-08-27 2012-12-25 Brocade Communications Systems, Inc. Defining an optimal topology for a group of logical switches
US8369347B2 (en) 2009-09-14 2013-02-05 Futurewei Technologies, Inc. Fiber channel over Ethernet and fiber channel switching based on Ethernet switch fabrics
US8599850B2 (en) 2009-09-21 2013-12-03 Brocade Communications Systems, Inc. Provisioning single or multistage networks using ethernet service instances (ESIs)
US8914598B2 (en) 2009-09-24 2014-12-16 Vmware, Inc. Distributed storage resource scheduler and load balancer
US8599864B2 (en) 2009-10-08 2013-12-03 Brocade Communications Systems, Inc. Transit switches in a network of logical switches
US20110085560A1 (en) * 2009-10-12 2011-04-14 Dell Products L.P. System and Method for Implementing a Virtual Switch
US8693485B2 (en) * 2009-10-14 2014-04-08 Dell Products, Lp Virtualization aware network switch
EP2497234B1 (en) 2009-11-02 2018-09-19 Marvell World Trade Ltd. Network device and method based on virtual interfaces
US20110113146A1 (en) 2009-11-10 2011-05-12 Li Gordon Yong Dynamic quality of service (qos) setup over wired and wireless networks
US20110134802A1 (en) 2009-12-09 2011-06-09 Cisco Technology, Inc. Determining A Routing Tree For Networks With Different Routing Protocols
WO2011074516A1 (en) 2009-12-15 2011-06-23 日本電気株式会社 Network system, method for controlling same, and controller
US8270420B2 (en) 2009-12-15 2012-09-18 Hewlett-Packard Development Company, L.P. iSCSI to FCoE gateway
US8705513B2 (en) * 2009-12-15 2014-04-22 At&T Intellectual Property I, L.P. Methods and apparatus to communicatively couple virtual private networks to virtual machines within distributive computing networks
US8295291B1 (en) 2009-12-21 2012-10-23 Juniper Networks, Inc. Computation of next hops within layer two networks
US8161156B2 (en) 2009-12-30 2012-04-17 Verizon Patent And Licensing, Inc. Feature delivery packets for peer-to-peer based feature network
JP5493926B2 (en) 2010-02-01 2014-05-14 日本電気株式会社 Interface control method, interface control method, and interface control program
US8619595B2 (en) 2010-02-05 2013-12-31 Cisco Technology, Inc. Fault isolation in trill networks
CN102158386B (en) 2010-02-11 2015-06-03 威睿公司 Distributed load balance for system management program
US8873401B2 (en) 2010-03-16 2014-10-28 Futurewei Technologies, Inc. Service prioritization in link state controlled layer two networks
US8369335B2 (en) 2010-03-24 2013-02-05 Brocade Communications Systems, Inc. Method and system for extending routing domain to non-routing end stations
US8249069B2 (en) 2010-03-30 2012-08-21 Cisco Technology, Inc. Forwarding multi-destination packets in a network with virtual port channels
JP5190084B2 (en) * 2010-03-30 2013-04-24 株式会社日立製作所 Virtual machine migration method and system
US8599854B2 (en) * 2010-04-16 2013-12-03 Cisco Technology, Inc. Method of identifying destination in a virtual environment
US8611352B2 (en) 2010-04-20 2013-12-17 Marvell World Trade Ltd. System and method for adapting a packet processing pipeline
US8345692B2 (en) 2010-04-27 2013-01-01 Cisco Technology, Inc. Virtual switching overlay for cloud computing
US9461840B2 (en) 2010-06-02 2016-10-04 Brocade Communications Systems, Inc. Port profile management for virtual cluster switching
US9270486B2 (en) 2010-06-07 2016-02-23 Brocade Communications Systems, Inc. Name services for virtual cluster switching
US8989186B2 (en) 2010-06-08 2015-03-24 Brocade Communication Systems, Inc. Virtual port grouping for virtual cluster switching
US8625616B2 (en) 2010-05-11 2014-01-07 Brocade Communications Systems, Inc. Converged network extension
US9231890B2 (en) 2010-06-08 2016-01-05 Brocade Communications Systems, Inc. Traffic management for virtual cluster switching
US8867552B2 (en) 2010-05-03 2014-10-21 Brocade Communications Systems, Inc. Virtual cluster switching
US8520595B2 (en) 2010-05-04 2013-08-27 Cisco Technology, Inc. Routing to the access layer to support mobility of internet protocol devices
US8335236B2 (en) 2010-05-06 2012-12-18 Cisco Technology, Inc. FCoE isolated port channels and FCoE session resynchronization in vPC/MCEC environments using DCBXP
US8724456B1 (en) 2010-05-19 2014-05-13 Juniper Networks, Inc. Network path selection for multi-homed edges to ensure end-to-end resiliency
US9491085B2 (en) 2010-05-24 2016-11-08 At&T Intellectual Property I, L.P. Methods and apparatus to route control packets based on address partitioning
US8667171B2 (en) 2010-05-28 2014-03-04 Microsoft Corporation Virtual data center allocation with bandwidth guarantees
BR112012018762B1 (en) 2010-05-28 2022-06-21 Huawei Technologies Co., Ltd System, network component and method for promoting communication between a plurality of access domains
US9246703B2 (en) 2010-06-08 2016-01-26 Brocade Communications Systems, Inc. Remote port mirroring
US8446914B2 (en) 2010-06-08 2013-05-21 Brocade Communications Systems, Inc. Method and system for link aggregation across multiple switches
US9806906B2 (en) 2010-06-08 2017-10-31 Brocade Communications Systems, Inc. Flooding packets on a per-virtual-network basis
US10033650B2 (en) 2010-06-08 2018-07-24 Brocade Communication Systems Llc Preserving quality of service across trill networks
US9608833B2 (en) 2010-06-08 2017-03-28 Brocade Communications Systems, Inc. Supporting multiple multicast trees in trill networks
US9628293B2 (en) 2010-06-08 2017-04-18 Brocade Communications Systems, Inc. Network layer multicasting in trill networks
US20110299533A1 (en) 2010-06-08 2011-12-08 Brocade Communications Systems, Inc. Internal virtual network identifier and internal policy identifier
US8897134B2 (en) 2010-06-25 2014-11-25 Telefonaktiebolaget L M Ericsson (Publ) Notifying a controller of a change to a packet forwarding configuration of a network element over a communication channel
CN104396192B (en) 2010-06-29 2018-03-06 华为技术有限公司 Dissymmetric network address encapsulates
US8588081B2 (en) 2010-07-14 2013-11-19 Cisco Technology, Inc. Monitoring a flow set to detect faults
US8873551B2 (en) 2010-07-30 2014-10-28 Cisco Technology, Inc. Multi-destination forwarding in network clouds which include emulated switches
US8462774B2 (en) 2010-08-04 2013-06-11 Alcatel Lucent Virtual IP interfaces on multi-chassis link aggregates
US9049098B2 (en) 2010-08-05 2015-06-02 Cisco Technology, Inc. Discovery of services provided by application nodes in a network
US8665267B2 (en) 2010-09-24 2014-03-04 Adobe Systems Incorporated System and method for generating 3D surface patches from unconstrained 3D curves
US20120099602A1 (en) 2010-10-25 2012-04-26 Brocade Communications Systems, Inc. End-to-end virtualization
US8634297B2 (en) 2010-11-01 2014-01-21 Cisco Technology, Inc. Probing specific customer flow in layer-2 multipath networks
US8762668B2 (en) 2010-11-18 2014-06-24 Hitachi, Ltd. Multipath switching over multiple storage systems
US8806031B1 (en) 2010-12-15 2014-08-12 Juniper Networks, Inc. Systems and methods for automatically detecting network elements
US8521884B2 (en) 2010-12-15 2013-08-27 Industrial Technology Research Institute Network system and method of address resolution
US20120163164A1 (en) 2010-12-27 2012-06-28 Brocade Communications Systems, Inc. Method and system for remote load balancing in high-availability networks
US8559335B2 (en) 2011-01-07 2013-10-15 Jeda Networks, Inc. Methods for creating virtual links between fibre channel over ethernet nodes for converged network adapters
US8755383B2 (en) 2011-03-21 2014-06-17 Avaya, Inc. Usage of masked ethernet addresses between transparent interconnect of lots of links (TRILL) routing bridges
US9054999B2 (en) 2012-05-09 2015-06-09 International Business Machines Corporation Static TRILL routing
US8605626B2 (en) 2011-05-18 2013-12-10 Cisco Technology, Inc. Method and apparatus for preserving extensions in multi-vendor trill networks
US20120294192A1 (en) 2011-05-19 2012-11-22 Hitachi, Ltd. Method and apparatus of connectivity discovery between network switch and server based on vlan identifiers
US9497073B2 (en) 2011-06-17 2016-11-15 International Business Machines Corporation Distributed link aggregation group (LAG) for a layer 2 fabric
CN102232279B (en) 2011-06-17 2013-09-11 华为技术有限公司 Method for detecting loop position in Ethernet, and Ethernet switching device
US9736065B2 (en) 2011-06-24 2017-08-15 Cisco Technology, Inc. Level of hierarchy in MST for traffic localization and load balancing
US9380132B2 (en) 2011-06-27 2016-06-28 Marvell Israel (M.I.S.L.) Ltd. FCoE over trill
US8559302B2 (en) 2011-06-29 2013-10-15 Fujitsu Limited Systems and methods for distributed service protection across plug-in units
US8619635B2 (en) 2011-06-29 2013-12-31 Telefonaktiebolaget L M Ericsson (Publ) E-tree using two pseudowires between edge routers with enhanced forwarding methods and systems
US20130003738A1 (en) 2011-06-29 2013-01-03 Brocade Communications Systems, Inc. Trill based router redundancy
US8467375B2 (en) 2011-07-07 2013-06-18 Ciena Corporation Hybrid packet-optical private network systems and methods
US8705551B2 (en) 2011-07-27 2014-04-22 Fujitsu Limited Method and system for management of flood traffic over multiple 0:N link aggregation groups
US20130034015A1 (en) 2011-08-05 2013-02-07 International Business Machines Corporation Automated network configuration in a dynamic virtual environment
US8966499B2 (en) 2011-09-09 2015-02-24 Microsoft Technology Licensing, Llc Virtual switch extensibility
US9185056B2 (en) 2011-09-20 2015-11-10 Big Switch Networks, Inc. System and methods for controlling network traffic through virtual switches
US8885643B2 (en) 2011-11-04 2014-11-11 Futurewei Technologies, Inc. Method for multicast flow routing selection
US8995272B2 (en) 2012-01-26 2015-03-31 Brocade Communication Systems, Inc. Link aggregation in software-defined networks
WO2013123982A1 (en) 2012-02-22 2013-08-29 Nokia Siemens Networks Oy Controlling access
US9154416B2 (en) 2012-03-22 2015-10-06 Brocade Communications Systems, Inc. Overlay tunnel in a fabric switch
US9184995B2 (en) 2012-04-11 2015-11-10 Gigamon Inc. Traffic visibility in an open networking environment
CN102801599B (en) 2012-07-26 2015-09-30 华为技术有限公司 A kind of communication means and system
US8855117B2 (en) 2012-08-08 2014-10-07 Cisco Technology, Inc. Scalable media access control protocol synchronization techniques for fabric extender based emulated switch deployments
US8937865B1 (en) 2012-08-21 2015-01-20 Juniper Networks, Inc. Scheduling traffic over aggregated bundles of links
US9602430B2 (en) 2012-08-21 2017-03-21 Brocade Communications Systems, Inc. Global VLANs for fabric switches

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050195813A1 (en) * 2004-02-23 2005-09-08 Sinett Corporation Unified architecture for wired and wireless networks
US20080080517A1 (en) * 2006-09-28 2008-04-03 At & T Corp. System and method for forwarding traffic data in an MPLS VPN
US20090052326A1 (en) * 2007-08-21 2009-02-26 Cisco Technology, Inc., A Corporation Of California Backward congestion notification
US20090092043A1 (en) * 2007-10-03 2009-04-09 Nortel Networks Limited Providing an abstraction layer in a cluster switch that includes plural switches
US20090116381A1 (en) * 2007-11-07 2009-05-07 Brocade Communications Systems, Inc. Method and system for congestion management in a fibre channel network
US20100103939A1 (en) * 2008-10-24 2010-04-29 Carlson Scott M Determining the configuration of an ethernet fabric
US7787480B1 (en) * 2009-03-04 2010-08-31 Juniper Networks, Inc. Routing frames in a trill network using service VLAN identifiers

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10841375B2 (en) 2013-11-01 2020-11-17 Hewlett Packard Enterprise Development Lp Protocol agnostic storage access in a software defined network topology
US10361912B2 (en) * 2014-06-30 2019-07-23 Huawei Technologies Co., Ltd. Traffic switching method and apparatus
US10644935B2 (en) 2014-07-22 2020-05-05 Huawei Technologies Co., Ltd. Method for configuring fibre channel storage area network, and apparatus
US10594565B2 (en) 2014-12-19 2020-03-17 Hewlett Packard Enterprise Development Lp Multicast advertisement message for a network switch in a storage area network
US20170034047A1 (en) * 2015-07-31 2017-02-02 Brocade Communications Systems, Inc. Graceful recovery of a multicast-enabled switch
US10439929B2 (en) * 2015-07-31 2019-10-08 Avago Technologies International Sales Pte. Limited Graceful recovery of a multicast-enabled switch
US10693832B1 (en) * 2015-12-30 2020-06-23 Avago Technologies International Sales Pte. Limited Address resolution protocol operation in a fibre channel fabric
US10355753B2 (en) * 2016-03-03 2019-07-16 Andrew Wireless Systems Gmbh Hybrid RAN/digital DAS repeater system with ETHERNET transport
US11190238B2 (en) 2016-03-03 2021-11-30 Andrew Wireless Systems Gmbh Hybrid RAN/digital DAS repeater system with ethernet transport
US20180167111A1 (en) * 2016-03-03 2018-06-14 Andrew Wireless Systems Gmbh Hybrid ran/digital das repeater system with ethernet transport
US10284457B2 (en) * 2016-07-12 2019-05-07 Dell Products, L.P. System and method for virtual link trunking
US11552906B2 (en) 2016-10-21 2023-01-10 Avago Technologies International Sales Pte. Limited Hard zoning of virtual local area networks in a fibre channel fabric
WO2019060004A1 (en) * 2017-09-21 2019-03-28 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
WO2019060001A1 (en) * 2017-09-21 2019-03-28 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
WO2019060003A1 (en) * 2017-09-21 2019-03-28 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
US20190089637A1 (en) * 2017-09-21 2019-03-21 Microsoft Technology Licensing, Llc Virtualizing dcb settings for virtual network adapters
US10860358B2 (en) 2017-09-21 2020-12-08 Microsoft Technology Licensing, Llc Virtualizing datacenter bridging settings for virtual network adapters
US11470021B2 (en) * 2018-10-26 2022-10-11 Cisco Technology, Inc. Managed midlay layers on a routed network
US20200136997A1 (en) * 2018-10-26 2020-04-30 Cisco Technology, Inc. Managed midlay layers on a routed network
US10630554B1 (en) 2018-10-29 2020-04-21 International Business Machines Corporation Input/output (I/O) performance of hosts through bi-directional bandwidth feedback optimization
US20220070625A1 (en) * 2018-12-19 2022-03-03 Telefonaktiebolaget Lm Ericsson (Publ) Reporting of multicast mac addresses
US12052638B2 (en) * 2018-12-19 2024-07-30 Telefonaktiebolaget Lm Ericsson (Publ) Reporting of multicast MAC addresses
US11240160B2 (en) 2018-12-28 2022-02-01 Alibaba Group Holding Limited Method, apparatus, and computer-readable storage medium for network control
US11368395B2 (en) * 2019-03-13 2022-06-21 Suzhou Kuhan Information Technologies Co., Ltd. System, method and apparatus for storage controller having multiple heterogeneous network interface ports
US20220247631A1 (en) * 2019-05-28 2022-08-04 Nippon Telegraph And Telephone Corporation Network management apparatus and method
US11582290B2 (en) * 2020-01-23 2023-02-14 Hewlett Packard Enterprise Development Lp Switch event ordering
US11431652B2 (en) * 2020-07-10 2022-08-30 Dell Products L.P. Automated multi-fabric link aggregation system
US20230353506A1 (en) * 2022-04-29 2023-11-02 Hewlett Packard Enterprise Development Lp Remote management of a switch stack
US11956164B2 (en) * 2022-04-29 2024-04-09 Hewlett Packard Enterprise Development Lp Remote management of a switch stack

Also Published As

Publication number Publication date
US20110299413A1 (en) 2011-12-08
US9461840B2 (en) 2016-10-04

Similar Documents

Publication Publication Date Title
EP3041173B1 (en) Virtual cluster switching
US11438219B2 (en) Advanced link tracking for virtual cluster switching
US20170155599A1 (en) Virtual cluster switching
US9942173B2 (en) Distributed configuration management for virtual cluster switching
US9461911B2 (en) Virtual port grouping for virtual cluster switching
US9848040B2 (en) Name services for virtual cluster switching
US9231890B2 (en) Traffic management for virtual cluster switching
US8625616B2 (en) Converged network extension
US9485148B2 (en) Fabric formation for virtual cluster switching
US8995435B2 (en) Port profile analytics
US8995287B2 (en) AMPP active profile presentation
US10673703B2 (en) Fabric switching

Legal Events

Date Code Title Description
AS Assignment

Owner name: BROCADE COMMUNICATIONS SYSTEMS LLC, CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:BROCADE COMMUNICATIONS SYSTEMS, INC.;REEL/FRAME:044891/0536

Effective date: 20171128

AS Assignment

Owner name: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED, SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROCADE COMMUNICATIONS SYSTEMS LLC;REEL/FRAME:047270/0247

Effective date: 20180905

Owner name: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROCADE COMMUNICATIONS SYSTEMS LLC;REEL/FRAME:047270/0247

Effective date: 20180905

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

AS Assignment

Owner name: BROCADE COMMUNICATIONS SYSTEMS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:VOBBILISETTY, SURESH;CHATWANI, DILIP;REEL/FRAME:055104/0926

Effective date: 20110615

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION