WO2002088875A2 - Communication de donnees dans un reseau - Google Patents

Communication de donnees dans un reseau Download PDF

Info

Publication number
WO2002088875A2
WO2002088875A2 PCT/US2002/012428 US0212428W WO02088875A2 WO 2002088875 A2 WO2002088875 A2 WO 2002088875A2 US 0212428 W US0212428 W US 0212428W WO 02088875 A2 WO02088875 A2 WO 02088875A2
Authority
WO
WIPO (PCT)
Prior art keywords
data
destination
path
communication
network
Prior art date
Application number
PCT/US2002/012428
Other languages
English (en)
Other versions
WO2002088875A3 (fr
Inventor
Michael S. Foster
Michael A. Dorsett
James C. Braatz
Rodney A. Hughes
Turan A. Dao
Original Assignee
The Boeing Company
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/046,640 external-priority patent/US20020159437A1/en
Priority claimed from US10/046,572 external-priority patent/US20030210685A1/en
Application filed by The Boeing Company filed Critical The Boeing Company
Priority to AU2002303416A priority Critical patent/AU2002303416A1/en
Publication of WO2002088875A2 publication Critical patent/WO2002088875A2/fr
Publication of WO2002088875A3 publication Critical patent/WO2002088875A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227—Filtering policies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/35—Switches specially adapted for specific applications
    • H04L49/356—Switches specially adapted for specific applications for storage area networks
    • H04L49/357—Fibre channel switches

Definitions

  • the following disclosure relates generally to computer networks, and more particularly to techniques for communicating data through networks.
  • the Internet has emerged as a critical commerce and communications platform for businesses and consumers worldwide.
  • These applications include e-commerce, e-mail, electronic file transfers, and online interactive applications.
  • e-commerce electronic commerce
  • e-mail electronic file transfers
  • online interactive applications As the number of users of and uses for the Internet increases, so does the complexity and volume of Internet traffic. Because of this traffic and its business potential, a growing number of companies are building businesses around the Internet and developing mission-critical business applications to be provided by the Internet.
  • EDNs enterprise data networks
  • LAN web server local area network
  • SAN storage area network
  • the processing and storage elements attached to these sub-networks may have access to a wide area network (WAN) or metropolitan area network (MAN) through a bridging device commonly known as an edge switch.
  • WAN wide area network
  • MAN metropolitan area network
  • edge switch a bridging device commonly known as an edge switch.
  • each of these subnetworks typically uses a distinct protocol and associated set of hardware and software, including network interface adapters, network switches, network operating systems, and management applications.
  • QOS Quality Of Service
  • current architectures typically assign one or more network addresses to each node in a network (e.g., logical network addresses such as IP addresses and/or physical network addresses such as Media Access Control (“MAC”) addresses), and network routing and switching devices use the network addresses of a destination node to route transmissions of data from a source node to that destination node.
  • logical network addresses such as IP addresses and/or physical network addresses such as Media Access Control (“MAC”) addresses
  • Figure 1 is a network diagram illustrating various nodes of an example Fibre Channel Fabric network that are inter-communicating.
  • Figures 2A-2C illustrate an example of Virtual Identifier Network Interface Controller ("NIC") embodiments using virtual identifiers to inter-communicate through an example Fibre Channel Fabric network.
  • NIC Virtual Identifier Network Interface Controller
  • Figure 3 is a block diagram illustrating a node using an embodiment of the disclosed Virtual Identifier NIC to communicate with other nodes.
  • Figure 4 is a flow diagram of an embodiment of the Communication Registrar routine.
  • Figure 5 is a flow diagram of an embodiment of the Outgoing Communication Translator routine.
  • FIG. 6 is a flow diagram of an embodiment of the Verify Communication Transmittal subroutine.
  • Figure 7 is a flow diagram of an embodiment of the Incoming Communication Translator routine.
  • Figure 8 is a flow diagram illustrating an embodiment of topology discovery processing of a component of the interconnect fabric module.
  • Figure 9 is a flow diagram illustrating an embodiment of topology discovery processing of the network manager.
  • Figure 10 is a flow diagram illustrating an embodiment of a process for establishing a path by the network manager.
  • Figure 11 is a flow diagram illustrating an embodiment of a process for identifying virtual identifiers by the network manager.
  • Figure 12 is a flow diagram illustrating an embodiment of a process for initializing label tables by the network manager.
  • Figure 13 is a block diagram illustrating a distributed network manager in one embodiment.
  • Figure 14 is a flow diagram illustrating an embodiment of processing reserved addresses by a component of an interconnect fabric module.
  • Figure 15 is a block diagram illustrating components of an embodiment of an interconnect fabric module.
  • Figure 16 is a block diagram illustrating components of an embodiment of a switch protocol controller.
  • Figure 17 is a block diagram illustrating example contents of one embodiment of a label table.
  • Figure 18 is a block diagram illustrating the format of an example frame.
  • Figure 19 is a diagram illustrating logic of an arbitrator of a switch protocol controller in one embodiment.
  • Figure 20 is a block diagram illustrating an embodiment of a transmit controller.
  • Figure 21 is a block diagram illustrating an example interconnection of IFMs to form an interconnect fabric connecting various nodes.
  • Figure 22 is a block diagram illustrating an example of mapping a virtual identifier to a port map.
  • Figure 23 is a block diagram illustrating an example of switch protocol controller caching in one embodiment.
  • Figure 24 is a block diagram illustrating an interconnect fabric configuration with multiple direct links between interconnect fabric modules.
  • Figure 25 is a block diagram illustrating an example use of equivalent ports.
  • Figure 26 is a diagram illustrating logic of an embodiment of the equivalent port service.
  • Figure 27 illustrates an example of preempting a connection.
  • Figure 28 is a diagram illustrating logic of an embodiment of processing a preemption signal.
  • Figure 29 is a diagram illustrating logic of an embodiment of distributed class 3 multicasting.
  • Figure 30 is a block diagram illustrating an example of node-to-node data communication using QOS communication parameters over a network fabric.
  • Figure 31 is a dataflow diagram illustrating an example of using QOS communication parameters for a data communication.
  • Figure 32 illustrates an example of configuring a node to use QOS communication parameters and of monitoring the network to determine whether to modify those configured communication parameters.
  • each node of a network first registers with a network manager for the network, such as to identify to the network manager the node and/or one or more hardware ports of the node that are connected to the network.
  • a network node Before a network node communicates data to other network nodes, such as from software executing on that node (e.g., an application program), that node (or software executing on the node) registers that data communication with the network manager, such as in a dynamic manner for each data communication when it is to occur and/or by pre-registering data communications of an indicated type.
  • the network manager in some embodiments provides to the node an identifier corresponding to one or more destination nodes that is to be used for routing the data communication to those destinations.
  • the network manager When the network manager receives a request to register a data communication (also referred to in some embodiments as an "application login request") from a network node that is to be a source of the data communication, the network manager determines whether to allow the data communication, and in some situations determines one or more appropriate destination nodes for the data.
  • An appropriate destination determination can be made in a variety of ways, such as by selecting appropriate destinations based on a type of the data communication, on information about destinations specified in the request, and/or on other accessible information about nodes, such as nodes that are interested in receiving the data communication.
  • the network manager configures one or more intermediate network routing devices along a path between the source node and the one or more destination nodes in such a manner that the data communication will reach the appropriate destinations.
  • the network manager selects one or more virtual identifiers to be used with a registered data communication (or with multiple data communications of a specified type), configures each of the intermediate network routing devices along the path to properly handle data communications using those virtual identifiers, and then notifies the source node of the selected virtual identifiers for use with the data communication.
  • the network manager also determines one or more Quality Of Service (“QOS”) parameters that are appropriate for a registered data communication, such as based on a type of the data to be communicated.
  • QOS Quality Of Service
  • types of QOS levels also referred to as “service levels” or “QOS policies”
  • QOS communication parameters can be selected for use with a data communication in order to ensure that a selected QOS level will be achieved, such as particular communication priority levels and/or a communication Class Of Service for networks that support such communication parameters.
  • the network manager After selecting QOS communication parameters for a data communication, the network manager notifies the source node and/or some or all of the intermediate network routing devices along the selected path of the selected QOS communication parameters for use with the data communication.
  • the network manager may also perform other tasks in some embodiments to ensure that a selected QOS level is achieved for one or more data communications, including selecting an appropriate path through the network (e.g., a path through an appropriate group of intermediate routing devices and/or that is configured to allow load balancing between two or more of the intermediate routing devices along the path), controlling which other data communications are allowed to use some or all of the selected path, controlling a rate or time at which such other data communications can occur, and/or enabling preemption of other data communications that are using some or all of the selected path.
  • an appropriate path through the network e.g., a path through an appropriate group of intermediate routing devices and/or that is configured to allow load balancing between two or more of the intermediate routing devices along the path
  • controlling which other data communications are allowed to use some or all of the selected path controlling a rate or time at which such other data communications can occur
  • enabling preemption of other data communications that are using some or all of the selected path.
  • QOS levels that are actually achieved for some or all data communications may also in some embodiments be monitored (e.g., in real-time), and if so the QOS communication parameters that are assigned to a data communication and/or the path to be used for the data communication may be dynamically modified based on differences between actual achieved QOS levels and desired QOS levels.
  • a source network node can then transmit the data to one or more appropriate destination nodes.
  • the source network node transmits the data to a first intermediate network routing device along a path to the destinations (e.g., via a hardware port on the node that is connected to that first intermediate network routing device) along with an indication corresponding to the one or more destination nodes.
  • the network manager has notified the source node of one or more virtual identifiers to be used for the data communication, the source node uses those virtual identifiers as the indication corresponding to the destination nodes.
  • the source node uses those QOS communication parameters with the data communication.
  • destination nodes can be indicated in other manners, such as with network addresses associated with the destination nodes.
  • the intermediate network routing device determines how to forward the data communication based on the destination indication and the manner in which the network manager has configured the intermediate network routing device. For example, in embodiments in which the network manager has selected one or more virtual identifiers to be used with the data communication, the network manager configures each intermediate network routing device along the path assigned to those virtual identifiers to forward data communications using those virtual identifiers to a next of the intermediate network routing devices along the path (or, for the last intermediate network routing device, to forward the data communication directly to a destination node).
  • the intermediate network routing devices along the path may be configured to forward that data communications in accordance with those QOS communication parameters, or the intermediate network routing devices may instead forward the data communication using the same QOS communication parameters as were used with the data communication when received.
  • data communications are also filtered during their transmission, such as by the source node, the destination node, and/or one or more of the intermediate network routing devices.
  • filtering can be performed in a variety of ways and for a variety of purposes (e.g., security and/or to minimize network traffic to ensure QOS parameters).
  • the filtering can ensure that a data communication has the correct combination of virtual identifiers.
  • the filtering can also ensure that a data communication is being transmitted using appropriate QOS parameters.
  • a variety of actions may be taken, such as to block the transmission of that data communication, to notify a designated observer (e.g., the network manager) of the problem, to modify the transmission so as to remove the problem, etc.
  • a designated observer e.g., the network manager
  • one or more Virtual Identifier (“VI”) Network Interface Controller (“NIC”) facilities on each node facilitate the use of virtual identifiers in communicating data.
  • VI Virtual Identifier
  • NIC Network Interface Controller
  • a VI NIC on a node receives an indication that a data communication to one or more remote nodes is to occur, such as from an application executing on the node, the VI NIC will identify an appropriate transmittal virtual identifier that can be used to route the data communication through the network to the appropriate remote destination nodes without being assigned to or directly associated with those destination nodes.
  • Such data communications can include both transitory connectionless transmittals of data (e.g., unidirectional transmittals from a source to a destination) and non-transitory connections that allow multiple distinct transmittals of data (e.g., a persistent dedicated connection that allows a connection-initiating source and a connection destination to transmit data back and forth).
  • transitory connectionless transmittals of data e.g., unidirectional transmittals from a source to a destination
  • non-transitory connections that allow multiple distinct transmittals of data (e.g., a persistent dedicated connection that allows a connection-initiating source and a connection destination to transmit data back and forth).
  • the VI NIC can identify an appropriate transmittal virtual identifier for routing a data communication in various ways.
  • the VI NIC will register some or all outgoing data communications with a network manager for the network, and will receive an appropriate transmittal virtual identifier to be used for that communication from the network manager. If an indicated data communication corresponds to a previously registered data communication (e.g., to an existing connection or to a previous communication to the same destination and in the same transmission manner), however, the VI NIC could instead in some embodiments use the previously received transmittal virtual identifier for that data communication rather than perform an additional registration for the indicated data communication.
  • the manners in which a data communication can be transmitted vary with the transmission characteristics that are supported by a network, and can include factors such as a particular Class Of Service ("COS”) or transmission priority.
  • COS Class Of Service
  • the assignment of paths to such virtual path identifiers is performed in a dynamic fashion after an indication is received that a data communication is to occur, such as by the network manager upon receipt of a data communication registration.
  • the assigning of a virtual path identifier to a path can include the configuring of each of one or more intermediate routing devices (e.g., routers or switches) between the source and the destination, such as by the network manager, so that when one of the routing devices receives a data communication that includes the virtual identifier it will forward the communication in an appropriate manner either directly to the destination or instead to a next routing device along the path that is similarly configured.
  • the VI NIC can also assist in determining appropriate destinations for an indicated data communication, either directly or in conjunction with the network manager (e.g., by registering the data communication with the network manager), with the transmittal virtual identifier for that data communication selected so as to route the data communication to those destinations.
  • the indicated data communication may explicitly specify a destination, such as with a destination network address, while in other situations a destination may not be specified, such as when an application is publishing information and is relying on a third party to route the information to one or more current subscribers for that information.
  • the VI NIC and/or the network manager can select one or more destinations that are appropriate for the indicated data communication, even if the specified destination is not among the selected destinations.
  • This destination selection can be made by considering one or more of various factors, including any destinations specified, any expressions of interest made by potential recipients in the data communication (e.g., subscription requests), the type of data being communicated, the manner of the data communication (e.g., a specified COS and/or transmission priority), the identity or type of the source node and/or source application, the type of a destination application, etc.
  • any destinations specified e.g., any expressions of interest made by potential recipients in the data communication (e.g., subscription requests), the type of data being communicated, the manner of the data communication (e.g., a specified COS and/or transmission priority), the identity or type of the source node and/or source application, the type of a destination application, etc.
  • a source of an indicated data communication may specify a destination using a destination network address that is not mapped to any node in the network, and if so the VI NIC and/or the network manager could then select an appropriate destination for that destination network address.
  • Multiple destinations can also be selected for an indicated data communication, even if that data communication specified a single destination (which may or may not be one of the selected destinations). If so, a single transmittal virtual identifier can be used to route the data communication to each of the multiple selected destinations, such as by configuring one or more intermediary routing devices to divide received communications that use that transmittal virtual identifier so as to forward a copy of such received communications to each of multiple destinations (or multiple next routing devices).
  • virtual identifiers correspond to paths through a network that are specific to a source. If so, a single virtual identifier can be used by different sources for different paths, such as to different destinations if the different paths do not overlap.
  • the use of virtual identifiers also allows a path corresponding to a virtual identifier to be reconfigured in a manner transparent to a source using that virtual identifier, such as to correspond to a different path to the same destination or to a path to a different destination.
  • the VI NIC when a data communication indicated by a source can result in bi-directional communication (e.g., a response from one or more of the destinations), the VI NIC also identifies a response virtual identifier that can be used for routing data from one or more of the destinations back to the source. If the VI NIC registers the data communication with a network manager, this response virtual identifier may be received from the network manager. After identifying this response virtual identifier, the VI NIC associates it with information indicating how to process received data communications that are routed using the response virtual identifier.
  • such received data communications are processed by forwarding the data communications to one or more resources associated with the destination node, such as an executing application program, a file on storage, or a device that is part of the node.
  • resources associated with the destination node such as an executing application program, a file on storage, or a device that is part of the node.
  • a VI NIC for the source node may associate the response virtual identifier with that source application so that received responses can be forwarded to that source application (which then becomes the destination application for those received communications).
  • a virtual identifier with a corresponding destination application to which a data communication will be forwarded can be performed in various ways.
  • software applications that communicate using TCP/IP mechanisms often use TCP/IP sockets, which include a combination of an IP address and a software port number specific to a computing device using that IP address.
  • the response virtual identifier can be associated with socket information for the source application.
  • a destination node associates transmittal virtual identifiers used to route data communications lo that destination with an appropriate resource local to the destination node, such as based on information provided to the destination node by the network manager as part of the registering of those data communications and/or based on information included as part of the data communications.
  • the VI NIC When the VI NIC has access to application-specific information for a destination application for a received communication, such as TCP/IP socket information that is associated with a response virtual identifier, the VI NIC can use the information to provide additional benefits. For example, many network nodes and/or applications executing on such nodes require that various information be correctly specified in a received communication in order for that communication to be accepted, such as for security reasons. One example is that a destination application using TCP/IP communication mechanisms may require that any received transmissions include the correct TCP/IP socket information corresponding to that application. However, the previously discussed use of transmittal virtual identifiers can result in valid communications being received having incorrect TCP/IP socket information for a destination application, as discussed in greater detail below.
  • the VI NIC that receives the communication can replace the incorrect included TCP/IP socket information with the correct information for the application by using the TCP/IP socket information that is associated with the transmittal virtual identifier used to route the communication.
  • the VI NIC may verify the accuracy of the received communication in various ways before performing such information replacement.
  • virtual identifiers can result in valid received communications that have incorrect information for a destination application in various ways. For example, if a source application specifies a destination IP address and that destination IP address is included in the data being communicated (e.g., in a location reserved for such a destination network address), but a VI NIC for that source application identifies one or more destinations that do not correspond to that destination IP address (e.g., that have other IP addresses), then the data communication will include a specified destination IP address that does not correspond to the IP addresses used by applications at the identified destinations. In addition, if multiple destinations with different IP addresses are identified by the VI NIC when only a single destination IP address was specified, most of the destinations will receive communications that do not include correct IP address information.
  • the VI NIC that receives the communication can replace the incorrect included IP address information with the correct IP address information for the application by using the TCP/IP socket information that is associated with the virtual identifier used to route the communication.
  • the VI NIC that sends the data communication can perform the information replacement if that VI NIC has access to the necessary application-specific information for the destination application.
  • a VI NIC can also identify information related to routing a data communication other than a transmittal virtual identifier, either directly or in conjunction with the network manager (e.g., by registering the data communication with the network manager). For example, the VI NIC may identify one or more Quality Of Service ("QOS") parameters that relate to a manner in which the data communication should occur, such as a specified COS and/or a priority to be used for the transmission of the data. If so, the VI NIC can also use such QOS parameters when transmitting data for that data communication.
  • QOS Quality Of Service
  • Fibre Channel A Comprehensive Introduction
  • Robert W. Kembel published by Northwest Learning Associates, Inc., and which is hereby incorporated by reference in its entirety.
  • FIG. 1 is a network diagram illustrating various nodes of an example Fibre Channel fabric-based interconnect network that are inter-communicating using virtual identifiers.
  • multiple interconnect fabric modules (“IFMs”) 110 with high-speed switching capabilities are used as intermediate routing devices to form an interconnect fabric, and multiple nodes 105, a network manager 115 and a Multi-Protocol Edge Switch (“MPEX”) 120 are connected to the fabric.
  • Each of the nodes has at least one VI NIC that uses virtual identifiers when communicating and receiving data.
  • the MPEX is used to connect the Fibre Channel network to an external network, such as an Ethernet-based network, and similarly includes at least one VI NIC. Data is transmitted through the interconnect fabric using frames such as those defined by the Fibre Channel standard.
  • an IFM can be dynamically configured to interconnect its communications ports so that data can be transmitted through the interconnected ports.
  • the network manager receives a registration indication from a VI NIC for a data communication from a source node to a destination node, the network manager selects transmittal and response virtual identifiers to be used by the source and destination nodes when sending frames to each other.
  • the network manager also identifies a path through the IFMs and their ports which frames will use when moving between the nodes.
  • the network manager then configures the IFMs of the identified path so that when a frame that indicates the transmittal or response virtual identifiers is received at one of the IFMs, that frame is forwarded to the destination or source nodes via the path as appropriate. While the transmittal and response virtual identifiers thus use the same path (in opposite directions) in this example embodiment, they can use distinct paths in other embodiments.
  • Each IFM may maintain a virtual identifier table for each of its ports that maps virtual identifiers to its destinations ports.
  • the IFM uses the virtual identifier for that frame and the virtual identifier table for the source port to identify a destination port through which the frame is to be forwarded.
  • a virtual identifier identifies a path between devices, rather than identifying a source or a destination device.
  • a virtual identifier includes both a domain address and a virtual address.
  • Each IFM is assigned a domain address, with the IFMs that are assigned the same domain address being in the same domain.
  • the IFMs use the domain addresses to forward frames between domains, and the network manager may also configure the IFMs with inter- domain paths.
  • an IFM receives a frame whose virtual identifier has a domain address that matches its domain address, then the frame has arrived at its destination domain. The IFM then forwards the frame in accordance with the virtual address of the virtual identifier. If, however, the domain addresses do not match, then the frame has not arrived at its destination domain, and the IFM forwards the frame using an inter- domain path.
  • the virtual identifier table for an IFM port may thus be divided in some embodiments into a domain address table and a virtual address table that respectively map domain addresses and virtual addresses to destination ports through which frames are to be forwarded.
  • Figures 2A-2C illustrate an example of VI NIC embodiments using virtual identifiers to inter-communicate through an example Fibre Channel Fabric network.
  • Figure 2A illustrates various VI NICs 250, 255, 270, 275 and 280 that are inter-communicating through a Fibre Channel fabric-based interconnect network that includes IFMs 262, 264 and 266.
  • Figure 2B illustrates a table containing information related to each of multiple example data communications discussed below.
  • a first example data communication begins when VI NIC 250, which is connected to port 0 of IFM 262, initiates a data communication to VI NIC 270, which is connected to port 25 of IFM 264.
  • This data communication is indicated to be a persistent connection, and VI NIC 250 receives a transmittal virtual identifier A (e.g., from a network manager for the network, not shown) to be used for routing communications to VI NIC 270.
  • VI NIC 270 correspondingly receives a response virtual identifier B to be used for routing responses to VI NIC 250.
  • virtual identifiers can be represented in various formats, such as 24-bit identifiers in a Fibre Channel network.
  • VI NIC 250 is also notified of the response virtual identifier B supplied to VI NIC 270 so that VI NIC 250 can recognize communications received from VI NIC 270 as being part of the persistent connection and can forward those received data communications in an appropriate manner (e.g., to an executing application (not shown) on the node to which VI NIC 250 belongs).
  • VI NIC 270 similarly maps received data communications using virtual identifier A to an appropriate destination on the node to which VI NIC 270 belongs.
  • the transmittal and response virtual identifiers A and B each correspond to a path through IFMs 262 and 264.
  • VI NIC 250 data communications from VI NIC 250 using the transmittal virtual identifier A will be received at port 0 of IFM 262, and will be forwarded by that port along link 262a to output port 29 of IFM 262. That output port is statically connected to port 5 of IFM 264, which will receive the data communications using the transmittal virtual identifier A and will forward them along link 264a to output port 25 of IFM 264. VI NIC 270 will then receive the data communication. Data communications from VI NIC 270 to VI NIC 250 will return in a similar manner along that same path in an opposite direction.
  • Figure 2B illustrates a table reflecting the example data communications between the various VI NICs, and entries 1a and 1b of the table represent the dedicated connection between VI NICs 250 and 270 that was just discussed.
  • the determination of which of an associated pair of virtual identifiers is the transmittal virtual identifier and which is the response virtual identifier is made with respect to the source VI NIC using the virtual identifiers to route a data communication.
  • VI NIC 250 uses virtual identifier A as its transmittal virtual identifier and virtual identifier B as its response virtual identifier for the dedicated connection, while VI NIC 270 uses them in an opposite matter.
  • VI NIC 250 initiates a connectionless data communication to VI NIC 275, which is connected to port 28 of IFM 264.
  • Transmittal and response virtual identifiers C and D are provided to VI NIC 250 to be used for the data communication, with the transmittal virtual identifier C corresponding to a path including link 262a from port 0 of IFM 262 to port 29 of IFM 262, followed by link 264b from port 5 of IFM 264 to port 28 of IFM 264.
  • a response virtual identifier is provided to VI NIC 250 so that response information can be provided to VI NIC 250 if necessary, such as an error message indicating that data communication was not successful.
  • At least the transmittal virtual identifier C will also be provided to VI NIC 275 so that received data communications can be recognized and forwarded in an appropriate manner. Entry 2 of the table illustrated in Figure 2B corresponds to this data communication.
  • VI NIC 250 Immediately after the data communication to VI NIC 275, VI NIC 250 initiates a data communication to VI NIC 280, which is connected to port 20 of IFM 266.
  • VI NIC 250 receives transmittal and response virtual identifiers E and F, with the path corresponding to transmittal identifier E including link 262c from port 0 to port 31 of IFM 262 and link 266a from port 0 to port 20 of IFM 266.
  • Port 31 of IFM 262 is statically connected to port 0 of IFM 266. Entry 3 of the table illustrated in Figure 2B corresponds to this data communication.
  • port 0 of IFM 262 is configured (barring any reconfigurations) to route data communications from VI NIC 250 that use any one of the transmittal virtual identifiers A, C or E. While port 0 forwards data communications in the illustrated embodiments to various specified ports, other configurations could instead be used in other embodiments (or as alternatives to the specified paths, such as if problems with the specified paths occur).
  • port 0 could be configured to forward data communications for all the transmittal virtual identifiers to port 29 of IFM 262, and port 5 of IFM 264 could then be configured to forward data communications using transmittal virtual identifier E to port 10 of IFM 264 for communication to port 8 of IFM 266, which could then forward that received data communication to port 20 of IFM 266 for delivery to VI NIC 280.
  • port 0 could instead be configured to forward data communications for the transmittal virtual identifier C along link 262b to port 30 of IFM 262, and port 6 of IFM 264 could then forward the data communications to port 28 of IFM 264 along link 264c.
  • VI NIC 275 next initiates a data communication that is determined (e.g., by the network manager) should be routed to VI NICs 250 and 280. VI NIC 275 then receives a transmittal virtual identifier E and response virtual identifier G to be used for the data communication.
  • transmittal virtual identifier E corresponds to two paths through the network that lead to the two destination VI NICs.
  • a path to VI NIC 250 includes link 264c from port 28 to port 6 of IFM 264 and link 262b from port 30 to port 0 of IFM 262. Port 6 of IFM 264 is statically connected to port 30 of IFM 262.
  • the path from VI NIC 275 to destination VI NIC 280 includes links 264b from port 28 to port 10 of IFM 264 and link 266b from port 8 to port 20 of IFM 266. Port 10 of IFM 264 is statically connected to port 8 of IFM 266. Entry 4 of the table illustrated in Figure 2B corresponds to this data communication.
  • port 28 of IFM 264 is configured such that when it receives a data communication using the transmittal virtual identifier E from VI NIC 275, the port divides the received data communication and sends a copy of the data communication to both port 6 and port 10 of IFM 264 for forwarding.
  • this single transmittal virtual identifier is used to send a data communication to multiple destinations.
  • the port that initially receives the data communication i.e., port 28 of IFM 264 in this example
  • port 28 of IFM 264 could instead be configured to send only a single copy of the received data communication to port 6 of IFM 264
  • port 30 of IFM 262 could instead be configured to send a copy of the received data communication to both ports
  • port 28 of IFM 264 could be configured as initially discussed, but port 30 of IFM 262 could instead be configured to send copies of the received data communication to both ports 0 and 28 of IFM 262 if VI NIC 255 is determined to be another destination for the data communication.
  • the transmittal virtual identifier E used by VI NIC 275 in this most recent example data communication is identical to the transmittal virtual identifier E previously used by VI NIC 250 for data communication to VI NIC 280.
  • the paths corresponding to virtual identifiers are relative to the source from which those data communications originate, and thus different VI NICs can use the same virtual identifier to correspond to different paths and to different destinations. This is possible since each of the ports of each of the IFMs can be separately configured in the illustrated embodiment to handle data communications having specified virtual identifiers.
  • port 28 of IFM 264 is configured to forward data communications received from VI NIC 275 that use the transmittal virtual identifier E to ports 6 and 10 of IFM 264
  • port 0 of IFM 262 is configured to forward a data communication received from VI NIC 250 that uses the transmittal virtual identifier E to port 31 of IFM 262.
  • FIG. 2C illustrates an example of a virtual identifier translation table used by VI NIC 250 when transmitting and receiving the example data communications.
  • multiple applications programs are executing on a node to which VI NIC 250 corresponds and are using TCP/IP socket communication mechanisms to specify their data communications.
  • the VI NIC can identify various QOS communication parameters to be associated with each data communication.
  • Each entry in the virtual identifier translation table corresponds to a distinct data communication of which the VI NIC has been notified. For example, entry
  • the data communication for entry 1 was initiated by an executing source application opening a TCP/IP socket having a destination of IP address "128.32.78.105" and a destination node software port of 3523, with this TCP/IP socket information stored in column 221 of the table.
  • information is stored in column 223 of the virtual identifier translation table to enable received data communications to be forwarded to the appropriate executing application, which in this case is the source application.
  • the VI NIC 250 determines (e.g., based on the received indication of the data communication) that the source application has source socket information that includes an IP address of "153.83.28.125" and a software port number of 3025 for the node on which the application is executing.
  • the VI NIC 250 also determines appropriate transmittal and response virtual identifiers for the data communication, as well as various QOS parameters related to the data communication (e.g., by registering the data communication with a network manager that supplies the virtual identifiers and QOS parameters).
  • the transmittal and response virtual identifiers are stored in columns 225 and 227 of the table respectively, and the QOS communication parameters are stored in one or more columns 229.
  • the identified QOS communication parameters include a specified COS and an authorized minimum and maximum transmission priority. As shown, this data communication is assigned a COS of "1" (e.g., which may correspond to dedicated connections) and a transmission priority range between 0 and 127 (e.g., the full range for a 7-bit priority value).
  • entries 2, 3 and 4 of the virtual identifier translation table correspond to example communications 2, 3 and 4 listed in the table illustrated in Figure 2B.
  • a single executing application may have multiple virtual identifier pairs shown in different entries of the virtual identifier translation table, such as entries 1 and 2 which share the same TCP/IP socket routing information in column 223.
  • the data communications may be routed to different destination nodes, such as is shown with entries 1 and 3 of the table.
  • the destination for an indicated data communication may be selected on the basis of information other than a specified TCP/IP destination socket, and if so the virtual identifier translation table would instead store in column 221 at least the minimal set of information needed to distinguish between the different data communications of which it is notified.
  • indicated data communications have destinations selected based solely on a type of the executing application or on a type of the data being transmitted, an indication of that type of information could be stored in column 221 instead of the destination TCP/IP socket information.
  • an application could have multiple distinct virtual identifiers that can be used to communicate with a single destination, such as if the virtual identifiers are assigned to different paths through the network or have differing associated QOS parameters.
  • different applications on a source node could in various embodiments use the same or different virtual identifiers to communicate with a single destination, regardless of whether different virtual identifiers were assigned to different paths through the network.
  • Entry 4 in the virtual identifier translation table reflects a data communication initiated by a source other than VI NIC 250, in this case being VI NIC 275.
  • VI NIC 250 will store data in column 223 of the virtual identifier translation table indicating how to forward those received data communications (such as based on destination TCP/IP socket information included in a first of those received data communications), but need not store identification information for VI NIC 275 in column 221 since the example data communication is a 1-way connectionless transmittal.
  • the transmittal virtual identifier E used by VI NIC 275 to route the data communication to VI NIC 250 is shown in column 227 of entry 4 of the virtual identifier translation table as being the response virtual identifier for the data communication, since from the perspective of VI NIC 250 the virtual identifier is used for received data communications.
  • other types of information could be stored in the virtual identifier translation table (e.g., connection preemption information) or existing types of information may not be present, and that the existing information could also be stored in other ways (e.g., by having separate virtual identifier translation tables for outgoing and incoming data communications, or by having a separate table for virtual identifier information and for QOS parameters).
  • Figure 3 illustrates a node computing device 300 suitable for executing an embodiment of a VI NIC that uses virtual identifiers when transmitting and receiving data communications, as well as illustrating various other node computing devices 350 with which node 300 can inter-communicate.
  • the nodes are inter-connected through an Interconnect Fabric 380, and a Network Manager 370 is similarly connected to the Fabric.
  • the node computing device 300 includes a CPU 305, various I/O devices 310, storage 320 and memory 330.
  • the I/O devices include at least one network interface 312 which connects the node to the Interconnect Fabric, as well as computer-readable media drive 313 and various other I/O devices 314.
  • An embodiment of a VI NIC 340 is executing in memory, and it includes a Communication Registrar component 342, an Outgoing Communication Translator component 344 and an Incoming Communication Translator component 346.
  • VI NIC in the illustrated embodiment includes multiple components executing in the main memory of the node, those skilled in the art will appreciate that other arrangements are possible in other embodiments, such as implementing a VI NIC together with a network interface on a single plug-in card that attaches to a bus for the node and that may include stand-alone memory and/or processing capabilities including hard-wired logic.
  • some or all of the VI NIC components may be a device driver for the node, such as for one of the network Menaces.
  • multiple VI NICs may be executing on a single node, such as to correspond to multiple network interfaces.
  • multiple application programs 335 are also executing in memory, and can initiate or receive data communications with application programs executing on remote nodes.
  • the VI NIC is notified of the data communication, and the Communication Registrar component then registers that the communication with the Network Manager.
  • the Communication Registrar component receives a pair of transmittal and response virtual identifiers from the Network Manager as well as various QOS communication parameters for that data communication.
  • the Communication Registrar component retrieves and uses network manager communication parameters 327 from storage that may include a transmittal virtual identifier to route the communication to the Network Manager and a response virtual identifier to recognize the information received back from the Network Manager.
  • the network manager communication parameters can be obtained in various ways, such as directly from the Network Manager during initialization of the node and/or the network.
  • the Communication Registrar component After the Communication Registrar component receives the virtual identifier pair and other information for the registered data communication, it stores that information in the virtual identifier translation table 325 on storage for use when transmitting and receiving data communications.
  • the Outgoing Communication Translator receives notification of the communication to be performed. If the initial notification used by the Communication Registrar to initiate registration was itself an indication to perform a communication, the Outgoing Communication Translator component can receive this notification from the Communication Registrar component after the registration has been completed.
  • the Outgoing Communication Translator component analyzes the information provided about the data communication to be performed, maps that data communication to a corresponding entry in the virtual identifier translation table in order to determine the appropriate transmission information to be used for the data communication, and then transmits the data using the information retrieved from the virtual identifier translation table.
  • the Outgoing Communication Translator may also need to perform additional formatting of the data to be transmitted, such as to generate one or more appropriate Fibre Channel frames for the illustrated example in which the network is a Fibre Channel Interconnect Fabric.
  • the Outgoing Communication Translator component may verify the accuracy of the communication indicated by the application program before transmitting the communication, such as to ensure that a priority requested by the application program to be used for the transmission falls within the transmission priority bounds assigned to the data communication.
  • the Incoming Communication Translator component is notified when the network interface receives incoming data communications that are routed using virtual identifiers. Upon receiving notification of such a received data communication, the Incoming Communication Translator determines the transmittal virtual identifier used to route the data communication to the node and uses the virtual identifier translation table to map that virtual identifier to one or more of the application programs executing in memory. Upon determining one or more appropriate application programs to receive the data communication, the VI NIC then forwards the received data communication to those application programs.
  • the necessary information for forwarding the received data communication will already be present in the virtual identifier translation table based on the Communication Registrar component having previously registered that data communication.
  • the necessary information to forward the received data communication to an executing application program may or may not already be present in the virtual identifier translation table.
  • the Network Manager will have supplied the appropriate information to the VI NIC (e.g., to the Communication Registrar component) before the data communication is received, and the information could be stored in the virtual identifier translation table at that time.
  • the appropriate information for forwarding the received data communication may be added to the virtual identifier translation table at the time that the data communication is received, such as by the Incoming Communication Translator component analyzing information included in the data communication to identify the needed information.
  • the Incoming Communication Translator component will also process received data communications in various ways before forwarding them to one or more appropriate application programs. For example, in some embodiments application programs may expect received data communications to include information specific to the receiving application, such as one or more network addresses associated with that application. If the VI NIC has access to the appropriate information for the application, such as from the virtual identifier translation table, the Incoming Communication Translator component can add that information to a received data communication when it is missing or incorrect (or for every received data communication). For example, when the executing applications are using TCP/IP socket mechanisms or more generally receiving data in the form of IP packets, the Incoming Communication Translator component could ensure that the data communication forwarded to an executing application includes the appropriate IP address and/or port number associated with that application. In addition, those skilled in the art will appreciate that the Incoming Communication Translator component may need to reformat received information into an appropriate form for the application receiving the information, such as by converting a received Fibre Channel frame into one or more IP packets.
  • node computing device 300 is merely illustrative and is not intended to limit the scope of the present invention.
  • Computing device 300 may be connected to other devices that are not illustrated, including one or more networks such as the Internet or via the World Wide Web.
  • computing device 300 could be one part of an EDN, such as by being a device at any one or more of the EDN sub-networks.
  • Various available products could be used as network interfaces and/or to implement some or all of the functionality of a VI NIC, including products from Banderacom, Inc. and Mellanox Technologies.
  • the functionality provided by the illustrated VI NIC components may in some embodiments be combined in fewer components or distributed in additional components.
  • the functionality of some of the illustrated components may not be provided and/or other additional functionality may be available.
  • components and data structures may also be stored (e.g., as instructions or structured data) on a computer-readable medium, such as a hard disk, a memory, a network, or a portable article to be read by an appropriate drive.
  • the components and data structures can also be transmitted as generated data signals (e.g., as part of a carrier wave) on a variety of computer-readable transmission mediums, including wireless-based and wired/cable-based mediums. Accordingly, the present invention may be practiced with other computer system configurations.
  • Figure 4 is a flow diagram of an embodiment of the Communication Registrar routine 400.
  • the routine receives indications of new data communications from either a local executing application or from a remote network manager, registers new data communications indicated by local applications with the network manager and receives appropriate virtual identifiers and other information to be used for the data communication in response, and stores any received information from the network manager in the virtual identifier translation table for use in processing incoming and outgoing data communications.
  • the routine begins at step 405 where an indication is received of a new communication.
  • the routine continues to step 410 where it determines if the indication was received from the network manager, such as for a data communication initiated by a remote source. If the data communication is instead from a local executing application, the routine continues to step 415 to determine whatever information about the data communication that will be used to register the data communication with the network manager so that the network manager can determine appropriate destinations for the data communication. In the illustrated embodiment, information is determined about the type of data to be communicated, about the application that initiated the communication, and about any destination information specified by the local application.
  • the routine then continues to step 420 where it is determined if the indicated data communication needs to be registered with the network manager. For example, if the indicated data communication is a transmittal of data for an existing persistent connection, the connection will already have been registered and registration is not necessary. In other situations, even a new data communication may not need to be registered, such as a data communication that will be communicated in the same manner and to the same destination as a previous communication, as the information provided for the previous data communication may be able to be reused.
  • the routine compares whenever information is used to uniquely identify the destination and/or the manner of transmission for the indicated new data communication, and determines if there is a match for that information already present in the virtual identifier translation table. Those skilled in the art will appreciate that in other embodiments the routine may not make this determination and instead send registration information to the network manager for each new indicated data communication, such as if the network manager provides functionality to determine whether to reuse previously provided transmission information or to instead send new transmission information.
  • step 425 If it is determined in step 425 that the new indicated data communication needs to be registered, the routine continues to step 430 to send a communication registration notification to the network manager that includes the determined communication type information. The routine then continues to step 435 to receive from the network manager an indication of a pair of transmittal and response virtual identifiers for the data communication, as well as optionally receiving other communication parameters to be used as part of the communication. The routine then continues to step 440 to store the received information in the virtual identifier translation table, as well as to optionally store routing information with which to route a response data communication back to the application that initiated this new data communication.
  • step 440 the routine continues to step 490 to determine if there are more indications to receive. If so, the routine returns to step 405, and if not the routine continues to step 499 and ends.
  • step 445 receive transmission information related to that new data communication.
  • the network manager will supply information about a pair of transmittal and response virtual identifiers to be used to route the indicated data communication to the node and to be used to route any responses back to the originating node, and will optionally also supply other communication parameters that will be used as part of the data communications.
  • the routine then continues to step 450 to determine a local application to which the incoming communication is to be forwarded, such as based on information supplied by the network manager (e.g., TCP/IP socket information provided by the source application initiating the new data communication). After step 450, the routine continues to step 440 to store the received information from the network manager and the routing information for the local application in the virtual identifier translation table.
  • Figure 5 is a flow diagram of an embodiment of the Outgoing Communication Translator routine 500.
  • the routine receives indications of outgoing data communication, determines an appropriate transmittal virtual identifier to be used for routing the data communication as well as optionally determining other communication parameters to be used, and transmits the data communication using the determined transmittal virtual identifier and other determined communication parameters.
  • the routine begins at step 505 where an indication is received of an outgoing communication.
  • the routine continues to step 510 to execute a subroutine to verify that the communication transmittal is authorized. In other embodiments, this verification step may not be performed.
  • the routine next continues to step 515 to determine if the communication transmittal was determined to be authorized, and if not continues to step 520 to send an error message to the application that initiated the data communication. If it was instead determined in step 515 that the communication transmittal was authorized or if no authorization verification was performed, the routine continues to step 525 to map communication type information for the indicated data communication to a corresponding entry in the virtual identifier translation table for a registered data communication.
  • the communication type information to be used for the mapping may be destination TCP/IP socket information provided by the application that initiated the data communication.
  • the routine then continues to step 530 to format the outgoing data communication in a manner appropriate for the network type being used and to use the transmittal virtual identifier and any other communication parameters indicated in the virtual identifier translation table entry.
  • one or more Fibre Channei frames are generated by storing response and transmittal virtual identifiers in the Fibre Channel frame locations for source and destination identifiers and by storing the data to be communicated as the payload of the frames, and the header information for the frames is specified to correspond to a Fibre Channel COS and a priority that is consistent with the information in the virtual identifier translation table entry.
  • the routine then continues to step 535 to send the generated frames to the local IFM attached to the hardware output port to which the VI NIC corresponds.
  • step 590 the routine continues to step 590 to determine if there are more indications to be received. If so, the routine returns to step 505, and if not the routine continues to step 599 and ends.
  • FIG. 6 is a flow diagram of an embodiment of the Verify Communication Transmittal subroutine 600.
  • the subroutine receives an indication of a data communication and determines if the data communication is consistent with corresponding information that was stored in the virtual identifier translation table based on a prior registration for the data communication.
  • the subroutine begins at step 605 where an indication is received of a communication to be transmitted.
  • the subroutine continues to step 610 to determine if the virtual identifier translation table has an entry corresponding to the communication.
  • step 615 if there was not a corresponding entry, the subroutine to step 635 to return an indication that the data communication is not authorized.
  • step 615 If it is instead determined in step 615 that a corresponding entry was present, the subroutine continues to step 620 to determine if the manner of the transmittal of the data communication is consistent with the transmission information in the corresponding entry, including use of virtual identifiers and other communication parameters. In some embodiments, it is verified that the data communication includes a pair of transmittal and response virtual identifiers that were provided together by the network manager and that the COS and priority for the data communication corresponds to the specified COS and priority limits provided by the network manager. If in step 625 it is determined that the data communication transmittal was not consistent, the subroutine continues to step 635. If the data communication is instead determined to be consistent, the subroutine continues to step 630 to return an indication that the data communication is authorized. After step 630 or 635, the subroutine continues to step 699 and ends.
  • Figure 7 is a flow diagram of an embodiment of the Incoming Communication Translator routine 700.
  • the routine receives indications of incoming data communications and forwards those data communication to appropriate local destinations such as an executing application. In some situations, the routine can also modify the incoming data communication in various ways, such as to replace missing or incorrect destination application-specific information with appropriate information.
  • the routine begins at step 705 where an indication is received of an incoming data communication.
  • the routine continues to step 710 to execute a subroutine to determine whether the data communication is authorized. Those skilled in the art will appreciate that in some embodiments such data communication verification may not be performed.
  • the routine determines if the data communication is authorized, and if not continues to step 720 to send an error message to the sender of the communication. If it is instead determined in step 715 that the data communication is authorized or if the verification is not performed, the routine continues to step 725 to map the transmittal virtual identifier (or in some embodiments the pair of the transmittal and response virtual identifiers) used to route the data communication to a corresponding entry in the virtual identifier translation table.
  • step 730 the routine then determines if the data communication includes information specific to the destination application that is incorrect or is missing, such as a network address. If so and if the correct information is accessible, such as by being stored in the virtual identifier translation table, the routine then continues to step 735 to replace the included incorrect information with the correct information.
  • the routine determines if correct destination application-specific information is accessible it could always be used to replace information sent in a received data communication without checking if the included information is missing or incorrect.
  • step 735 the routine continues to step 740 to forward the received data communication to the appropriate local destination by using the routing information from the corresponding entry in the virtual identifier translation table.
  • the routine may format the data communication in a manner appropriate for the local destination, such as by converting the received data communication into IP packet format.
  • step 795 the routine then continues to step 795 to determine if there are more indications to be received. If so, the routine returns to step 705, and if not the routine continues to step 799 and ends.
  • the network manager configures one or more intermediate network routing devices along a path between the source node and the destination nodes in such a manner that the data communication will reach the appropriate destinations.
  • the network manager receives registration requests from source nodes to send data to destination nodes, configures the routing devices of the network to establish a path from each source node to its destination node, and provides a virtual identifier to each source node.
  • the virtual identifier may be a combination of a domain address and a virtual address, while in other embodiments the virtual identifier may consist only of a virtual address that identifies a path from the source node to the destination node.
  • the source node When the source node sends the data to its destination node, it provides the data along with the virtual identifier to a routing device of the network.
  • a source-side port of each routing device in the path uses the virtual identifier to identify a destination-side port through which the data and the virtual identifier are to be transmitted.
  • the network manager configures the routing devices by setting the mappings from a source-side port to a destination-side port for each routing device in the path.
  • the routing devices receive data via source-side ports and transmits data via destination-side ports.
  • the network manager may be centralized or distributed.
  • a centralized network manager may reside at one node connected to the interconnect fabric.
  • the centralized network manager may provide configuration information to the routing devices using in-band communications or out-of-band communications.
  • In- band communications refers to the use of the communications links connecting the ports of the routing devices.
  • Out-of-band communications refers to the use of communications links used specifically to connect the routing devices to the network manager.
  • a centralized network manager may alternatively reside within a routing device. Each routing device may have the capabilities to be the network manager. Upon initialization, the routing devices may coordinate to select which of the routing devices is to function as the network manager.
  • a distributed network manager in contrast, may have its functions performed at various manager devices connected directly to the routing devices.
  • the network manager at each manager device can control the routing device(s) to which it is directly connected.
  • the network manager at each manager device can communicate with the network managers at other manager devices via in-band or out-of-band communications to coordinate control of the routing devices.
  • the distributed network manager can have different functions performed at various manager devices.
  • the network manager identifies paths through the network from source nodes to destination nodes.
  • the paths may be identified initially when the network manager starts up, when the network topology (e.g., the routing devices of the network and their interconnections) changes (e.g., as a result of a failure), or dynamically when a registration request is received from a source node.
  • the network manager may identify paths dynamically at registration, but may re-identify paths when the topology of the network changes. Regardless of which of these techniques are used, the network manager would typically need to know the topology of the network to identify the paths.
  • the network manager dynamically discovers the topology of the network at initialization.
  • the network manager can discover the topology in several different ways.
  • the network manager can be provided with configuration information that identifies the routing devices of the network.
  • the network manager can use this configuration information to send a message to each routing device asking which of its ports are connected to another device.
  • the network manager can then send a query message via each connected port asking the connected-to device to identify itself and its port. From the responses to the query messages, the network manager can identify the connections (i.e., communications links) between the routing devices and thus the topology of the network.
  • the routing devices upon initialization can request the connected-to devices to provide their identifications.
  • the routing devices can then provide the identifications of the connected-to ports to the network manager.
  • the configuration information along with, the identifications of the connected-to ports describes the network topology.
  • the network manager can dynamically discover the identifications of the routing devices by sending query messages through the ports of the routing device to which it is directly connected. The network manager then becomes aware of each routing device that responds to the query. The network manager then sends a query message through the ports of each responding routing device. Alternatively, the network manager can send one query message to the routing device to which it is directly connected and that routing device can forward the query message via each of its ports to the routing device to which it is directly connected. Each port upon receiving the query message may send a message to the network manager with its identification along with the identification of the port to which it is directly connected.
  • each routing device may dynamically discover which of its ports are connected to other devices (e.g., nodes or other routing devices) at initialization.
  • Each port of a routing device may sense a characteristic of its communications link (e.g., voltage on a receive link) or may transmit a request and receive (or not receive) a response via its communications link to identify whether a device is connected.
  • the network manager may poll each routing device for an indication of which ports of the routing device are connected to other devices. The network manager can then send a query message to each connected-to port to identify the port to which it is connected.
  • the network manager establishes paths through the network of routing devices by configuring the ports of the routing devices along the path.
  • the network manager may identify a path from a source node to a destination node using conventional path identification techniques. For example, the network manager may use a shortest path algorithm to identify the path with the smallest number of communications links or may use a congestion-based algorithm that factors in actual or anticipated network traffic to identify the path.
  • the network manager then identifies a transmittal virtual identifier for the identified path, which will be sent by the source node along with the data to be transmitted to the destination node.
  • the network manager then configures each source-side port of each routing device along the path to forward frames sent with the identified virtual identifier to the destination-side port of the routing device that is connected to the next communications link in the path.
  • the configuration information may be stored in a label table (described below) for the port that maps virtual identifiers to destination-side ports.
  • a source-side port receives a frame with the identified virtual identifier, it then forwards the frame through the destination-side port in accordance with the configuration information.
  • the network manager identifies a virtual identifier that is not currently in use by any source-side port along the path.
  • a source-side port receives a frame addressed with the identified virtual identifier, there is no ambiguity as which port of the routing device is the destination-side port.
  • paths from two different source nodes to the same destination node may have a common sub-path.
  • the path from one source node may be through communications links A, X, Y, and Z
  • the path from the other source node may be through communications links B, X, Y, and Z.
  • the network manager may use the same virtual identifier for both paths and share the terminal portion of the already-configured paths.
  • the network manager may also establish a response path between the destination node and the source node.
  • the network manager may identify a new path or may use the same path that was identified between the source node and the destination node (but in the opposite direction).
  • the network manager then identifies a response virtual identifier and configures the ports along the path in a manner that is analogous to the configuration of the path from the source node to the destination node.
  • a source node sends a frame, it may also include the response virtual identifier in the frame. If so, when the destination node receives the frame it can respond to the source node by sending a frame addressed to that response virtual identifier.
  • the network manager may need to identify and configure a new path between a source node and a destination node. For example, the network manager may determine that, because of congestion, the required quality of service cannot be provided along the existing path or may detect a failure along the existing path. The network manager may be able to use the same virtual identifier to configure the new path. If the network manager uses each virtual identifier only once, then the network manager can use the same virtual identifier for the new path. If, however, the same virtual identifier is used to identify different paths, then it may be possible that the configuration of the new path may conflict with the configuration of another path that uses the same virtual identifier.
  • the network manager can change the path in a manner that is transparent to the source node. In particular, the network manager need not notify the source node of the change in the path. Also, if multiple destination nodes provide the same functionality, then the network manager may implement node load balancing by dynamically changing a path so that data will be sent to a different destination node. The use of these virtual identifiers allows the changes to be made without changing the response and transmittal virtual identifiers of the path.
  • the network manager may reserve one or more virtual identifiers for sending frames from a device (e.g., a routing device or node) to the network manager.
  • a device e.g., a routing device or node
  • a frame may include a registration request from a source node.
  • a routing device may detect when it has received a frame with a reserved virtual identifier and may forward the frame directly to the connected manager device for processing by the network manager.
  • a frame directed to the network manager may include a combination of a reserved virtual identifier and another virtual identifier. When a routing device detects such a frame, it may determine whether it is configured to forward frames directed to the other virtual identifier using in-band communications.
  • the routing device forwards the frame through the destination-side port identified by the other virtual identifier. If the routing device is not configured for the other virtual identifier, then the routing device sends the frame to the network manager via out-of- band communications. For example, the routing device may send the frame to its directly connected manager device. In this way, the network manager can configure the network so that certain frames are forwarded to certain manager devices that provide certain functions or services of the network manager.
  • a routing device is an interconnect fabric module ("IFM") with high-speed switching capabilities.
  • An interconnect fabric module can be dynamically configured to interconnect its communications ports so that data can be transmitted through the interconnected ports.
  • Multiple interconnect fabric modules can be connected to form an interconnect fabric through which nodes (e.g., computer systems) can be interconnected.
  • the interconnect fabric module may allow the creation of an interconnect fabric that is especially well suited for interconnecting devices utilizing multiple information types such as might be required by the devices of an enterprise data network (“EDN").
  • EDN enterprise data network
  • each interconnect fabric module identifies which of its ports are connected to other devices. The network manager then uses this information to send a message through each port that is connected to another device to identify the connected-to device.
  • Figure 8 is a flow diagram illustrating an embodiment of topology discovery processing of a component of the interconnect fabric module. Each port of an interconnect fabric module identifies whether it is connected to a port of another device, such as another switch or a node. The interconnect fabric module then provides to the network manager an indication of which of its ports are connected to other ports to assist in the discovery process. In blocks 801-804, the component determines whether each port is currently connected to another port.
  • the component selects the next port, beginning with the first. In decision block 802, if all the ports have already been selected, then the component completes, else the component continues at block 803. In decision block 803, the component determines whether the selected port is connected to another port. This determination may be made in various ways, such as based on various voltage levels of the communications links. If there is a connection, then the component continues at block 804, else the component loops to block 801 to select the next port of the interconnect fabric module. In block 804, the component notes the selected port as connected to another port and loops to block 801 to select the next port of the interconnect fabric module.
  • Figure 9 is a flow diagram illustrating an embodiment of topology discovery processing of the network manager.
  • the network manager first retrieves an indication of which ports of the interconnect fabric modules are connected to other devices.
  • the network manager then sends a query message through each of the indicated ports to the connected-to port.
  • the connected-to port receives the query message, it responds with an identification of its interconnect fabric module and its port number. In this way, the network manager can discover the topology of the interconnect fabric.
  • the network manager retrieves the indications of which ports of the interconnect fabric modules are connected to other ports.
  • the network manager selects the next interconnect fabric module that has not yet been selected, beginning with the first.
  • the network manager retrieves an indication of which ports of the selected interconnect fabric module are connected to other ports. The network manager may send the message using either in-band our out-of-band communications. The network manager then loops to block 901 to select the next interconnect fabric module. In blocks 904-910, the network manager determines the identity of each of the connected-to ports. In block 904, the network manager selects the next interconnect fabric module, beginning with the first. In decision block 904, if all the interconnect fabric modules have already been selected, then the network manager completes its discovery process, else the network manager continues at block 906.
  • the network manager loops sending a query message through each port of the selected interconnect fabric module that is connected to another port.
  • the network manager selects the next port of the selected interconnect fabric module that is connected to another port, beginning with the first.
  • decision block 907 if all such ports are already selected, then the network manager loops to block 904 to select the next interconnect fabric module, else the network manager continues at block 908.
  • the network manager sends a query message through the selected port of the selected interconnect fabric module.
  • the network manager receives the identification of the connected-to port of the selected port of the selected interconnect fabric module. The identification may include an indication of the interconnect fabric module and the port number of the connected-to port.
  • the network manager stores a mapping between the selected port of the selected interconnect fabric module and the connected-to port of the connected-to interconnect fabric module. These mappings define the topology of the network. The network manager then loops to block 906 to select the next port of the selected interconnect fabric module that is connected to another device.
  • the processing of the discovery of the network manager as described above assumes that the network manager initially is aware of all interconnect fabric modules of the interconnect fabric.
  • the network manager may become aware of additional interconnect fabric modules during the discovery process. For example, if the network manager is centralized, then it may initially send a query message through its port that is connected to the interconnect fabric. The receiving port responds with the identity of the interconnect fabric module and its port number. The network manager can then request that identified interconnect fabric module to provide a indication of which of its ports are connected to other ports. The network manager can then send a query message through each of the indicated ports to the connected-to ports. The connected-to ports then each respond with the identification of the connected-to interconnect fabric module and connected-to port. This process can be repeated by the network manager to identify all interconnect fabric modules that comprise the interconnect fabric.
  • FIG. 10 is a flow diagram illustrating an embodiment of a process for establishing a path by the network manager.
  • a path is typically established when a data communication is registered with the network manager.
  • An establish path component of the network manager may receive an indication of a source node and a destination and then identify paths of ports of interconnect fabric modules from the source node to the destination node, and optionally back from the destination node to the source node. The component then identifies virtual identifiers for the paths and initializes the label tables of the ports of the interconnect fabric modules along the identified paths.
  • a label table of a port contains mappings from virtual identifiers to destination-side ports through which a frame sent u ⁇ ing that virtual identifier is to be forwarded.
  • the component identifies the paths.
  • the path from the source node to the destination node and the path from the destination node to the source node use the same ports of the same interconnect fabric modules. That is, the paths use the same communications links.
  • the path in one direction may be different from the path in the other direction.
  • various well-known techniques for identifying paths can be used.
  • the component invokes an identify virtual identifier component passing an indication of the path from the source node to the destination node(s), and receives in response an indication of the transmittal virtual identifier that is to be used by the source node when sending a communication to the destination node along that path.
  • the invoked component may select a transmittal virtual identifier that is not currently in use by any of the source-side ports of the path.
  • a source-side port of the path is a port that receives data sent by a source node
  • a destination- side port of the path is a port through which data is transmitted on its way to the destination node.
  • the component again invokes an identify virtual identifier component passing an indication of a path from a destination node to the source node, and receives in response a response virtual identifier that is to be used by the destination node when sending a communication to the source node along that path.
  • the component invokes a component to initialize the label tables of the IFM source-side ports along the path associated with the transmittal virtual identifier.
  • the invoked component transmits instructions to each such source-side port indicating that the port is to update its label table to map the transmittal virtual identifier to a destination-side port of that IFM along the path.
  • the component invokes a component to similarly initialize the label tables of the IFM source-side ports along the path associated with the response virtual identifier. The component then completes.
  • Figure 11 is a flow diagram illustrating an embodiment of a process for identifying virtual identifiers by the network manager.
  • the identify virtual identifier component is provided an indication of a path.
  • the component may check every port along the path to identify a virtual identifier that is not currently used by a port along the path.
  • the component may identify virtual identifiers based on a sequential ordering. That is, the component may keep track of the last identified virtual identifier and increment that virtual identifier to identify the next available virtual identifier. In this way, each virtual identifier is unique.
  • the component loops selecting the next virtual identifier and determining whether it is available.
  • the virtual identifier may not be available for use if a port along the path already uses that virtual identifier.
  • the component selects the next virtual identifier, beginning with the first.
  • decision block 1102 if all the virtual identifiers have already been selected, then the component indicates that a virtual identifier could not be identified, else the component continues at block 1103.
  • blocks 1103-1105 the component loops selecting each port along the path and determining whether that port already uses the selected virtual identifier.
  • the component selects the next interconnect fabric module source port of the path, beginning with the first.
  • decision block 1104 if all the interconnect fabric module source ports of the path have already been selected, then the component designates the selected virtual identifier as the identified virtual identifier and completes, else the component continues at block 1105.
  • decision block 1105 if the selected virtual identifier is available at the selected interconnect fabric module port, then the component loops to block 1103 to select the next port along the path, else the component loops to block 1101 to select the next virtual identifier.
  • Figure 12 is a flow diagram illustrating an embodiment of a process for initializing label tables by the network manager.
  • the initialize label table component sends a command to each source port along the path indicating to add a mapping from the identified virtual identifier to the destination port of that interconnect fabric module.
  • the component is passed an indication of the path and the virtual identifier.
  • the component selects the next interconnect fabric module source port in the path, beginning with the first.
  • decision block 1202 if all the interconnect fabric module source ports along the path have already been selected, then the component completes, else the component continues at block 1203.
  • the component sends a message to be selected port indicating to add to its label table a mapping from the virtual identifier to the destination port of the path.
  • the component then loops to block 1201 to select the next interconnect fabric module port in the path.
  • the label tables of the ports in a path are initialized from the destination to the source to ensure that, when a port receives a frame, the label tables of all ports to the destination have already been initialized.
  • the crosspoint switch of an IFM may have more outputs than the number of ports of the IFM.
  • a crosspoint switch may have 34 inputs and outputs, but the IFM may have only 32 ports.
  • the IFM may use these additional ports of the crosspoint switch to route upper layer protocol frames, such as frames directed into a name server or other administrative services.
  • the additional output ports of the crosspoint switch may be connected to a manager device for the IFM.
  • An interconnect fabric module may have a list of "reserved" addresses that designate an upper layer protocol port.
  • an IFM determines that an address of its frame matches one of the reserved addresses, it enables the routing of that frame to an upper layer protocol port.
  • the routing to upper layer protocol ports may use the same arbitration mechanism as used for routing to non-upper layer protocol ports.
  • an output can be selectively switched between a communications port and an upper layer protocol port depending on whether the address of the destination identifier is reserved.
  • Figure 13 is a block diagram illustrating a distributed network manager in one embodiment.
  • the network manager may be implemented on a series of manager devices connected directly to the interconnect fabric modules.
  • the distributed network manager may communicate with each other using in-band communication of the interconnect fabric or using out-of-band communication that is independent of the interconnect fabric.
  • the crosspoint switch of an interconnect fabric module may have reserved ports for the distributed network manager. When an interconnect fabric module receives data that designates one of the reserved ports, then the interconnect fabric module forwards the data to the distributed network manager through the reserved port.
  • Figure 14 is a flow diagram illustrating an embodiment of processing reserved addresses by a component of an interconnect fabric module.
  • This component forwards a frame to the network manager via either in-band or out-of-band communications. With the use of in-band communications, the frame can be routed to the appropriate interconnect fabric module which can then send the frame to the network manager using the out-of-band communications.
  • the component continues at block 1402, else the component completes.
  • decision block 1402 if the virtual identifier parameter within the frame is in the label table, then the frame is to be forwarded using in-band communications and the component continues at block 1404.
  • the frame is to be forwarded directly to the network manager at the IFM's manager device using out-of-band communications and the component continues at block 1403.
  • the component forwards the frame to the administrative port corresponding to the manager and then completes.
  • the component forwards the frame based on the port map of the label table and then completes.
  • an intermediate network routing device determines how to forward the data communication based on a destination indication for the data communication and the manner in which the network manager has configured the intermediate network routing device.
  • the destination indications are virtual identifiers that are associated with paths through the network, such as virtual identifiers selected by the network manager for a path that the network manager has identified between source and destination nodes.
  • the network manager configures the IFMs along the path so that when a frame is received that indicates that virtual identifier, the frame is forwarded to the destination node via the path.
  • the network manager need only configure the IFMs once and the path will remain available, as each of the IFMs maintains a persistent virtual identifier table for each of its ports that maps virtual identifiers to destinations ports of the IFM.
  • an IFM may implement virtual identifier tables using a caching mechanism.
  • Each port of an IFM may have its own local cache of mappings from virtual identifiers to destination ports.
  • the IFM checks the cache of that source port to determine whether it has a mapping for the virtual address used for that frame. If not, the IFM checks a virtual identifier table that is shared by multiple ports. When the virtual identifier table has a mapping for the virtual identifier, then the IFM forwards the frame in accordance with that mapping.
  • the IFM also stores that mapping in the cache for the source port so that the mapping can be retrieved more quickly when a subsequent frame is received at the source port with that virtual identifier.
  • the IFM when the virtual identifier table does not have a mapping for the virtual identifier, the IFM requests the network manager or an external virtual identifier table to provide the mapping. When that mapping is provided by the network manager or the external table, the IFM stores it in the shared virtual identifier table and/or in the port's local cache.
  • an IFM may implement no caching, two-tiered caching, or three-tiered caching for virtual identifiers.
  • an IFM may implement inter-switch load balancing via groups of equivalent ports.
  • IFMs may themselves be interconnected to form an interconnect fabric for connecting nodes.
  • Two IFMs may have multiple links directly connecting their ports. Ports are considered equivalent when a frame can be selectively transmitted on any of the ports to reach its final destination.
  • the use of multiple links (and equivalent ports) between IFMs allows for a greater bandwidth between those IFMs.
  • the network manager may configure each IFM to indicate which groups of its ports are equivalent.
  • the IFM may have an equivalent ports table that maps each port to its equivalent ports. When the IFM receives a frame, it identifies a destination port based on the virtual identifier in the frame.
  • the IFM checks the equivalent ports table to determine whether there any equivalent ports. If so, and the equivalent port is not in use, the IFM forwards the frame through the equivalent port. In this way, IFMs can balance their load through the use of equivalent ports.
  • an IFM allows an existing connection between a source node and a destination node to be preempted by a request for a proposed connection that specifies a higher priority and specifies to preempt existing connections.
  • an IFM receives a connection request at a source port, it identifies a destination port. If the destination port is currently part of an existing connection and the proposed connection indicates to preempt, then the IFM determines whether the proposed connection or the existing connection has a higher priority. If the existing connection has a higher priority, then the IFM indicates that the proposed connection cannot be made. If, however, the proposed connection has a higher priority, then the IFM indicates that the existing connection is to be terminated and then proceeds to establish the proposed connection.
  • a device may send a frame that is to be multicasted to multiple destinations without acknowledgment.
  • the Fibre Channel communications standard refers to such a frame as a class 3 frame. Such frames are not guaranteed to be received by each destination.
  • an IFM receives such a frame, it identifies its destination ports through which the frame is to be forwarded and forwards the frame to each identified destination port that is not currently in use. If an identified destination port is currently in use, the IFM keeps the frame stored in the buffer until the identified destination port becomes available or until the time to live for the frame expires. When an identified destination port becomes available, the IFM forwards the frame to that destination port. In this way, the IFM increases the chances of the frame to being successfully received by all of its destinations.
  • the network manager may authenticate each node attempting to register to ensure that the node is not an imposter node. In this way, only previously authorized nodes can access the network.
  • the routing devices may also discard any communication that is addressed with one or more virtual identifiers that are not properly configured in the routing device. More generally, the routing device and nodes may check the header or other information of a communication to ensure that the communication is valid, and may disregard the communication if not valid.
  • a routing device may detect that a communication received from a node specifies a higher priority than the priority authorized by the network manager, and if so the routing device may discard the communication to prevent the use of the unauthorized priority.
  • a routing device may detect that a communication specifies a combination of virtual identifiers that is not authorized, and if so may discard or modify the communication.
  • the routing device may also remove or disable previously configured virtual identifiers to prevent use by nodes past an allotted time period or to prevent use by an imposter node.
  • a denial-of-service attack can be prevented because an unauthorized node can only send communications through the routing device to which it is directly connected. That routing device can detect that a communication is unauthorized and immediately discard the communication, thus preventing the targeted node from receiving such communications. Moreover, since the routing device that is directly connected to the unauthorized node handles the security, the unauthorized communications do not impact network bandwidth, except possibly for the bandwidth through the directly connected routing device.
  • the network manager coordinates network security with the routing devices and the nodes.
  • the network manager authenticates the node.
  • the network manager and the node may use a PKI-based ("Public Key Infrastructure") authentication technique.
  • a node may generate a private and public key pair.
  • the node then provides its public key to the network manager during authorization that may be coordinated by a person who is a network administrator.
  • the node can register with the network manager.
  • the node encrypts its registration request (or a portion of it) using its private key and then sends the encrypted registration request to the network manager.
  • the network manager decrypts the registration request using the node's public key.
  • the network manager If the request is correctly decrypted, then the network manager knows it was sent by an authorized node and proceeds with the registration. If, however, the request is not correctly decrypted, then the network manager knows that the request was sent by an imposter (or otherwise unauthorized) node and disregards the registration request. To ensure that a registration request is not intercepted and decrypted by an unauthorized node that has the authorized node's public key, the network manager may generate its own private and public key pair and provide its public key to the authorized node. An authorized node can then further encrypt the registration request with the network manager's public key. In this way, only the network manager can decrypt and recognize the registration request.
  • these encryption techniques can also be used to protect data communication registrations and other communications sent via the network. In addition, various other authentication techniques may be used during registration.
  • a routing device filters communications (e.g., communications sent from a directly connected node) so that unauthorized communications are not further transmitted through the network.
  • the routing device may filter communications in various ways, such as based on information contained in the header of the communication.
  • a source-side port that receives a communication may discard the communication if the virtual identifier used for the communication in not in the label table of the port.
  • the network manger when it configures a routing device during registration, may configure the source-side port with filter parameters other than the virtual identifier. For example, the network manager may provide the source-side port with the maximum priority or the classes of service that the node is authorized to use.
  • the port When the port receives a communication, it determines whether any of the filter parameters are unauthorized and, if so, discards the communication.
  • the routing device may also notify the network manager of the unauthorized communication. Because the filtering is performed at the ports, unauthorized communications have minimal impact on internal routing device performance and on overall network performance.
  • the security of the network is further enhanced by the removal of virtual identifiers from the routing device and from the nodes.
  • a virtual identifier When a virtual identifier is removed from a routing device or a node, communications directed to that virtual identifier will no longer be accepted by the routing device or node.
  • a virtual identifier may be removed for various reasons, including when the network manager requests that it be removed, when a routing device or node detects an expiration of the virtual identifier, and/or when the routing device or node detects an error at the physical layer (e.g., due to changing a physical piece of hardware).
  • the network manager may request that a virtual identifier be removed as part of a node's de-registration process.
  • the de-registration may be initiated by the network manager or by the node itself. In either case, the network manager may send a request to remove the virtual identifier to each source-side port along the path associated with that virtual identifier. The network manager may also send a request that the node itself remove its virtual identifier. Routing devices and nodes may also automatically remove a virtual identifier after a certain timeout period, such as a period specified by the network manager, the routing device or the node. The routing device or node may base the timeout on the last use of the virtual identifier, such as by restarting the timeout period whenever a communication is received or sent using that virtual identifier.
  • a certain timeout period such as a period specified by the network manager, the routing device or the node.
  • the routing device or node may also remove a virtual identifier when certain events (e.g., errors) are detected at the physical layer. For example, the physical layer of a routing device may detect that the communications link between the routing device and a node has been removed (e.g., the line has been unplugged from the source-side port of the routing device). In such a case, the routing device may automatically remove all the virtual identifiers associated with that node (e.g., stored in the label table of the source-side port). In this way, an imposter node cannot then be connected to the routing device and start sending communications using the virtual identifiers of the disconnected node.
  • events e.g., errors
  • the routing devices are not configured until a node registers (i.e., just-in-time configuration)
  • the length of time that the network is configured to support a node tends to be minimized and tends to be on an as-needed basis.
  • the configuring of the network on an as-needed basis tends to reduce the opportunities an imposter node has to access the network and tends to free up network resource to be used by other authorized nodes.
  • FIG. 15 is a block diagram illustrating components of one embodiment of an IFM.
  • the IFM 1500 includes 32 switch protocol controllers ("SPCs") 1501 , a crosspoint switch 1502, a switch control unit (“SCU”) 1503, a field programmable gate array (“FPGA”) monitor 1504, an arbitration bus 1505, and an IFM identifier 1506.
  • the IFM has 32 bi-directional communication ports, and one of the SPCs controls each communications port.
  • Each SPC is responsible for decoding the header information of a frame, arbitrating access to destination ports, and transmitting the received frame through the crosspoint connections to one or more communication ports.
  • the SCU receives requests for crosspoint connections from the SPCs, configures the crosspoint switch accordingly, and directs the SPCs to transmit their frames through the crosspoint connections as configured.
  • the crosspoint switch provides full crossbar functionality such that each port of the IFM can be simultaneously connected to any number of the other ports, and includes 34 inputs and 34 outputs (numbered 0-33).
  • the FPGA monitor connects to an IFM manager (not shown), which in the illustrated embodiment is a single board computer that provides an interface for configuring the IFM, and may also provide an interface to upper layer protocol services such as a name server or alias server.
  • the monitor may provide a variety of network/IFM monitoring functions, such as determining QOS levels being achieved on some or all of the paths and/or links in the network (e.g., the links through the IFM 1500) and/or determining QOS levels being achieved for some or all data communications, and can perform the monitoring in a variety of well-known manners.
  • network/IFM monitoring functions such as determining QOS levels being achieved on some or all of the paths and/or links in the network (e.g., the links through the IFM 1500) and/or determining QOS levels being achieved for some or all data communications, and can perform the monitoring in a variety of well-known manners.
  • the monitoring can occur in a distributed manner and the various monitoring information can be combined (e.g., to combine multiple groups of link-specific information to determine path-specific information).
  • FIG. 16 is a block diagram illustrating components of an embodiment of an SPC corresponding to a port on an IFM.
  • the SPC 1600 includes a receive controller 1601 , a decoder 1602, a header processor 1603, a frame buffer 1604, a transmit controller 1605, and an arbitrator 1606.
  • the receive controller is connected to the input (i.e., receive side) of a port and may perform a serial-to-parallel conversion of a received frame.
  • the decoder provides the header information of the received frame to the header processor and stores the frame in the frame buffer.
  • the header processor includes a processor 1607, a label table 1608, and an equivalent port table 1609.
  • the label table contains port maps that indicate to which other ports of the IFM switch a frame should be routed (i.e., the "switch destination ports") if the SPCs corresponding port is the port through which the frame is received (i.e., the "switch source port") based on the destination indicator used for the frame.
  • the processor retrieves the port map from the label table for the received frame and provides the port map to the arbitrator.
  • the equivalent port table indicates groups of ports that are equivalent in the sense that a frame can be sent through any port of an equivalent group to reach the destination. If one port in an equivalent port group is currently in use, then an SPC can equivalents route the frame to any available port in the equivalent port group.
  • the arbitrators of the SPCs coordinate access to the SCU so that an SPC can request the SCU to configure the crosspoint switch in accordance with the port map.
  • the arbitrators and the SCU are connected to an arbitration bus.
  • the arbitrator is also connected to the output (i.e., transmit side) of the corresponding port for transmitting control frames.
  • the transmit controller transmits frames stored in the frame buffer to the switch destination ports via the crosspoint switch when the SCU indicates that the crosspoint switch has been configured appropriately.
  • Figure 17 is a block diagram illustrating example contents of one embodiment of a label table.
  • the entries of the label table are port maps that are indexed by a virtual identifier, with each port map having one bit for each of the 32 ports of the IFM.
  • a bit value of 1 for a port indicates that frames using that indexing virtual identifier should be routed to that port.
  • the first entry in the label table contains a bit value of 1 in the column corresponding to port 2 and contains a bit value of 0 in all the other columns corresponding to ports 0, 1 , and 3-31.
  • the entry in the label table indicates that only port 2 of the IFM should be used as a switch destination port.
  • the second entry in the label table indicates that frames directed to the virtual identifier of 1 are to be routed to switch destination ports 2-31 , but not to ports 0 and 1.
  • the label table of each SPC contains 8K entries, but the size of the label table can be adjusted in other embodiments to meet overall performance goals of the IFM. Because each SPC has its own label table, a frame using a virtual identifier of 0 or 1 that is received via another port would be routed in accordance with a port map in the label table for that other port.
  • Figure 18 is a block diagram illustrating the format of a frame in one embodiment.
  • the illustrated frame is in Fibre Channel format.
  • a frame contains a start-of-frame portion, a header portion, a data portion, and an end-of-frame portion.
  • the header portion includes a 24-bit destination address field, a 24-bit source address field, an 8-bit control field, an 8-bit type field, and an 8-bit priority field.
  • the data portion is variable length and contains up to 2112 bytes. In some embodiments, virtual identifiers are used in place of addresses in the destination address and/or source address fields.
  • the control field indicates whether the frame is a control frame or a data frame.
  • a control frame may include response frames (e.g., an acknowledge frame), fabric control frames, flow control management frames, and link control frames.
  • the flow control management and link control frames are standard Fibre Channel defined frames.
  • the type field indicates the type of data in the data field.
  • a data frame contains payload data that is to be sent from one node to another node using the interconnect fabric.
  • the class of a frame specifies whether a frame is to be sent with or without a connection (e.g., Fibre Channel class 1 represents a connection with acknowledgment).
  • the class field may indicate a class, a priority value, and a preemption flag. Start-of-connection and end-of-connection frames delimit a connection.
  • a connection is a bi-directional, physical connection from a source node through the interconnect fabric to destination node.
  • the IFMs cooperate to establish a physical connection between the source and destination nodes.
  • the physical connection is maintained until an end-of-connection frame is sent via the connection or until a frame that has a priority higher than the connection and that designates to preempt conflicting connections (i.e., its preemptive flag is set) is received by an IFM that needs to use one of its port that is dedicated to the existing connection.
  • FIG 19 is a diagram illustrating example logic of one embodiment of an arbitrator of an SPC.
  • the arbitrator communicates with the SCU via the arbitration bus, which in the illustrated embodiment follows the IEEE 896 Fut ⁇ rebus+ arbitration protocol.
  • the arbitration bus is a wired-or bus in which multiple arbitrators can drive their information onto the bus simultaneously. Based on the information that is being driven on the arbitration bus, each arbitrator determines whether it is the arbitrator with the highest priority that is currently driving the bus. When an arbitrator decides that it does not have the highest priority, it stops driving its information onto the bus. Ultimately, the arbitrator with the highest priority will remain driving the bus.
  • the SCU retrieves the information from the arbitration bus, which includes the port map for the virtual identifier being used, the switch source port number, and the class.
  • the SCU then configures the crosspoint switch to crosspoint connect the input of the switch source port to the output of each switch destination port identified by the port map.
  • the SCU then notifies the arbitrator with the highest priority that the crosspoint switch has been configured.
  • the arbitration bus includes 32 port status lines to indicate whether the corresponding port is currently in use. The SCU sets and clears the status lines as it configures the crosspoint switch.
  • the arbitrator in general, does not participate in arbitrations until all the switch destination ports indicated by the port map become available.
  • the arbitrator raises an arbitration signal on the arbitration bus. If the arbitration signal is already raised, then the arbitrator waits until the arbitration signal is lowered before raising the signal. It is possible that two arbitrators can raise the arbitration signal simultaneously. If so, the arbitrator with the highest priority frame is given control of the arbitration bus.
  • the arbitrator drives a competition number comprising the 7-bit priority of the frame and the 5-bit port number of its port onto the arbitration bus.
  • decision block 1903 if the arbitrator does not have the highest priority, then it stops driving the competition number and other data onto the arbitration bus in block 1904 and then continues to block 1901 so it can eventually raise the arbitration signal and try again.
  • the arbitrator drives the port map, its 5-bit port number, and class onto the arbitration bus.
  • the controller stops driving any data on the arbitration bus, lowers the arbitration signal, and receives confirmation from the SCU when the crosspoint switch has been appropriately configured.
  • the arbitrator signals the transmit controller to transmit the frame to the crosspoint switch.
  • the arbitrator clears the arbitration bus, and then completes. At that point, other arbitrators detect that the arbitration signal has been lowered and can then arbitrate access to the SCU.
  • FIG 20 is a block diagram illustrating an embodiment of a transmit controller.
  • the transmit controller 2000 includes a frame generator 2001 , a multiplexor 2002, and an encoder 2003.
  • the transmit controller when directed by the arbitrator either generates and transmits a control frame or transmits the frame currently stored in a frame buffer.
  • the encoder forwards the frame to the crosspoint switch for transmission through the switch destination ports.
  • FIG. 21 is a block diagram illustrating an example interconnection of IFMs to form an interconnect fabric connecting various nodes.
  • the IFMs 2101 , 2102, 2103, and 2104 form a fully connected interconnect fabric.
  • An interconnect fabric is fully connected when each IFM is directly connected to each other IFM.
  • IFM 2101 is directly connected to IFM 2102 via link 2162, to IFM 2103 via link 2163, and to IFM 2104 via link 2164.
  • Each IFM is also directly connected to various nodes.
  • IFM 2101 is directly connected to nodes 2110. All IFM ports are generic fabric ports ("G-ports") until connected to either other IFMs or nodes.
  • G-ports generic fabric ports
  • FIG. 21 illustrates that a connection has been established between node 2111 and node 2146.
  • Node 2111 is directly connected to port 0 of IFM 2101.
  • Port 0 of IFM 2101 is currently connected to port 30 via the crosspoint connection 2171.
  • Port 30 of IFM 2101 is directly connected to port 29 of IFM 2104 via link 2164.
  • Port 29 of IFM 2104 is currently connected to port 16 via the crosspoint connection 2174.
  • Port 16 of IFM 2104 is directly connected to node 2146.
  • FIG 22 is a block diagram illustrating an example of mapping a virtual identifier to a port map.
  • each IFM has an IFM identifier 2201 , which in the illustrated embodiment contains a domain address that has been assigned to the IFM.
  • the label tables are each sub-divided into a virtual address label table and a domain address label table. If a virtual identifier for a received frame includes a domain address and a virtual address, an SPC processing the frame determines whether the domain address of the virtual identifier matches the domain address assigned to the IFM. If so, then the SPC uses the virtual address label table to retrieve the port map and route the frame.
  • the SPC instead uses a domain address label table to retrieve the port map.
  • the domain address label table port map will be used to route the frame to one of the IFMs in the domain corresponding to the virtual identifier's domain address, and that IFM will then use the virtual address portion of the virtual identifier to route the frame to its destination.
  • An SPC may include a virtual identifier buffer 2202, a comparator 2205, a domain address label table 2206, a virtual address label table 2207, and a selector 2208.
  • the comparator inputs are the domain addresses of the IFM and of the virtual identifier.
  • the comparator signals whether the domain addresses match.
  • the domain address label table is indexed by the domain address of the virtual identifier and outputs the indexed port map.
  • the virtual address label table is indexed by the virtual address of the virtual identifier and outputs the indexed port map.
  • the port maps of the domain address label table and the virtual address label table are input to the selector, which selects a port map based on the input generated by the comparator.
  • the port map is selected from the virtual address label table when the domain addresses of the IFM and of the virtual identifier match and from the domain address label table when they do not match.
  • multiple SPCs of an IFM may share a single label table, which may include both a virtual address label table and a domain address label table in embodiments in which a virtual identifier includes a domain address and a virtual address.
  • the contents of the label table may be dynamically modified to reflect routing algorithms used by a manager of the interconnect fabric.
  • Each SPC that shares a single label table may include a local label table cache in which it stores recently retrieved port maps from the shared label table. An SPC resolves a virtual identifier into its corresponding port map by first checking its local label table cache.
  • the SPC accesses the shared label table.
  • the use of local label tables and a shared label table represents a two-tier caching system.
  • the SPCs additionally use a third caching system tier.
  • the third tier provides access to an extended label table that contains port maps not currently contained in the shared label table.
  • an SPC uses an extended label table interface to retrieve a port map for that virtual identifier from a device that is external to the IFM.
  • FIG. 23 is a block diagram illustrating an example of SPC caching in one embodiment.
  • four SPCs 2310, 2320, 2330, and 2340 share a label table 2350.
  • the four SPCs may be contained on a single board or chip referred to as a quad SPC 2300.
  • Each SPC also has a local label table cache, such as local label table cache 2311 for SPC 2310.
  • the extended label table interface 2360 provides access to, in one embodiment, an IFM manager that receives requests for port maps not currently stored in the shared label table and provides the requested port maps. Alternatively, the extended label table interface can provide access directly to an external label table.
  • the IFM manager may access an overall manager of the interconnect fabric to retrieve the port maps.
  • various well-known caching techniques may be used to implement the described two- tier or three-tier caching system.
  • IFMs may be interconnected to provide inter-switch load balancing.
  • two IFMs may have multiple direct links between them to increase the bandwidth of transmissions between the IFMs.
  • Figure 24 is a block diagram illustrating an interconnect fabric configuration with multiple direct links between IFMs.
  • IFM 2401 and IFM 2402 have three direct links 2410, 2411 , and 2412 between them.
  • the use of multiple direct links allows multiple frames to be transmitted simultaneously between the directly linked IFMs.
  • three nodes directly linked to IFM 2401 may simultaneously have connections established to three different nodes directly linked to IFM 2402.
  • IFM 2401 is also indirectly linked to IFM 2402 via links 2413 and links 2414, 2415, and 2416 through IFM 2404.
  • Ports of an IFM are equivalent when they can be used interchangeably to route frames to their destination. Equivalent ports may have similarly configured label tables to facilitate their use as equivalents.
  • each SPC has an equivalent port table that defines which ports of the IFM are logically equivalent to one another.
  • the SPCs of an IFM may share an equivalent port table.
  • ports 0, 1 , and 2 may be equivalent ports for both IFM 2401 and IFM 2402.
  • an equivalent port service of the SPC determines whether the ports of the port map are currently available. If a port is not currently available, the equivalent port service determines from the equivalent port table whether an equivalent port is available. If so, the equivalent port service modifies the port map so that the frame is routed through the equivalent port. For example, if a port map designates port 0 of IFM 2401 , but port 0 is currently in use, then the equivalent port service may select port 1 as an equivalent to replace port 0 in the port map (assuming port 1 is not currently in use).
  • FIG. 25 is a block diagram illustrating an example use of equivalent ports.
  • Equivalent port service 2503 inputs a port map that may be selected based on virtual identifier 2501 and virtual identifier label table 2502.
  • the equivalent port service also inputs equivalent port table 2504.
  • the equivalent port table contains an entry for each port of the IFM.
  • Each entry, referred to as an equivalent port map, contains a bit for each port of the IFM.
  • the entry for port 0 has its bits for port 1 and port 2 set to indicate that port 0, port 1 , and port 2 are equivalent.
  • the entries for ports 1 and 2 have their bits set in a similar manner.
  • the equivalent port service also inputs the port status lines, which indicates the current status of each of the ports of the IFM.
  • an equivalent port map When the equivalent port service receives a port map it determines whether the designated ports are available based on the port status. If a designated port is not available, the equivalent port service retrieves the equivalent port map for that designated port. The equivalent port service then determines whether any of the equivalent ports are available. If an equivalent port is available, then the equivalent port service changes the port map to designate an available equivalent port. If no equivalent ports are available, then the equivalent port service leaves the port map unchanged.
  • an equivalent port map may also have a priority associated with each port, and if so may select equivalent ports based on their , associated priority. The priorities may be useful, for example, when ports are equivalent, but the cost of routing a frame through the ports are different.
  • port 3 of IFM 2401 may be equivalent to port 0, port 1 , and port 2, but the cost of routing a frame through port 3 may be higher because the frame would travel through IFM 2404 on its way to IFM 2402. If an equivalent port is selected, then the same equivalent port may be selected for subsequent frames of the same sequence of frames. This selection will help ensure that the frames in the sequence are received at the destination in the order that they were transmitted.
  • Figure 26 is a diagram illustrating logic of an embodiment of the equivalent port service.
  • the equivalent port service receives an input port map and processes each designated port of the input port map to determined whether to substitute an equivalent port in a port map to be output.
  • the service may first initialize the output port map so that no ports are designated.
  • the service selects the next designated port of the input port map, beginning with the first.
  • decision block 2602 if all the designated ports have already been selected, then the service completes, else the service continues at block 2603.
  • decision block 2603 if the selected port is available, then the service designates the selected port in the output port map and proceeds to select the next designated port of the input port map.
  • the service in block 2605 retrieves the equivalent port map for the selected port from the equivalent port table.
  • the service selects the next designated port of the selected equivalent port map, beginning with the first.
  • decision block 2607 if all designated ports of the equivalent port map have already been selected, then the service continues at block 2608, else the service continues at block 2609.
  • block 2608 the service designates the selected port in the output port map because no equivalent ports are available and completes. The service may repeat this process as ports become available.
  • decision block 2609 if the selected port of the equivalent port map is available, then the service continues at block 2610, else the service loops to block 2606 to select the next designated port of the equivalent port map.
  • the service designates the selected port of the equivalent port map in the output port map, and then loops to select the next designated port of the input port map.
  • the IFMs allow an existing connection to be preempted when a connection with a higher priority is to be established that conflicts with the existing connection.
  • Figure 27 illustrates an example of preempting a connection.
  • a connection is established between node 2703 and node 2704.
  • the connection includes link 2706, a crosspoint connection between port 0 and port 1 of IFM 2701 , link 2707, a crosspoint connection between port 4 and port 6 of IFM 2702, and link 2708.
  • node 2705 sends a start-of-connection frame with a higher priority than the existing connection and with its preemption bit (flag) set.
  • the SPC for port 2 When the SPC for port 2 receives the frame, it selects the switch destination port through which the connection is to be built.
  • the SPC may use the equivalent port service to identify an equivalent port that is available if the port designated by the port map is in use. If port 1 is designated in the port map and there is no equivalent port that is available, however the SPC'for port 2 detects a conflict.
  • the SPC then sets a flag indicating that the conflicting port (i.e., port 0 or 1 ) should participate in the ensuing arbitration.
  • the SPC for port 2 then sets the arbitration flag and the conflicting port and port 2 participates in the arbitration.
  • the conflicting port loses, its SPC sends a frame through its connection in both directions indicating that the connection is to be removed, and the SPC for port 2 then establishes a crosspoint connection between port 1 and port 2 and transmits its start-of-connection frame. Conversely, if the conflicting port wins the arbitration, then the existing connection remains established, and the SPC for port 2 sends a frame indicating that the connection cannot be established in its input direction.
  • Figure 28 is a diagram illustrating logic of an embodiment of processing a preemption signal. This processing is performed when an SPC detects a preemption signal on the arbitration bus.
  • decision block 2801 if it is determined that the SPCs port is currently in a connection, then the SPC may need to participate in the arbitration.
  • decision block 2802 if this port is the conflicting port (e.g., the port that established the connection), then this port participates in the arbitration and continues processing at block 2803.
  • the SPC for this port participates in the arbitration.
  • decision block 2804 if the SPC of this port loses the arbitration, then it continues at block 2805 to disconnect the connection, else it leaves the connection established.
  • block 2805 the SPC sends a disconnect frame in the direction of its input and output.
  • block 2807 the SPC indicates to the SCU to remove the crosspoint connection for this port.
  • the Fibre Channel standard defines a class 3 protocol that provides a connectionless protocol for transmitting frames without an acknowledgment. Because the protocol is connectionless and no acknowledgment is used, the class 3 protocol can be used for multicasting, that is sending a frame from one node to multiple nodes. Class 3 protocol also specifies that frame delivery is not guaranteed. Traditionally, when a Fibre Channel switch receives a class 3 frame for multicasting, it routes that class 3 frame through as many of the destination ports as are currently available and then discards that frame. In one embodiment, an SPC buffers a class 3 multicasting frame and sends the frame through the multicast ports as they become available.
  • multiframe buffering can be used with communications services other than class 3 of Fibre Channel. In particular, it can be used with any non-acknowledged datagram service, also referred to as a packet service.
  • multiframe buffering can be used to interleave the transmission of a multicast frame with other frames (e.g., connectionless frames).
  • the multiframe buffering algorithm may, for example, give a highest priority score to the multicast frame only when at least one of the multicast ports is currently available.
  • Figure 29 is a diagram illustrating logic of an embodiment of distributed class 3 multicasting. This logic is performed when a class 3 frame with multicasting is received at an SPC.
  • the SPC identifies the multicast ports that are currently available. The multicast ports may be the set of ports indicated by the port map to which a virtual identifier maps.
  • the SPC participates in arbitration.
  • decision block 2903 if the SPC wins the arbitration, then it continues at block 2904, else it continues at block 2901 to again participate in an arbitration.
  • the SPC sends the frame and updates the port map stored in a temporary buffer to reflect those ports through which the frame has been sent.
  • decision block 2905 if the multicast is complete (i.e., the frame has been transmitted through each multicast port), then processing completes, else processing continues to participate in an arbitration to send the frame as additional ports become available. Additional Quality Of Service Information
  • FIG. 30 is a block diagram illustrating an example of node-to-node data communication using QOS communication parameters over a network fabric.
  • a source application 3005 on a source node 3000 generates a request to the local operating system (OS) 3010 on the source node to read data from a destination or to write data to a destination, such as by using the standard application command for a read or a write operation and identifying various application-layer QOS specifications (e.g., via the "Type Of Service" field or priority bits in an IP packet) for the data communication.
  • OS local operating system
  • the OS provides the request to the network driver 3015 (e.g., a VI NIC) of the source node in a standard manner, and the network driver translates the request into appropriate network-layer QOS communication parameters based on the type of the network.
  • the network supports multiple COSes, transmission priorities, and the ability to preempt existing network connections, and the network driver uses some or all of those features to enable the data communication to achieve a desired QOS level (e.g., a level of bandwidth, latency, and/or reliability).
  • the driver builds a network frame that includes selected QOS communication parameters, and transmits the network frame to the network fabric 3040.
  • the network fabric receives the frame and routes it to a destination node 3050 using the QOS parameters embedded in the frame, thus delivering the data with the desired QOS level.
  • a network driver 3065 at the destination translates the embedded network-layer QOS communication parameters into appropriate application-layer QOS specifications for a destination application 3055 (e.g., by converting the network frame into one or more IP packets that include appropriate values), and delivers the translated data to the application via the destination OS 3060.
  • the destination driver and/or destination OS may also utilize the network-layer QOS communication parameters or application-layer QOS specifications to provide QOS- based application notification and data delivery.
  • An example of using QOS communication parameters for a data communication from a source node is illustrated in Figure 31.
  • an application data request 3105 from a source application includes a variety of information about the data request, such as a content type, size, destination indication, etc.
  • This included information is processed by a network frame builder process 3110 (e.g., under the control of the network driver) in order to determine appropriate network-layer QOS communication parameters to use with the data communication.
  • the included information is used to access a provisioning table 3115 (e.g., stored together with other information as a VI translation table) for the source node to determine network-layer QOS communication parameters that are appropriate for the data request.
  • the contents of the provisioning table can be generated in a variety of ways, such as by being managed by a network manager for the network.
  • the frame builder process can use the accessed network-layer QOS communication parameters to generate a network frame 3120, and then forward the generated frame to the network fabric.
  • applications do not need knowledge of network-layer QOS information in order to send or receive data in an appropriate manner. Instead, applications can register one or more data communications (or types of data communication) with an appropriate manager, and a provisioning table on the application's node can be configured with appropriate QOS communication parameters for each data communication (or data communication type).
  • Figure 32 illustrates an example of configuring a node to use QOS communication parameters and of monitoring a network to determine whether to modify those configured communication parameters.
  • a content/services manager 3235 of a network manager 3230 initially receives various general data information for each of various source applications 3210 and/or source nodes 3205 (e.g., services required by or available from each source application or node and/or information that is available from each source application or node), such as during initialization of those source nodes and/or source applications.
  • such information can be provided to the network manager in other ways, such as by being stored in an accessible location (e.g., via an LDAP server).
  • the network manager can use that information to determine QOS levels that are desirable for the provided services and types of information (e.g., by allocating available bandwidth and/or other network resources between the types of information and services). Appropriate QOS communication parameters that will support those QOS levels can then be selected by the network manager. If one or more network monitors 3260 are available for the network, the network manager may also configure the network monitors to monitor for specified types of network information (e.g., QOS levels or QOS communication parameters).
  • the selected QOS communication parameters can then be used by a configuration manager 3240 portion of the network manager to configure node provisioning tables 3215 as appropriate, such as in response to data communication registrations (e.g., requests to supply specified data or to connect to a specified type of service). If the network manager also receives network loading information from the network monitors, the network manager can use that loading information before initial configuration of a provisioning table in order to influence the selection of appropriate QOS communication parameters (or ranges of parameters) and/or after the initial configuration to dynamically update the configured QOS communication parameters in the provisioning table to ensure that desired QOS levels are maintained.
  • data communication registrations e.g., requests to supply specified data or to connect to a specified type of service.
  • data communication registration requests are initially allowed with the lowest possible priority and least overhead COS to provide a desired QOS level and, as network loading changes and/or other requests are serviced, the network manager adjusts (e.g., in a manner transparent to the source node and/or source application) the configured COS and priority values that are used for registered information types.
  • QOS communication parameters can be selected and managed in a variety of ways.
  • one or more applications on the node upon system power-up, initialization, or the attachment of a node to the network, one or more applications on the node send network communications requirements to the network manager using a predefined communications protocol and address.
  • the network manager defines data communication types (e.g., based on types of content or information being requested or provided) and selects appropriate associated QOS communication parameters to support QOS levels desired for those defined data communication types.
  • the selected QOS communication parameters for some or all data communication types are then communicated back to the nodes by the network manager.
  • the node can map the request to one of these defined data communication types and use the associated QOS communication parameters for the communication.
  • the network manager can also configure the one or more network monitors to monitor performance of the network for the QOS communication parameters allocated by the network manager.
  • the QOS communication parameters are provided to nodes in a dynamic manner. For example, when an application on a node wishes to communicate over the network, it may send a data communication registration request (e.g., a request-to-connect for a defined or default information type) to the network manager. The network manager can then determine if the request can be supported, and if so can configure the node's provisioning table with the appropriate QOS parameters and respond to the application with an "authorization to send" confirmation (e.g., an affirmative response message, or the absence of a denial message).
  • a data communication registration request e.g., a request-to-connect for a defined or default information type
  • the network manager can then determine if the request can be supported, and if so can configure the node's provisioning table with the appropriate QOS parameters and respond to the application with an "authorization to send" confirmation (e.g., an affirmative response message, or the absence of a denial message).
  • the network manager can also provide additional information related to the registered data communication in the response message in some embodiments, such as a destination identifier (e.g., a virtual identifier or a virtual address) for use with the data communication.
  • a destination identifier e.g., a virtual identifier or a virtual address
  • the node's network driver builds one or more network frames using appropriate QOS communication parameters from the provisioning table for the data communication type being sent, and sends the frames.
  • the fabric delivers the frames using the embedded QOS communication parameters so as to provide the desired QOS levels for the data communication type.
  • the QOS communication parameters and/or QOS levels to be supported for a data communication or type of data communication are determined as part of a negotiation process between a source application and the network manager. For example, when making a data communication registration request, a source application may request specific desired or required QOS levels and/or QOS communication parameters for the data communication, or ranges of such levels or communication parameters. In response, the network manager may determine whether the requested QOS levels and/or communication parameters can be supported, and may select the QOS levels and/or communication parameters to be enabled for the data communication based in part or entirely on the request.
  • the network manager may deny the request and/or enable alternative QOS levels and/or communication parameters.
  • the source application may be able to accept the alternative values suggested by the network manager and/or continue the negotiation, while in other embodiments the source application may receive a denial of the registration request and opt to make an additional request with less restrictive desired or required QOS values.
  • desired QOS levels are provided by dynamically managing defined QOS communication parameters, such as by modifying QOS communication parameters that are specified in some or all of the network nodes' provisioning tables to reflect a current state of the network (e.g., based on monitoring of the network).
  • a network monitor can report network loading on an operational network (e.g., on a COS and priority basis) to the network manager.
  • the network manager can, for example, increase priority values and/or modify COS types that are associated with that data communication type in nodes' provisioning tables.
  • the network manager can decrease priority and/or COS values for a data communication type.
  • desired QOS levels are provided for a specific data communication and/or type of data communication by dynamically modifying paths to which the data communications of interest are assigned and/or by dynamically limiting other data communications that share some or all of the assigned paths. For example, if other data communications or other network problems are preventing data communications of interest from achieving the desired QOS levels for those data communications of interest, the network manager could dynamically modify the path which the data communications of interest are using (e.g., selecting a new path with increased availability and/or bandwidth due to multiple equivalent ports between intermediate network routing devices along the path), such as in a manner transparent to the source and destination applications participating in the data communications (e.g., by modifying the label tables of the intermediate network routing devices along the path).
  • those other data communications can by dynamically limited in various ways. For example, the paths used by some or all of the other data communications could be changed so that they do not intersect with the paths for the data communications of interests (or so that they do not intersect with portions of those paths with are too congested).
  • the rate or times at which other data communications are allowed to leave a source node and/or to be sent through the network could be controlled in various ways, such as by configuring a source node's provisioning table (or VI translation table) to not allow specified data communications to be sent (e.g., until re-configured, or at specified times) and/or by configuring some or all intermediate network routing devices to filter or delay (e.g., via buffers within the intermediate network routing devices) certain data communications at certain times.
  • additional techniques are used for selecting and managing QOS for the network. These can include defining default QOS communication parameters (e.g., COS and priority values) for some or all defined data communication types, defining increments for raising or lowering values of QOS communication parameters for some or all data communication types, defining thresholds (e.g., network load thresholds) to determine when to raise or lower values of QOS communication parameters for some or all data communication types, defining minimum and/or maximum values for QOS communication parameters for some or all data communication types, and performing network monitoring in such a manner as to provide network loading information for some or all IFM internal links and for end-to- end paths.
  • QOS communication parameters e.g., COS and priority values
  • thresholds e.g., network load thresholds
  • routines discussed above may be provided in alternative ways, such as being split among more routines or consolidated into less routines.
  • illustrated routines may provide more or less functionality than is described, such as when other illustrated routines instead lack or include such functionality respectively, or when the amount of functionality that is provided is altered.
  • data structures discussed above may be structured in different manners, such as by having a single data structure split into multiple data structures or by having multiple data structures consolidated into a single data structure.
  • illustrated data structures may store more or less information than is described, such as when other illustrated data structures instead lack or include such information respectively, or when the amount or types of information that is stored is altered.

Abstract

L'invention concerne des techniques de communication de données dans un réseau, visant à satisfaire à différents niveaux de qualité de service ('QOS'). Les niveaux QOS peuvent être sélectionnés sur la base d'un certain type de données ou de communication de données, par ex. par un gestionnaire de réseau après enregistrement de la communication de données, et les paramètres de communication QOS sont sélectionnés pour la communication de données, afin d'assurer la réalisation des niveaux QOS. D'autres techniques peuvent également être utilisées pour assurer la réalisation d'un niveau QOS sélectionné, ces techniques consistant à sélectionner une voie adéquate dans le réseau, contrôler d'autres communications de données qui utilisent certaines ou toutes les voies sélectionnées, et/ou permettre la préemption d'autres communications de données. La réalisation des niveaux QOS peut également être surveillée, et les paramètres et/ou voies de communication QOS pour une communication de données peuvent être modifiés de manière dynamique, en fonction de différences existant entre des niveaux QOS effectivement atteints et des niveaux QOS souhaités.
PCT/US2002/012428 2001-04-27 2002-04-19 Communication de donnees dans un reseau WO2002088875A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2002303416A AU2002303416A1 (en) 2001-04-27 2002-04-19 Communicating data through a network

Applications Claiming Priority (60)

Application Number Priority Date Filing Date Title
US28692201P 2001-04-27 2001-04-27
US28708101P 2001-04-27 2001-04-27
US28691801P 2001-04-27 2001-04-27
US28707501P 2001-04-27 2001-04-27
US28706901P 2001-04-27 2001-04-27
US28712101P 2001-04-27 2001-04-27
US28712001P 2001-04-27 2001-04-27
US28706801P 2001-04-27 2001-04-27
US60/287,081 2001-04-27
US60/287,069 2001-04-27
US60/287,068 2001-04-27
US60/286,918 2001-04-27
US60/287,120 2001-04-27
US60/287,121 2001-04-27
US60/287,075 2001-04-27
US60/286,922 2001-04-27
US31415801P 2001-08-21 2001-08-21
US31408801P 2001-08-21 2001-08-21
US60/314,158 2001-08-21
US60/314,088 2001-08-21
US31428701P 2001-08-22 2001-08-22
US60/314,287 2001-08-22
US10/044,164 2001-10-26
US10/046,640 US20020159437A1 (en) 2001-04-27 2001-10-26 Method and system for network configuration discovery in a network manager
US10/046,333 US20020188754A1 (en) 2001-04-27 2001-10-26 Method and system for domain addressing in a communications network
US10/046,333 2001-10-26
US10/046,640 2001-10-26
US10/039,505 US20030189927A1 (en) 2001-04-27 2001-10-26 Method and system for multiframe buffering in a routing device
US10/044,182 2001-10-26
US10/061,564 2001-10-26
US10/046,572 2001-10-26
US10/046,572 US20030210685A1 (en) 2001-04-27 2001-10-26 Method and system for interswitch deadlock avoidance in a communications network
US10/068,329 US20020161887A1 (en) 2001-04-27 2001-10-26 Method and system for performing security via de-registration in a communications network
US10/039,784 US6993023B2 (en) 2001-04-27 2001-10-26 Parallel analysis of incoming data transmissions
US10/039,703 US20020159458A1 (en) 2001-04-27 2001-10-26 Method and system for reserved addressing in a communications network
US10/062,245 US20040004966A1 (en) 2001-04-27 2001-10-26 Using virtual identifiers to route transmitted data through a network
US10/046,334 US7068667B2 (en) 2001-04-27 2001-10-26 Method and system for path building in a communications network
US10/062,199 2001-10-26
US10/061,564 US20020159456A1 (en) 2001-04-27 2001-10-26 Method and system for multicasting in a routing device
US10/066,014 US20020159453A1 (en) 2001-04-27 2001-10-26 Method and system for label table caching in a routing device
US10/044,164 US20020167902A1 (en) 2001-04-27 2001-10-26 Method and system for performing security via virtual addressing in a communications network
US10/039,404 2001-10-26
US10/039,703 2001-10-26
US10/039,784 2001-10-26
US10/066,159 2001-10-26
US10/039,814 2001-10-26
US10/066,014 2001-10-26
US10/068,329 2001-10-26
US10/039,404 US6996058B2 (en) 2001-04-27 2001-10-26 Method and system for interswitch load balancing in a communications network
US10/039,505 2001-10-26
US10/066,217 2001-10-26
US10/039,814 US20020161923A1 (en) 2001-04-27 2001-10-26 Method and system for reconfiguring a path in a communications network
US10/066,217 US20020159468A1 (en) 2001-04-27 2001-10-26 Method and system for administrative ports in a routing device
US10/039,877 US20020159389A1 (en) 2001-04-27 2001-10-26 Method and system for connection preemption in a communications network
US10/039,877 2001-10-26
US10/066,159 US7042877B2 (en) 2001-04-27 2001-10-26 Integrated analysis of incoming data transmissions
US10/046,334 2001-10-26
US10/044,182 US20030204618A1 (en) 2001-04-27 2001-10-26 Using virtual identifiers to process received data routed through a network
US10/062,199 US7068666B2 (en) 2001-04-27 2001-10-26 Method and system for virtual addressing in a communications network
US10/062,245 2001-10-26

Publications (2)

Publication Number Publication Date
WO2002088875A2 true WO2002088875A2 (fr) 2002-11-07
WO2002088875A3 WO2002088875A3 (fr) 2003-05-15

Family

ID=27586863

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/US2002/012429 WO2002088981A1 (fr) 2001-04-27 2002-04-19 Analyse de transmissions de donnees entrantes
PCT/US2002/012698 WO2002088876A2 (fr) 2001-04-27 2002-04-19 Procede et systeme conçus pour un adressage virtuel dans un reseau de communications
PCT/US2002/012428 WO2002088875A2 (fr) 2001-04-27 2002-04-19 Communication de donnees dans un reseau

Family Applications Before (2)

Application Number Title Priority Date Filing Date
PCT/US2002/012429 WO2002088981A1 (fr) 2001-04-27 2002-04-19 Analyse de transmissions de donnees entrantes
PCT/US2002/012698 WO2002088876A2 (fr) 2001-04-27 2002-04-19 Procede et systeme conçus pour un adressage virtuel dans un reseau de communications

Country Status (4)

Country Link
EP (1) EP1388075A4 (fr)
JP (3) JP2005502228A (fr)
AU (1) AU2002258931A1 (fr)
WO (3) WO2002088981A1 (fr)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9641432B2 (en) 2013-03-06 2017-05-02 Icu Medical, Inc. Medical device communication method
US9971871B2 (en) 2011-10-21 2018-05-15 Icu Medical, Inc. Medical device update system
US10042986B2 (en) 2013-11-19 2018-08-07 Icu Medical, Inc. Infusion pump automation system and method
US10238801B2 (en) 2009-04-17 2019-03-26 Icu Medical, Inc. System and method for configuring a rule set for medical event management and responses
US10242060B2 (en) 2006-10-16 2019-03-26 Icu Medical, Inc. System and method for comparing and utilizing activity information and configuration information from multiple medical device management systems
US10238799B2 (en) 2014-09-15 2019-03-26 Icu Medical, Inc. Matching delayed infusion auto-programs with manually entered infusion programs
US10311972B2 (en) 2013-11-11 2019-06-04 Icu Medical, Inc. Medical device system performance index
US10314974B2 (en) 2014-06-16 2019-06-11 Icu Medical, Inc. System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy
US10434246B2 (en) 2003-10-07 2019-10-08 Icu Medical, Inc. Medication management system
US10692595B2 (en) 2018-07-26 2020-06-23 Icu Medical, Inc. Drug library dynamic version management
US10741280B2 (en) 2018-07-17 2020-08-11 Icu Medical, Inc. Tagging pump messages with identifiers that facilitate restructuring
US10765799B2 (en) 2013-09-20 2020-09-08 Icu Medical, Inc. Fail-safe drug infusion therapy system
US10861592B2 (en) 2018-07-17 2020-12-08 Icu Medical, Inc. Reducing infusion pump network congestion by staggering updates
US10898641B2 (en) 2014-04-30 2021-01-26 Icu Medical, Inc. Patient care system with conditional alarm forwarding
US11235100B2 (en) 2003-11-13 2022-02-01 Icu Medical, Inc. System for maintaining drug information and communicating with medication delivery devices
US11309070B2 (en) 2018-07-26 2022-04-19 Icu Medical, Inc. Drug library manager with customized worksheets
US11328805B2 (en) 2018-07-17 2022-05-10 Icu Medical, Inc. Reducing infusion pump network congestion by staggering updates
US11571508B2 (en) 2013-08-30 2023-02-07 Icu Medical, Inc. System and method of monitoring and managing a remote infusion regimen
US11574737B2 (en) 2016-07-14 2023-02-07 Icu Medical, Inc. Multi-communication path selection and security system for a medical device
US11587669B2 (en) 2018-07-17 2023-02-21 Icu Medical, Inc. Passing authentication token to authorize access to rest calls via web sockets
US11605468B2 (en) 2015-05-26 2023-03-14 Icu Medical, Inc. Infusion pump system and method with multiple drug library editor source capability

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020159458A1 (en) 2001-04-27 2002-10-31 Foster Michael S. Method and system for reserved addressing in a communications network
JP2007079885A (ja) * 2005-09-14 2007-03-29 Hitachi Ltd データ入出力負荷分散方法、データ入出力負荷分散プログラム、計算機システムおよび管理サーバ
US8982887B2 (en) 2007-05-18 2015-03-17 International Business Machines Corporation System, method and program for making routing decisions
JP5164628B2 (ja) * 2008-03-24 2013-03-21 株式会社日立製作所 ネットワークスイッチ装置、サーバシステム及びサーバシステムにおけるサーバ移送方法
US8914482B2 (en) * 2010-05-26 2014-12-16 Microsoft Corporation Translation of technology-agnostic management commands into multiple management protocols
US9332005B2 (en) * 2011-07-11 2016-05-03 Oracle International Corporation System and method for providing switch based subnet management packet (SMP) traffic protection in a middleware machine environment
AU2014255719B2 (en) * 2013-04-19 2017-04-13 Entuity Limited Identifying an egress port of a device
US8954583B1 (en) 2014-01-20 2015-02-10 Shape Security, Inc. Intercepting and supervising calls to transformed operations and objects
FR3022420B1 (fr) * 2014-06-13 2018-03-23 Bull Sas Methodes et systemes de gestion d'un reseau d'interconnexion
JP6533434B2 (ja) * 2015-08-11 2019-06-19 日本電信電話株式会社 局側光終端装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5781624A (en) * 1996-02-16 1998-07-14 Lucent Technologies Inc. Method for sharing network resources by virtual partitioning
US5790546A (en) * 1994-01-28 1998-08-04 Cabletron Systems, Inc. Method of transmitting data packets in a packet switched communications network
US5892754A (en) * 1996-06-07 1999-04-06 International Business Machines Corporation User controlled adaptive flow control for packet networks
US6091709A (en) * 1997-11-25 2000-07-18 International Business Machines Corporation Quality of service management for packet switched networks

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5250943A (en) * 1991-03-29 1993-10-05 International Business Machines Corporation GVT-NET--A Global Virtual Time Calculation Apparatus for Multi-Stage Networks
JP3071007B2 (ja) * 1991-10-22 2000-07-31 富士通株式会社 通信ネットワーク制御方式
JPH0795245A (ja) * 1993-09-24 1995-04-07 Toshiba Corp 網間接続方法及びこれを用いた通信システム
JPH07283820A (ja) * 1994-04-13 1995-10-27 Nec Corp パス設定システム
US5774067A (en) * 1995-06-07 1998-06-30 International Business Machines Corporation Flash-flooding multi-stage interconnection network with parallel path seeking switching elements
US5689506A (en) * 1996-01-16 1997-11-18 Lucent Technologies Inc. Multicast routing in multistage networks
US5892766A (en) * 1996-02-22 1999-04-06 Fujitsu, Ltd. Method and apparatus for coordinating access to an output of a routing device in a packet switching network
US5940596A (en) * 1996-03-25 1999-08-17 I-Cube, Inc. Clustered address caching system for a network switch
US5917820A (en) * 1996-06-10 1999-06-29 Cisco Technology, Inc. Efficient packet forwarding arrangement for routing packets in an internetwork
US6147976A (en) * 1996-06-24 2000-11-14 Cabletron Systems, Inc. Fast network layer packet filter
US5872783A (en) * 1996-07-24 1999-02-16 Cisco Systems, Inc. Arrangement for rendering forwarding decisions for packets transferred among network switches
JPH1084385A (ja) * 1996-09-10 1998-03-31 Nippon Telegr & Teleph Corp <Ntt> 複数サーバとの通信接続装置及び通信接続方法
JP3579208B2 (ja) * 1997-03-11 2004-10-20 株式会社東芝 ノード装置及びメッセージ交換方法
US6195335B1 (en) * 1997-06-27 2001-02-27 International Business Machines Corporation Data switch
US6006264A (en) 1997-08-01 1999-12-21 Arrowpoint Communications, Inc. Method and system for directing a flow between a client and a server
US6078963A (en) * 1998-01-16 2000-06-20 At&T Corp. Router with de-centralized processing using intelligent ports
US6216173B1 (en) * 1998-02-03 2001-04-10 Redbox Technologies Limited Method and apparatus for content processing and routing
US5999531A (en) * 1998-04-17 1999-12-07 Cabletron Systems, Inc. Method and system for identifying ports and forwarding packets in a multiport switch
JP3633321B2 (ja) * 1998-10-23 2005-03-30 富士通株式会社 広域負荷分散装置及び方法
JP2000183976A (ja) * 1998-12-21 2000-06-30 Matsushita Electric Ind Co Ltd 集線／変換装置
JP4189076B2 (ja) * 1999-03-26 2008-12-03 株式会社東芝 耐障害コンピュータシステム
JP3670157B2 (ja) * 1999-03-31 2005-07-13 三菱電機株式会社 Ｉｐパケット転送システム
US6779016B1 (en) 1999-08-23 2004-08-17 Terraspring, Inc. Extensible computing system
US6598034B1 (en) * 1999-09-21 2003-07-22 Infineon Technologies North America Corp. Rule based IP data processing
CN1415159A (zh) * 1999-09-24 2003-04-30 戴尔帕德通讯公司 可伸缩的通信系统
US6381242B1 (en) * 2000-08-29 2002-04-30 Netrake Corporation Content processor

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5790546A (en) * 1994-01-28 1998-08-04 Cabletron Systems, Inc. Method of transmitting data packets in a packet switched communications network
US5781624A (en) * 1996-02-16 1998-07-14 Lucent Technologies Inc. Method for sharing network resources by virtual partitioning
US5892754A (en) * 1996-06-07 1999-04-06 International Business Machines Corporation User controlled adaptive flow control for packet networks
US6091709A (en) * 1997-11-25 2000-07-18 International Business Machines Corporation Quality of service management for packet switched networks

Cited By (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10434246B2 (en) 2003-10-07 2019-10-08 Icu Medical, Inc. Medication management system
US11235100B2 (en) 2003-11-13 2022-02-01 Icu Medical, Inc. System for maintaining drug information and communicating with medication delivery devices
US10242060B2 (en) 2006-10-16 2019-03-26 Icu Medical, Inc. System and method for comparing and utilizing activity information and configuration information from multiple medical device management systems
US11194810B2 (en) 2006-10-16 2021-12-07 Icu Medical, Inc. System and method for comparing and utilizing activity information and configuration information from multiple device management systems
US11013861B2 (en) 2009-04-17 2021-05-25 Icu Medical, Inc. System and method for configuring a rule set for medical event management and responses
US10238801B2 (en) 2009-04-17 2019-03-26 Icu Medical, Inc. System and method for configuring a rule set for medical event management and responses
US11654237B2 (en) 2009-04-17 2023-05-23 Icu Medical, Inc. System and method for configuring a rule set for medical event management and responses
US9971871B2 (en) 2011-10-21 2018-05-15 Icu Medical, Inc. Medical device update system
US11626205B2 (en) 2011-10-21 2023-04-11 Icu Medical, Inc. Medical device update system
US9641432B2 (en) 2013-03-06 2017-05-02 Icu Medical, Inc. Medical device communication method
US11470000B2 (en) 2013-03-06 2022-10-11 Icu Medical, Inc. Medical device communication method
US10333843B2 (en) 2013-03-06 2019-06-25 Icu Medical, Inc. Medical device communication method
US11571508B2 (en) 2013-08-30 2023-02-07 Icu Medical, Inc. System and method of monitoring and managing a remote infusion regimen
US10765799B2 (en) 2013-09-20 2020-09-08 Icu Medical, Inc. Fail-safe drug infusion therapy system
US10311972B2 (en) 2013-11-11 2019-06-04 Icu Medical, Inc. Medical device system performance index
US11501877B2 (en) 2013-11-11 2022-11-15 Icu Medical, Inc. Medical device system performance index
US11763927B2 (en) 2013-11-19 2023-09-19 Icu Medical, Inc. Infusion pump automation system and method
US10042986B2 (en) 2013-11-19 2018-08-07 Icu Medical, Inc. Infusion pump automation system and method
US11037668B2 (en) 2013-11-19 2021-06-15 Icu Medical, Inc. Infusion pump automation system and method
US10898641B2 (en) 2014-04-30 2021-01-26 Icu Medical, Inc. Patient care system with conditional alarm forwarding
US11628246B2 (en) 2014-04-30 2023-04-18 Icu Medical, Inc. Patient care system with conditional alarm forwarding
US10314974B2 (en) 2014-06-16 2019-06-11 Icu Medical, Inc. System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy
US11628254B2 (en) 2014-06-16 2023-04-18 Icu Medical, Inc. System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy
US10646651B2 (en) 2014-06-16 2020-05-12 Icu Medical, Inc. System for monitoring and delivering medication to a patient and method of using the same to minimize the risks associated with automated therapy
US11574721B2 (en) 2014-09-15 2023-02-07 Icu Medical, Inc. Matching delayed infusion auto-programs with manually entered infusion programs
US10799632B2 (en) 2014-09-15 2020-10-13 Icu Medical, Inc. Matching delayed infusion auto-programs with manually entered infusion programs
US11289183B2 (en) 2014-09-15 2022-03-29 Icu Medical, Inc. Matching delayed infusion auto-programs with manually entered infusion programs
US10238799B2 (en) 2014-09-15 2019-03-26 Icu Medical, Inc. Matching delayed infusion auto-programs with manually entered infusion programs
US11605468B2 (en) 2015-05-26 2023-03-14 Icu Medical, Inc. Infusion pump system and method with multiple drug library editor source capability
US11574737B2 (en) 2016-07-14 2023-02-07 Icu Medical, Inc. Multi-communication path selection and security system for a medical device
US10741280B2 (en) 2018-07-17 2020-08-11 Icu Medical, Inc. Tagging pump messages with identifiers that facilitate restructuring
US11152110B2 (en) 2018-07-17 2021-10-19 Icu Medical, Inc. Tagging pump messages with identifiers that facilitate restructuring
US11923076B2 (en) 2018-07-17 2024-03-05 Icu Medical, Inc. Converting pump messages in new pump protocol to standardized dataset messages
US11328804B2 (en) 2018-07-17 2022-05-10 Icu Medical, Inc. Health checks for infusion pump communications systems
US11483403B2 (en) 2018-07-17 2022-10-25 Icu Medical, Inc. Maintaining clinical messaging during network instability
US11483402B2 (en) 2018-07-17 2022-10-25 Icu Medical, Inc. Maintaining clinical messaging during an internet outage
US11328805B2 (en) 2018-07-17 2022-05-10 Icu Medical, Inc. Reducing infusion pump network congestion by staggering updates
US11881297B2 (en) 2018-07-17 2024-01-23 Icu Medical, Inc. Reducing infusion pump network congestion by staggering updates
US11152108B2 (en) 2018-07-17 2021-10-19 Icu Medical, Inc. Passing authentication token to authorize access to rest calls via web sockets
US11152109B2 (en) 2018-07-17 2021-10-19 Icu Medical, Inc. Detecting missing messages from clinical environment
US11587669B2 (en) 2018-07-17 2023-02-21 Icu Medical, Inc. Passing authentication token to authorize access to rest calls via web sockets
US11373753B2 (en) 2018-07-17 2022-06-28 Icu Medical, Inc. Converting pump messages in new pump protocol to standardized dataset messages
US11139058B2 (en) 2018-07-17 2021-10-05 Icu Medical, Inc. Reducing file transfer between cloud environment and infusion pumps
US10964428B2 (en) 2018-07-17 2021-03-30 Icu Medical, Inc. Merging messages into cache and generating user interface using the cache
US10950339B2 (en) 2018-07-17 2021-03-16 Icu Medical, Inc. Converting pump messages in new pump protocol to standardized dataset messages
US10861592B2 (en) 2018-07-17 2020-12-08 Icu Medical, Inc. Reducing infusion pump network congestion by staggering updates
US11670416B2 (en) 2018-07-17 2023-06-06 Icu Medical, Inc. Tagging pump messages with identifiers that facilitate restructuring
US11783935B2 (en) 2018-07-17 2023-10-10 Icu Medical, Inc. Health checks for infusion pump communications systems
US10692595B2 (en) 2018-07-26 2020-06-23 Icu Medical, Inc. Drug library dynamic version management
US11309070B2 (en) 2018-07-26 2022-04-19 Icu Medical, Inc. Drug library manager with customized worksheets
US11437132B2 (en) 2018-07-26 2022-09-06 Icu Medical, Inc. Drug library dynamic version management

Also Published As

Publication number Publication date
JP2004537881A (ja) 2004-12-16
JP2007166666A (ja) 2007-06-28
WO2002088876A3 (fr) 2003-10-30
WO2002088981A1 (fr) 2002-11-07
WO2002088875A3 (fr) 2003-05-15
EP1388075A4 (fr) 2008-01-16
EP1388075A1 (fr) 2004-02-11
WO2002088876A2 (fr) 2002-11-07
JP2005502228A (ja) 2005-01-20
AU2002258931A1 (en) 2002-11-11

Similar Documents

Publication Publication Date Title
US7164656B2 (en) Communicating data through a network so as to ensure quality of service
WO2002088875A2 (fr) Communication de donnees dans un reseau
US6529963B1 (en) Methods and apparatus for interconnecting independent fibre channel fabrics
US9270639B2 (en) Load balancing among a cluster of firewall security devices
US6049528A (en) Trunking ethernet-compatible networks
US9143390B2 (en) Method and system for a unified system management
US20030210685A1 (en) Method and system for interswitch deadlock avoidance in a communications network
US20050025051A1 (en) Automated router load balancing
US8151318B1 (en) Method and apparatus for reliably and asymmetrically distributing security information within a fibre channel fabric
EP1244265A2 (fr) Service d&#39;implémentation de politique intégrée pour réseau de communication
JP5261432B2 (ja) 通信システム、パケット転送方法、ネットワーク交換装置、アクセス制御装置、及びプログラム
US11171883B1 (en) Peering-fabric routing using switches having dynamically configurable forwarding logic
CN113853776B (zh) 用于网络架构的方法、系统和计算机可读介质
US20040111605A1 (en) Method for authenticating multiple channels within a single fibre channel link
US7392318B1 (en) Method and system for balancing a traffic load in a half-duplex environment
KR100431207B1 (ko) 엠피엘에스（ｍｐｌｓ）기반망에서의 엑스트라넷아이피-브이피엔（ｉｐ－ｖｐｎ）서비스 제공 방법
WO2022161288A1 (fr) Procédé et appareil d&#39;envoi de trafic, système, et support de stockage
WO2023041923A1 (fr) Procédé et dispositif de transfert de données unidirectionnel
EP1998509A1 (fr) Transparent backup IP router dans un réseau local
KR100702783B1 (ko) 동일 ｍａｃ 주소를 갖는 가입자 단말 처리 시스템 및 처리방법
WO2002089384A2 (fr) Utilisation d&#39;identificateurs virtuels pour l&#39;acheminement de donnees, et traitement de donnees acheminees sur un reseau

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP