US20050141506A1 - Methods, systems and computer program products for cluster workload distribution - Google Patents
Methods, systems and computer program products for cluster workload distribution Download PDFInfo
- Publication number
- US20050141506A1 US20050141506A1 US11/045,633 US4563305A US2005141506A1 US 20050141506 A1 US20050141506 A1 US 20050141506A1 US 4563305 A US4563305 A US 4563305A US 2005141506 A1 US2005141506 A1 US 2005141506A1
- Authority
- US
- United States
- Prior art keywords
- communication protocol
- protocol stack
- routing
- address
- stack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1017—Server selection for load balancing based on a round robin mechanism
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/10015—Access to distributed or replicated servers, e.g. using brokers
Definitions
- the present invention relates to network communications and more particularly to network communications to a cluster of data processing systems.
- IP Internet Protocol
- IP Internet Protocol
- All physical adapter devices in such a network including those for client and server hosts, are identified by an IP Address which is unique within the network.
- IP Address is unique within the network.
- One valuable feature of IP is that a failure of an intermediate router node or adapter will not prevent a packet from moving from source to destination, as.long as there is an alternate path through the network.
- TCP Transmission Control Protocol/Internet Protocol
- IP Transmission Control Protocol/Internet Protocol
- VIPA Virtual IP Address
- TCP/IP Transmission Control Protocol/IP
- V2R5 Virtual IP Address
- Examples of VIPAs and their user may be found in U.S. Pat. Nos. 5,917,997, 5,923,854, 5,935,215 and 5,951,650.
- a VIPA is configured the same as a normal IP address for a physical adapter, except that it is not associated with any particular device.
- the TCP/IP stack on System/390 simply looks like another router.
- the inbound IP function of the TCP/IP stack notes that the IP address of the packet is in the TCP/IP stack's Home list of IP addresses and forwards the packet up the TCP/IP stack.
- the “home list” of a TCP/IP stack is the list of IP addresses which are “owned” by the TCP/IP stack. Assuming the TCP/IP stack has multiple adapters or paths to it (including a Cross Coupling Facility (XCF) path from other TCP/IP stacks in a Sysplex), if a particular physical adapter fails, the attached routing network will route VIPA-targeted packets to the TCP/IP stack via an alternate route.
- the VIPA may, thus, be thought of as an address to the stack, and not to any particular adapter.
- VIPAs may remove hardware and associated transmission media as a single point of failure for large numbers of connections, the connectivity of a server can still be lost through a failure of a single stack or an MVS image.
- the VIPA Configuration manual for System/390 tells the customer how to configure the VIPA(s) for a failed stack on another stack, but this is a manual process. Substantial down time of a failed MVS image or TCP/IP stack may still result until operator intervention to manually reconfigure the TCP/IP stacks in a Sysplex to route around the failed TCP/IP stack or MVS image.
- DNS Dynamic Name Server
- WLM DNS/Workload Manager
- DNS/WLM may prefer to configure DNS/WLM to send a Time to Live that is greater than zero, in an attempt to limit network-wide traffic to resolve names. Problems arising from these various scenarios may be reduced if the IP address with which the client communicates does not change. However, as described above, to affect such a movement of VIPAs between TCP/IP stacks requires operator intervention and may result in lengthy down times for the applications associated with the VIPA.
- HACMP High-Availability Coupled Multi-Processor
- Automatic restart of failing applications also provides faster recovery of a failing application or node. This may be acceptable when the application can be restarted in place, but is less useful when the application is moved to another node, unless the IP address known to the clients can be moved with the application, or dynamic DNS updates with alternate IP addresses can be propagated to a DNS local to clients sufficiently quickly.
- IP Address Migration Application controls all IP addresses in the cluster.
- the cluster is connected via a shared-medium LAN.
- IP address aliasing is used to provide addresses to individual applications over a single adapter, and these aliases are located via Address Resolution Protocol (ARP) and ARP caches in the TCP/IPs.
- ARP Address Resolution Protocol
- the application monitors all server applications and hardware, and reallocates aliased IP addresses in the event of failure to surviving adapters and nodes.
- This approach allows applications of a failing node to be distributed among surviving nodes, but it may require the monitoring application to have complete knowledge of the application and network adapter topology in the cluster. In this sense, it is similar to existing Systems Management applications such as those provided by International Business Machines Corporation's Tivoli® network management software, but the IP Address Migration Application has direct access to adapters and ARP caches.
- the application also requires a dedicated IP address for inter-application communication and coordination.
- VIPA dynamic virtual IP addresses
- a dynamic VIPA may be automatically moved from protocol stack to protocol stack in a predefined manner to overcome failures of a particular protocol stack (i.e. VIPA takeover). Such a predefined movement may provide a predefined backup protocol stack for a particular VIPA.
- VIPA takeover was made available by International Business Machines Corporation (IBM), Armonk, N.Y., in System/390 V2R8 which had a general availability date of September, 1999.
- scalability and load balancing are also issues which have received considerable attention in light of the expansion of the Internet. For example, it may be desirable to have multiple servers servicing customers. The workload of such servers may be balanced by providing a single network visible IP address which is mapped to multiple servers.
- Such a mapping process may be achieved by, for example, network address translation (NAT) facilities, dispatcher systems and IBM's Dynamic Name Server/Workload Management DNS/WLM systems. These various mechanisms for allowing multiple servers to share a single IP address are illustrated in FIGS. 1 through 3 .
- NAT network address translation
- dispatcher systems and IBM's Dynamic Name Server/Workload Management DNS/WLM systems.
- FIG. 1 illustrates a conventional network address translation system as described above.
- a client 10 communicates over a network 12 to a network address translation system 14 .
- the network address translation system receives the communications from the client 10 and converts the communications from the addressing scheme of the network 12 to the addressing scheme of the network 12 ′ and sends the messages to the servers 16 .
- a server 16 may be selected from multiple servers 16 at connect time and may be on any host, one or more hops away. All inbound and outbound traffic flows through the NAT system 14 .
- FIG. 2 illustrates a conventional DNS/WLM system as described above.
- the server 16 is selected at name resolution time when the client 10 resolves the name for the destination server from DNS/WLM system 17 which is connected to the servers 16 through the coupling facility 19 .
- the DNS/WLM system of FIG. 2 relies on the client 10 adhering to the zero time to live.
- FIG. 3 illustrates a conventional dispatcher system.
- the client 10 communicates over the network 12 with a dispatcher system 18 to establish a connection.
- the dispatcher routes inbound packets to the servers 16 and outbound packets are sent over network 12 ′ but may flow over any available path to the client 10 .
- the servers 16 are typically on a directly connected network to the dispatcher 18 and a server 16 is selected at connect time.
- Such a dispatcher system is illustrated by the Interactive Network Dispatcher function of the IBM 2216 and AIX platforms.
- the same IP address that the Network Dispatcher node 18 advertises to the routing network 12 is activated on server nodes 16 as loopback addresses.
- the node performing the distribution function connects to the endpoint stack via a single hop connection because normal routing protocols typically cannot be used to get a connection request from the endpoint to the distributing node if the endpoint uses the same IP address as the distributing node advertises.
- Network Dispatcher uses an application on the server to query a workload management function (such as WLM of System/390), and collects this information at intervals, e.g. 30 seconds or so.
- Applications running on the Network Dispatcher node can also issue “null” queries to selected application server instances as a means of determining server instance health.
- Cisco Systems offers a Multi-Node Load Balancing function on certain of its routers that perform the distribution function. Such operations appear similar to those of the IBM 2216.
- AceDirector from Alteon provides a virtual IP address and performs network address translation to a real address of a selected server application. AceDirector appears to observe connection request turnaround times and rejection as a mechanism for determining server load capabilities.
- Methods, systems and computer program products provide communications with a plurality of application instances executing on a cluster of data processing systems having a plurality of communication protocol stacks associated therewith utilizing a single Internet Protocol (IP) address.
- IP Internet Protocol
- a first of the plurality of communication protocol stacks is established as a routing communication protocol stack associated with the single IP address.
- Ones of the plurality of communication protocol stacks which are associated with the single IP address are defined as candidate target protocol stacks and an identification of candidate target protocol stacks and the routing communication protocol stack is distributed to the communication protocol stacks.
- the routing communication protocol stack is notified when an instance of the plurality of application instances associated with a candidate target communication protocol stack listens to a port of the single IP address to establish a current actual target stack.
- a request to establish a connection to the single IP address and the port of the single IP address is received and a routing table entry corresponding to the current actual target stack is created to provide a routing path associated with the IP address and the port from the routing communication protocol stack to the current actual target stack. Communications to the port of the IP address received by the routing communication protocol stack are routed based on the routing table.
- all of the plurality of communication protocol stacks of the cluster of data processing systems are defined as associated with the single IP address.
- enumerated ones of the plurality of communication protocol stacks of the cluster of data processing systems may be defined as associated with the single IP address.
- a first of the plurality of. communication protocol stacks is established as a routing communication protocol stack associated with the single IP address for routing communications associated with at least one specified port associated with the IP address.
- Ones of the plurality of communication protocol stacks which are associated with the at least one specified port associated with the IP address are defined as candidate target communication protocol stacks and an identification of the candidate target protocol stacks and the routing communication protocol stack distributed to the plurality of communication protocol stacks.
- the routing communication protocol stack is notified when an instance of the plurality of application instances associated with a candidate target communication protocol stack listens to the at least one port of the single IP address.
- the at least one port may be a plurality of ports identified in a port list associated with the single IP address. Furthermore, the least one port may be all ports associated with-the single IP address.
- the first communication protocol stack is notified that the instance of the plurality of application instances associated with a candidate target communication protocol stack has terminated listening to the port of the single IP address.
- the routing table entry corresponding to the candidate target communication protocol stack may be removed so as to remove the routing path associated with the IP address, the port and the candidate target communication protocol stack.
- Such notification may be provided by sending a termination message through a cross coupling facility of the cluster of data processing systems.
- the routing communication protocol stack is established and the plurality of communication protocol stacks defined by incorporating a VIPADISTribute statement in a VIPADynamic definition block associated with the first communication protocol stack.
- the VIPADISTribute statement defines a IP addressas a dynamic routable virtual IP address (VIPA), identifies ports associated with the VIPA which are routable, and identifies communication protocols stacks associated with the VIPA.
- VIPA dynamic routable virtual IP address
- the distribution of an identification of protocol stacks which are associated with the single IP address and an identification of the first of the plurality of communication protocol stacks may be accomplished by broadcasting a list including the definitions of the VIPADISTRibute statement.
- the routing communication protocol stack is notified that a connection utilizing the single IP address and the port of the singe IP address has terminated.
- the routing table entry corresponding to the connection to the candidate target communication protocol stack associated with the connection is then removed so as to remove the routing path associated with the IP address, the port and the second communication protocol stack.
- routing communication protocol stack when a connection request is received by the routing communication protocol stack, one of the communication protocol stacks identified as having an application instance bound to the single IP address and listening to the port associated with the request is selected.
- the routing path may then be incorporated in the routing table for the selected communication protocol stack.
- systems, methods and computer program products are provided for communicating with a plurality of application instances executing on a cluster of data processing systems having a plurality of communication protocol stacks associated therewith utilizing a single IP address may be provided by a routing communication protocol stack configured to receive communications to the single IP address and to forward to the received communications to a selected communication protocol stack from the plurality of communication protocol stacks.
- a destination port table associated with the routing communication protocol stack which identifies communication protocol stacks having application instances bound to the single IP address and listening to a predefined port of the single IP address.
- a current routing table associated with the routing communication protocol stack which identifies routing paths from the routing communication protocol stack to a communication protocol stack associated with a connection utilizing the single IP address and the predefined port.
- the routing communication protocol stack is further configured to build the destination port table based on status messages received from communication protocol stacks of the plurality of communication protocol stacks having application instances bound to the single IP address and listening to a predefined port of the single IP address in response to distribution by the routing communication protocol stack of an identification of the single IP address and the predefined port as a routable IP address.
- the routing communication protocol stack may be further configured to incorporate into the current routing table routing paths associated with new connection messages received by the routing communication.protocol stack from communication protocol stacks which initiate connections utilizing the single IP address and the predefined port as a source address.
- the routing communication protocol stack is further configured to receive requests for connections to the single IP address and the predefined port, select communication protocol stacks for the connections from the destination port table and update the current routing table with path information associated with the connections and the selected communication protocol stacks.
- the routing communication protocol stack may also be further configured to receive connection termination messages from communication protocol stacks which detect termination of connections utilizing the single IP address and predefined port and to update the current routing table to remove routing paths associated with the terminated connections.
- the predefined port may be a plurality of predefined ports identified in a port list associated with the single IP address or the predefined port may be all ports associated with the single IP address.
- the present invention may be embodied as methods, apparatus/systems and/or computer program products.
- FIG. 1 is block diagram of a conventional network address translation system
- FIG. 2 is block diagram of a conventional DNS/WLM system
- FIG. 3 is block diagram of a conventional dispatcher system
- FIG. 4 is block diagram of a cluster of data processing systems incorporating embodiments of the present invention.
- FIG. 5 is a flowchart illustrating operations for initialization of a routing protocol stack incorporating distributable VIPAs according to embodiments of the present invention
- FIG. 6 is a flowchart illustrating operations of a server protocol stack according to embodiments of the present invention.
- FIG. 7 is a flowchart illustrating operations for a incoming communications to the routing protocol stack according to embodiments of the present invention.
- FIG. 8 is a flowchart illustrating operations of a routing protocol stack receiving communications from another protocol stack according to embodiments of the present invention.
- FIG. 9 is a flowchart illustrating operations of protocol stacks during failure of a routing protocol stack according to embodiments of the present invention.
- FIG. 10 is a flowchart illustrating operations of protocol stacks according to embodiments of the present invention incorporating moveable VIPAs for recovery of a failed routing protocol stack;
- FIG. 11 is a flowchart illustrating operations of a communication protocol stack which owns a moveable VIPA.
- FIG. 12 is a flowchart illustrating operations of a communications protocol stack in response to loss of ownership of a moveable VIPA.
- the present invention can take the form of an entirely hardware embodiment, an entirely software (including firmware, resident software, micro-code, etc.) embodiment, or an embodiment containing both software and hardware aspects.
- the present invention can take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code means embodied in the medium for use by or in connection with an instruction execution system.
- a computer-usable or computer-readable medium can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- the computer-usable or computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection having one or more wires, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM).
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- CD-ROM portable compact disc read-only memory
- the computer-usable or computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
- the present invention can be embodied as systems, methods, or computer program products which allow for a single IP address being associated with a plurality of communication protocol stacks in a cluster of data processing systems by providing a routing protocol stack which associates a Virtual IP Address (VIPA) and port with other communication protocol stacks in the cluster and routes communications to the VIPA and port to the appropriate communication protocol stack.
- VIPAs capable of being shared by a number of communication protocol stacks are referred to herein as “dynamic routable VIPAs” (DVIPA).
- a cluster of data processing systems is illustrated in FIG. 4 as a cluster nodes in Sysplex 10 .
- the data processing systems 20 , 24 , 28 , 32 and 36 illustrated in FIG. 4 may be operating system images, such as MVS images, executing on one or more computer systems. While the present invention will be described primarily with respect to the MVS operating system executing in a System/390 environment, the data processing systems 20 , 24 , 28 , 32 and 36 may be mainframe computers, mid-range computers, servers or other systems capable of supporting dynamic routable Virtual IP Addresses and which are capable of error recovery as described herein.
- the data processing systems 20 , 24 , 28 , 32 and 36 have associated with them communication protocol stacks 22 , 26 , 30 , 34 and 38 , which may be TCP/IP stacks.
- the communication protocol stacks 22 , 26 , 30 , 34 and 38 have been modified to incorporate a VIPA distribution function 23 as described herein for providing dynamic routable VIPAs so as to provide a single IP address for multiple communication protocol stacks.
- each of the communication protocol stacks 22 , 26 , 30 , 34 and 38 illustrated in FIG. 4 incorporate the VIPA distribution function 23
- the present invention may be carried out on any system where two or more communication protocol stacks in a cluster of data processing systems support dynamic routable VIPAs. If a communication protocol stack does not support dynamic routable VIPA, then the dynamic routable VIPA messages according to the present invention would be ignored by the communication protocol stack. Thus, the present invention provides backward compatibility with existing communication protocol stacks.
- the communication protocol stacks 22 , 26 , 30 , 34 and 38 may communicate with each other through a coupling facility 40 of the Sysplex 10 , for example, utilizing XCF messaging. Furthermore, the communication protocol stacks 22 and 38 may communicate with an external network 44 such as the Internet, an intranet, a Local Area Network (LAN) or Wide Area Network (WAN) utilizing the Enterprise System Connectivity (ESCON) 42 . Thus, a client 46 may,utilize network 44 to communicate with an application executing on an MVS image in Sysplex 10 through the communication protocol stacks 22 and 38 which may function as routing protocol stacks as described herein.
- LAN Local Area Network
- WAN Wide Area Network
- ESCON Enterprise System Connectivity
- data processing system 20 has associated with it communication protocol stack 22 which is associated with MVS image MVS 1 which has application APP A executing on MVS image MVS 1 and utilizing communication protocol stack 22 to allow access to, for example, client 46 through network 44 .
- data processing system 24 has associated with it communication protocol stack 26 which is associated with MVS image MVS 2 which has a second instance of application APP A and an instance of application APP B executing on MVS image MVS 2 which may utilize communication protocol stack 26 for communications.
- Data processing system 28 has associated with it communication protocol stack 30 which is associated with MVS image MVS 3 which has a second instance of application APP B executing on MVS image MVS 3 which may utilize communication protocol stack 30 for communications.
- Data processing system 32 has associated with it communication protocol stack 34 which is associated with MVS image MVS 4 which has a third instance of application APP A executing on MVS image MVS 4 which may utilize communication protocol stack 34 for communications.
- data processing system 36 has associated with it communication protocol stack 38 which is associated with MVS image MVS 5 which has a third instance of application APP B executing on MVS image MVS 5 which may utilize communication protocol stack 38 for communications.
- the VIPA distribution function 23 allows for protocol stacks which are defined as supporting DVIPAs to share the DVIPA and communicate with network 44 through a routing protocol stack such that all protocol stacks having a server application which is associated with the DVIPA will appear to the network 44 as a single IP address.
- Such dynamically routable VIPAs may be provided by designating a protocol stack, such as protocol stack 22 , as a routing protocol stack, notifying other protocol stacks of the routing protocol stack and having other protocol stacks notify the routing protocol stack when an application which binds to the DVIPA is started.
- the routing protocol stack may provide work load balancing by distributing connections to the other protocol stacks on MVS images executing server applications which bind to the DVIPA to balance workload. Furthermore, in particular embodiments of the present invention, scalability and availability may be provided by allowing all protocol stacks for MVS images which execute applications which bind to the DVIPA to have communications routed through the routing protocol stack without user intervention to establish the routing path.
- VIPA distribution function 23 allows automated movement of a routing protocol function to a backup stack. Another aspect of the VIPA distribution function 23 allows recovery of a failed routing stack without disruption to connections through the backup stack.
- the communication protocol stacks 22 , 26 , 30 , 34 and 38 may be configured as to which stacks are routing stacks, backup routing stacks and server stacks. Different DVIPAs may have different sets of backup stacks, possibly overlapping.
- the definition of backup stacks may be the same as that for the VIPA takeover function described in U.S. patent application Ser. No. 09/401,419, entitled “METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS FOR AUTOMATED MOVEMENT OF IP ADDRESSES WITHIN A CLUSTER” which is incorporated herein by reference as if set forth fully herein.
- DVIPAs In providing for DVIPAs, up to five or more aspects of DVIPA operation may be addressed: 1) initialization and definition of DVIPAs and the affected protocol stacks; 2) incoming communications from network 44 to the DVIPA; 3) connections originated by a protocol stack (i.e. outgoing to network 44 ); 4)failure of a routing stack; and 5) recovery of a routing stack.
- application APP A is associated with DVIPA VA 1 which may be associated with the respective first, second and third instances of APP A; and application APP B likewise has DVIPA VB 1 associated with the respective first, second and third instances of APP B.
- Configuration of a dynamic routable VIPA may be provided by a new definition block established by a system administrator for each routing communication protocol stack 22 and 38 .
- the new definition block defines dynamic routable VIPAs for which a communication protocol stack operates as the primary communication protocol stack.
- Backup protocol stacks may be defined as described of the VIPA takeover procedure.
- a definition block “VIPADynamic” may be defined as
- the definitions within the VIPADynamic block for a protocol stack supporting moveable VIPAs are:
- the definitions in the VIPADYNamic block for defining a VIPA as a dynamic routable VIPA are:
- a range of IP addresses may be defined as DVIPAs utilizing the VIPARange definition.
- VIPARange definition of the form:
- the communication protocol stacks 22 and 38 which are designated as routing protocol stacks as they have connections to the network 44 and include VIPADISTribute statements in the VIPADynamic block, publish the distribution information through messages broadcast by the VIPA takeover function 23 of each of the communication protocol stacks 22 , 26 , 30 , 34 and 38 to the other communication protocol stacks 22 , 26 , 30 , 34 and 38 .
- the communication protocol stacks 22 , 26 , 30 , 34 and 38 communicate to all partner communication protocol stacks the complete list of dynamic routable VIPAs, their associated ipAddrList and portlist and the primary and backup definitions for the communication protocol stack.
- a communication protocol stack 22 , 26 , 30 , 34 and 38 When a communication protocol stack 22 , 26 , 30 , 34 and 38 receives the DVIPA information it notes if it is identified as a candidate target protocol stack or as a backup stack. If the protocol stack is a candidate target stack, it monitors its applications and sends a message to the defined routing stack when an application instance is bound to the DVIPA and listens on a defined port. If the protocol stack is a backup stack it stores the DVIPA information for use in the event of failure of the primary routing stack.
- the VIPADEFine statements may be:
- the VIPABackup statements for MVS 1 and MVS 5 of FIG. 4 may be:
- FIGS. 5 through 12 are flowchart illustrations of operations of protocol stacks incorporating embodiments of the present invention.
- each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams can be implemented by computer program instructions.
- These program instructions may be provided to a processor to produce a machine, such that the instructions which execute on the processor create means for implementing the functions specified in the flowchart and/or block diagram block or blocks.
- the computer program instructions may be executed by a processor to cause a series of operational steps to be performed by the processor to produce a computer implemented process such that the instructions which execute on the processor provide steps for implementing the functions specified in the flowchart and/or block diagram block or blocks.
- blocks of the flowchart illustrations and/or block diagrams support combinations of means for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions.
- FIG. 5 illustrates operations of a routing communication protocol stack, such as the protocol stacks 22 and 38 in FIG. 4 in the present example.
- the dynamic routable VIPA is defined as described above to include the candidate target stack XCF IP addresses and the ports for the DVIPA (block 100 ).
- the protocol stack 22 has DVIPA DVAL identified as the dynamic routable VIPA
- port 60 is routable and the candidate target stacks are communication protocol stacks corresponding to XCF addresses XCF 1 , XCF 2 , and XCF 4 .
- the protocol stack 38 has DVIPA DVB 1 identified as the dynamic routable VIPA
- port 60 is routable and the candidate target stacks are specified by the “ALL” value and may be any stack in the cluster.
- the routing communication protocol stack distributes the list of DVIPAs, ports and candidate target stacks to each of the stacks in the cluster (block 102 ). Such a distribution may be carried out by, for example, broadcasting the information as part of a VIPA_list as is utilized in VIPA takeover.
- communication protocol stacks 22 and 38 would distribute their information to the other communication protocol stacks 22 , 26 , 30 , 34 and 38 .
- the routing communication protocol stacks 22 and 38 also advertise their respective DVIPAs as IP addresses through the routing protocol utilized to communicate with the network 44 (block 104 ). Alternatively, as described below, with reference to FIGS. 11 and 12 , ownership of the DVIPAs for communications on the network 44 may be established through the IP Assist function of Queued Direct I/O for OSA Express adapters.
- the routing communication protocol stacks also wait for messages from the other communication protocol stacks which identify applications which are bound to their DVIPAs and listen on an identified port (block 106 ). As the messages are received, the routing communication protocol stacks build a Destination Port Table (DPT) which identifies those stacks having instances of applications bound to the DVIPA and listening on an identified port (block 108 ). Thus, the routing communication protocol stacks, such as the communication protocol stacks 22 and 38 , are notified of which communication protocol stacks have applications bound to the DVIPA and which are available to distribute connections to the DVIPA so as to balance workload between the applications.
- DPT Destination Port Table
- FIG. 6 illustrates operations carried out by a VIPA distribution function 23 of a communication protocol stack upon receiving a message from another communication protocol stack.
- a protocol stack receives a message (block 120 )
- the protocol stack determines if the message contains a VIPA list (block 122 ). If not, then operations of the VIPA distribution function 23 terminate. If DVIPA information is present in the message, then the VIPA distribution function 23 determines if the communication protocol stack is identified as a candidate target stack for the DVIPA (block 124 ).
- the protocol stack adds the DVIPA as a non-advertised or internal VIPA address (i.e. not advertised to the routing protocol), if it is not already active as such, which may be utilized by the communication protocol stack in a manner similar to a loopback address (block 126 ).
- the communication protocol stack also monitors the addresses and ports associated with application instances utilizing the protocol stack and, if an application utilizing the protocol stack is bound or binds to the DVIPA and listens on a port identified in the VIPA list as a DVIPA port (block 128 ), the protocol stack sends a message to the routing communication protocol stack associated with the DVIPA to notify the routing communication protocol stack that communications may be routed to the application through the candidate target stack (block 130 ).
- Such candidate target protocol stacks which have applications bound to the DVIPA and listening on a port associated with the DVIPA may be referred to as a “current actual target” and, as described above, are identified in the DPT of the routing communication protocol stack as available for receiving connections.
- a message may also be sent if an application instance bound to a DVIPA and listening to a port identified in the VIPA list terminates so that the VIPA distribution function 23 of the routing communication protocol stack may maintain an up-to-date DPT. While the sending of a message to notify the routing communication protocol stack of the existence of an application bound to the DVIPA and listening to a port of the DVIPA is illustrated in FIG. 6 as responsive to receiving a message from the routing communication protocol stack, as will be appreciated by those of skill in the art, once the DVIPA is active, such messages could be sent any time the candidate target stack detects that an application is bound to the DVIPA and listening on a DVIPA port.
- the candidate target protocol stack may also determine if there are any active connections to the DVIPA (block 131 ). If so, then a connection message may be sent to the routing protocol stack (block 133 ) to notify it of the existence of the connection. In such a manner the routing protocol stack may incorporate the connection in its current routing table as described herein. Such a connection message may allow for movement of connections between routing protocol stacks, for example, to recover from a failure of a routing protocol stack.
- a communication protocol stack may be a backup for a routing communication protocol stack.
- the communication protocol stack may determine if it is a backup for the routing communication protocol stack associated with the VIPA list.
- the backup routing communication protocol stack may also determine it if has its own VIPADISTribute statement such that it would override the VIPA list information (block 135 ). If so, then the communication protocol stack need not maintain the VIPA list information as it will use its own information. Otherwise, the backup communication protocol stack maintains the information from the VIPA list so as to perform backup operations in the event of failure of the primary routing stack (block 134 ).
- the backup protocol stack may utilize a different distribution pattern than the primary protocol stack. Such differences may allow for reducing the disruption of a failed stack until the failed stack may be restored by, for example, adding candidate target stacks that are only utilized when a failure of the routing stack occurs.
- the protocol stack 22 of MVS 1 would broadcast a VIPA list (DVIPA_list — 1) identifying MVS 1 as the primary routing communication protocol stack, DVA 1 as a dynamic routable VIPA with port 60 as an associated port and the communication protocol stacks 22 , 26 and 34 as candidate target communication protocol stacks.
- the protocol stack 38 of MVS 5 would broadcast a VIPA list (DVIPA_list — 2) identifying MVS 1 as the primary routing communication protocol stack, DVB 1 as a dynamic routable VIPA with port 60 as an associated port and all of the communication protocol stacks. 22 , 26 30 , 34 and 38 as candidate target communication protocol stacks.
- the communication protocol stack 22 would be identified as a backup to the communication protocol stack 38 and the communication protocol stack 38 would be identified as a backup to the communication protocol stack 22 .
- the communication protocol stack 22 would retain the information in the distributed VIPA list for DVB 2 as the communication protocol stack 22 does not have a VIPADISTribute statement for DVB 2 .
- the communication protocol stack 38 need not retain the received VIPA list for DVA 1 as it has its own, explicit, VIPADISTribute statement for DVAL.
- communication protocol stack 26 When, for example, communication protocol stack 26 receives DVIPA_list — 1, it examines the list and determines that it is identified as a candidate target stack. Thus, the VIPA distribution function 23 of communication protocol stack 26 adds the DVIPA DVAL as a non-routable VIPA and determines if an application is executing which is bound to DVAL and listening to port 60 . For purposes of the present example, APP A is bound to DVAL and listening to port 60 so the communication protocol stack 26 sends a SRVSTAT message to communication protocol stack 22 identifying itself as a current actual target. The VIPA distribution function 23 of the communication protocol stack 22 incorporates the XCF address of the communication protocol stack 22 into its DPT. Messages to port 60 of the DVIPA may then be routed to the communication protocol stack 26 . Because no connections exist at this time a NEWCONN message is not sent.
- the communication protocol stack 30 When the communication protocol stack 30 receives DVIPA_list — 1, it examines the list and is not identified as a candidate target stack or as a backup to the communication protocol stack 22 and may disregard the list.
- the communication protocol stack 38 receives DVIPA_list — 1, it examines the list and is not identified as a candidate target stack but is identified as a backup to the communication protocol stack 22 . Thus, the communication protocol stack 38 stores the list for use in error recovery.
- any of the communication protocol stacks 22 , 26 , 30 , 34 and 38 receive the DVIPA_list — 2, then note that the “ALL” parameter is identified and add the DVIPA DVB 1 as a non-routable VIPA.
- These communication protocol stacks 22 , 26 , 30 , 34 and 38 the monitor for applications bound DVB 1 and listening on port 60 to determines if an application is executing which is bound to DVA 1 and listening to port 60 . If and when such an application binds to DVB 2 and listens on port 60 a SRVSTAT message is sent to the communication protocol stack 38 to identify the candidate target stack as a current actual target as described above. Furthermore, if a communication protocol stack is subsequently activated, it too will identify DVB 1 as a DVIPA and add DVB 1 as a non-routable VIPA.
- FIG. 7 illustrates operations of a routing communication protocol stack when a communication is received from the network 44 .
- the VIPA distribution function 23 of the communication protocol stack determines if the communication is to a DVIPA associated with the stack (block 140 ) by, for example, examining the IP address and port of the communication and comparing that with those of the DVIPAs for the protocol stack. If the communication is not to a DVIPA of the protocol stack, then operations of the VIPA distribution function 23 may terminate with respect to the communication.
- the VIPA distribution function 23 determines if the communication is a SYN to establish a connection to the DVIPA (block 142 ). If so, then the VIPA distribution function 23 may select a current actual target for the connection (i.e. a communication protocol stack with an application bound to the DVIPA and listening to the port specified by the communication)(block 144 ). Such a selection may, for example, be based on predefined criteria, utilizing a predefined selection pattern, such as round-robin, weighted round-robin or the like, or may be based on dynamic criteria, policies or combinations thereof. For example, the selection may be made to distribute workload between the candidate target stacks. Thus, a workload manger and/or a service policy agent may be consulted in selecting the candidate target stack.
- the VIPA distribution function 23 updates a current routing table (CRT) which defines the path from the routing communication protocol stack to the selected current actual target (block 146 ).
- CRT current routing table
- Such an update may take the form of creating an entry incorporating the source IP address, DVIPA and port and the XCF address of the selected current actual target.
- the message is also forwarded to the selected current actual target using the XCF address of the current actual target (block 150 ).
- the VIPA distribution function 23 of the routing communication protocol stack consults the CRT to route the communication and, if an appropriate destination communication protocol stack is found, routes the communication to the current actual target for the communication (block 152 ). The communication is then forwarded to the current actual target specified by the CRT (block 150 ).
- FIG. 8 illustrates operations of the VIPA distribution function 23 of the routing communication protocol stack when a message is received from another communication protocol stack.
- the VIPA distribution function 23 determines the type of message received (block 160 ). If the message is a SRVSTAT message, then, as described above, the application and communication protocol stack entries of the DPT are updated (block 166 ) by the VIPA distribution function 23 .
- the SRVSTAT message may be sent by a communication protocol stack both when an application instance binds to the DVIPA and listens to an associated port and when the application instance terminates.
- the SRVSTAT message may be utilized by the VIPA distribution function 23 to maintain the DPT with up-to-date information as to the current actual targets available for connections (block 166 ).
- the VIPA distribution function 23 may also determine if the message is a new connection message (NEWCONN). Such a message may be generated if an application bound to a DVIPA utilizing a port in the VIPA list initiates a connection or, as described above, if a communication protocol stack receives a VIPA list with a DVIPA which already have applications using the DVIPA for connections, then the VIPA distribution function 23 of the communication protocol stack sends a NEWCONN message to the routing communication protocol stack to notify the routing communication protocol stack of the connection. If the message is a NEWCONN message, then the VIPA distribution function 23 incorporates the connection into the CRT (block 164 ). Such an incorporation of the connection into the CRT may be carried out as described above for connections originated from network 44 .
- NGWCONN new connection message
- a third type of message which may be received by the VIPA distribution function 23 is a connection termination message (TERMCONN).
- TERMCONN connection termination message
- Such a message may be generated by a VIPA distribution function 23 when a connection terminates.
- the VIPA distribution function 23 of the communication protocol stack sends a TERMCONN message to the routing communication protocol stack to notify the routing communication protocol stack that the connection has ended and routing for the connection may be discontinued.
- the VIPA distribution function 23 removes the connection from the CRT (block 162 ).
- Such a removal of the connection from the CRT may be carried out by, for example, deleting or invalidating an entry in the CRT corresponding to the connection.
- the VIPA distribution function 23 determines that the SYN is to a dynamic routable VIPA for which it is the routing communication protocol stack, consults it DPT and optionally a workload management function (not shown) and selects a current actual target as a destination for the message.
- the VIPA distribution function 23 of the communication protocol stack 22 may select the communication protocol stack 26 as a destination.
- the communication protocol stack 22 creates an entry for the connection in its CRT and forwards the message to the communication protocol stack 26 . Subsequent messages from the network 44 to port 60 of DVA 1 from the source IP address will also be routed to the communication protocol stack 26 using the CRT entry.
- An instance of APP A of the communication protocol stack 26 bound to DVA 1 and utilizing port 60 may also establish a connection over network 44 either directly or through another protocol stack.
- the VIPA distribution function 23 of communication protocol stack 26 sends a NEWCONN message to the routing communication protocol stack 22 identifying the new connection.
- the VIPA distribution function 23 of communication protocol stack 22 receives the NEWCONN message and updates its CRT to route communications from the identified new connection to port 60 of DVA 1 to the communication protocol stack 26 .
- Such an identification may be made by, for example, providing the source IP address of the other end of the connection, the DVIPA and port and the XCF address of the communication protocol stack 26 to the routing communication protocol stack 22 .
- the VIPA distribution function 23 of the communication protocol stack 26 sends a TERMCONN message to the VIPA distribution function 23 of the communication protocol stack 22 .
- the VIPA distribution function 23 of the communication protocol stack 22 removes the entry from the CRT corresponding to the function and, thereby, terminates routing for the connection.
- FIGS. 9 and 10 illustrate operations of the VIPA distribution function 23 during failure of a routing communication protocol stack and during recovery of a routing communication protocol stack.
- the communication protocol stack identified as the backup stack for the dynamic routable VIPA takes over routing functions for that DVIPA so as to become a backup routing communication protocol stack.
- the backup routing communication protocol stack may broadcast the DVIPA list that it will utilize in routing connections for the DVIPA (block 201 ). This list may be the list provided by the primary routing communication protocol stack or a list defined explicitly for the backup routing communication protocol stack. Alternatively, the list may need only be distributed in the instance where the backup routing communication protocol stack has an explicitly defined DVIPA list.
- each of the communication protocol stacks is aware that it is a candidate target for a DVIPA and the identity of the highest ranking backup routing communication protocol stack. Therefore, the communication protocol stacks with application instances bound to the DVIPA and listening on an associated port may send a SRVSTAT message and a NEWCONN message(s) for connections to the DVIPA for the communication protocol stack to the backup routing communication protocol stack (block 202 ).
- the communication protocol stacks also associate the backup routing communication protocol stack with any connections utilizing the DVIPA so that subsequent messages for the DVIPA are sent to the backup routing communication protocol stack (block 204 ).
- the backup routing communication protocol stack utilizes the SRVSTAT messages and its information from the appropriate VIPA list to build a new DPT for the DVIPA (block 206 ).
- the backup routing communication protocol stack also receives NEWCONN messages from the server communication protocol stacks with existing DVIPA connections and constructs a CRT based on these messages (block 208 ).
- the routing information from the constructed CRT is incorporated into the backup routing communication protocol stack's own CRT (block 210 ).
- the backup routing communication protocol stack may also send its own DVIPA messages to the other communication protocol stacks to notify the other communication protocol stacks that it is performing the backup function (block 212 ). Such messages may be sent to prevent other backup communication protocol stacks in a list of backups from taking over the DVIPA. Details on the transfer of a VIPA to a backup are provided in United States Patent Application Serial No. 09 / 401 , 419 described above.
- the issuance of the SRVSTAT or the NEWCONN messages may be in response to the DVIPA messages sent by the backup routing communication protocol stack.
- embodiments of the present invention are not limited to the sequence illustrated in FIGS. 9 and 10 . Operations then continue with the backup routing communication protocol stack operating as the routing communication protocol stack described above.
- the primary communication protocol stack when the primary communication protocol stack recovers, it again broadcasts its VIPA list which is received by the other communication protocol stacks (block 300 ).
- the candidate target stacks send SRVSTAT messages to the recovered primary routing communication protocol stack (block 302 ) which identify the stacks which have application instances bound to the DVIPA and utilizing a port of the VIPA list.
- the recovered primary routing communication protocol stack also sends a DVIPA message to the backup communication protocol stack which receives the takeback message (block 304 ) and generates a NEWCONN message for all the connections which are routed through the backup communication protocol stack (block 306 ).
- the NEWCONN message is sent to the primary routing communication protocol stack (block 308 )and the primary routing communication protocol stack constructs a CRT based on the information form the message.
- the server stacks may send NEWCONN messages directly to the recovered primary routing stack either in response to receiving the VIPA list or the DVIPA message.
- routing of the existing connections may then be performed by the primary routing communication protocol stack.
- the backup routing communication protocol stack deletes the DVIPA and cleans up its routing and DPT tables (block 310 ). Thus, the routing of the DVIPA is returned to the recovered stack.
- the communication protocol stacks 26 and 34 have applications with connections routed through the DVIPA of the routing communication protocol stack 22 .
- the communication protocol stack 38 is defined as the backup to the communication protocol stack 22 . If the communication protocol stack 22 fails, then the communication protocol stacks 26 and 34 send SRVSTAT messages to the backup routing communication protocol stack 38 identifying them as current actual targets. The communication protocol stacks 26 and 34 also associate the communication protocol stack 38 with the DVIPA DVA 1 and send all subsequent messages to the backup routing communication protocol stack 38 .
- the communication protocol stack 38 builds its DPT from the SRVSTAT messages, receives NEWCONN messages for connections through the failed communication protocol stack 22 and creates routing information for incorporating in its CRT.
- the backup routing communication protocol stack 38 incorporates the routing information into its routing table and begins routing messages for the connections routed through the failed communication protocol stack.
- the backup routing communication protocol stack 38 may also send a DVIPA message to the other communication protocol stacks 26 , 30 and 34 to indicate that it is backing up the DVIPA DVA 1 .
- the primary routing communication protocol stack 22 When the primary routing communication protocol stack 22 is recovered, it sends a VIPA list to the other communication protocol stacks 26 , 30 , 34 and 38 .
- the VIPA list message signals the other communication protocol stacks 26 , 30 , 34 and 38 to send SRVSTAT messages to the communication protocol stack 22 so as to identify current actual targets for the DVIPA DVA 1 .
- the communication protocol stack 22 builds a DPT from the SRVSTAT messages.
- the backup routing communication protocol stack 38 also generates a NEWCONN message for each connection to DVA 1 routed through the backup routing communication protocol stack 38 and sends this message tot the communication protocol stack 22 .
- the server communication stacks may send NEWCONN messages in response to the VIPA list identifying existing connections to the DVIPA.
- the communication protocol stack 22 builds a CRT from the NEWCONN message(s).
- the backup routing communication protocol stack 38 cleans up its routing tables and deletes DVA 1 so that it no longer performs routing for DVA 1 .
- control of DVA 1 is transferred back to the communication protocol stack 22 with limited disruption to the connections.
- a dynamic VIPA may be established by an IOCTL or bind to a VIPA defined in the VIPARange as MOVEable NONDIRUPTIVE or in a VIPADEFine statement as MOVEable IMMEDiate.
- the DVIPA messages described above may be utilized to move the VIPA from an existing protocol stack to a new protocol stack with the new protocol stack routing the connection to the existing protocol stack.
- the dynamically routable VIPA routing function may be used with non-distributed (existing) dynamic VIPAs to allow immediate movement of a dynamic VIPA from a first protocol stack to a second protocol stack without requiring that all connections to the VIPA at the first protocol stack be terminated.
- the routing function may allow connections to the first stack to be maintained and new connections created to the second protocol stack by routing the old connections to the first protocol stack until they terminate. Thus, a gradual, non-disruptive takeback of connections may be achieved.
- FIGS. 11 and 12 illustrate operations of a communication protocol stack to facilitate the moveable VIPA.
- a dynamic VIPA when a dynamic VIPA is initialized, by a protocol stack it may be determined if the VIPA is a moveable nondisruptive VIPA (block 500 ). Initialization of the VIPA may occur, for example, by an application binding to a VIPA in a VIPA range which is defined as MOVEable NONDISRUPtive. If so, then the VIPA may be advertised to the routing protocol (block 502 ). Alternatively, if the IP gateway of a cluster of data processing systems is OSA Express using Queued Direct I/O, the IP Assist function may automatically update the OSA Express adapter when a “home list” of a protocol stack is updated.
- the operations of block 502 may be skipped. It may also be determined if the VIPA is active on another stack (block 504 ). Such a determination may be made by previous messages from the other stack advertising the existence of the VIPA. If the VIPA is not active on another stack, then the current stack may establish connections using the VIPA without using the other stack as routing stack (block 510 ). The current stack may also create routing and destination port tables as described above in the event that the VIPA is moved to another stack or in the event that other stacks look to the current stack as a routing communication protocol stack.
- the current stack receives connection administration messages from the other stack or stacks identifying existing connections to the stack (block 506 ).
- the current stack builds a routing table from these connection administration messages and routes information to these existing connections (block 508 ).
- Such a communication sequence may result from the advertisement that the current stack owns the VIPA.
- the current stack may also receive subsequent requests for connections to the VIPA and termination messages for existing connections to the VIPA so as to manage routing of information to the appropriate communication protocol stacks as described herein. Thus, ownership of the VIPA may be immediately transferred to a new communication protocol stack without disrupting existing connections.
- FIG. 12 illustrates operations of a communication protocol stack which owns a VIPA in response to a new communication protocol stack claiming ownership.
- notification of the VIPA on another communication protocol stack is received (block 520 ) and the VIPA is associated with the new communication protocol stack (block 522 ) such that all messages concerning new or existing connections are sent to the new communication protocol stack. It may also be determined if there are existing connections to the VIPA (block 524 ). If no connections exist, then the transfer of ownership is complete merely by associating the VIPA with the new communication protocol stack and deleting the DVIPA from the current communication protocol stack (block 525 ). However, if connections exist, then an existing connection message is sent to the new communication protocol (block 526 ).
- such a message may be for all connections associated with the stack or only for those which the stack is an endpoint node.
- Such a message may be a NEWCONN message as described above.
- these functions are also transferred to the new communication protocol stack and so the routing tables and destination port tables of the current communication protocol stack may be purged of the connections (block 528 ).
- the communication protocol stack 38 which functions as a primary routing communication protocol stack for DVIPA DVB 1 , has not been started.
- the communication protocol stack 30 is initialized and an instance APP B started and binds to a VIPA in the VIPARange statement, such as DVB 1 .
- connections to DVB 1 would be established to the communication protocol stack 30 .
- the communication protocol stack 26 was initialized and an instance of APP B bound to DVB 1 , the communication protocol stack 26 would send a VIPA list identifying the communication protocol stack 26 as the owner of the VIPA and, in response, the communication protocol stack 30 would send a. NEWCONN message.
- the communication protocol stack 26 would receive the NEWCONN message and incorporate a route for the connection to the communication protocol stack 30 into its routing tables.
- the communication protocol stack 30 would send all subsequent messages regarding the connection to the communication protocol stack 26 .
- the communication protocol stack 38 When the communication protocol stack 38 is initialized, it would assume ownership of the VIPA as the primary routing communication protocol stack and would send a VIPA list to the other communication protocol stacks identifying itself as such. It may also send a takeback message as well to take ownership of the VIPA.
- the communication protocol stacks 30 and 26 which have existing connections to the VIPA would send a message identifying the existing connections to the communication protocol stack 38 which would incorporate routing paths for the connections into its routing table.
- connection administration message refers to messages between communication protocol stacks which are utilized to manage the routing of TCP/IP messages between the communication protocol stacks.
- the NEWCONN, TERMCONN and VIPA lists may be considered connection administration messages.
Abstract
Description
- The present application is related to concurrently filed and commonly assigned U.S. patent application Ser. No. ______, entitled “METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS FOR NON-DISRUPTIVELY TRANSFERRING A VIRTUAL INTERNET PROTOCOL ADDRESS BETWEEN COMMUNICATION PROTOCOL STACKS” (Attorney Docket No. 5577-207) and U.S. patent application Ser. No. ______, entitled “METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS FOR FAILURE RECOVERY FOR ROUTED VIRTUAL INTERNET PROTOCOL ADDRESSES” (Attorney Docket No. 5577-206), the disclosures of which are incorporated herein by reference as if set forth fully herein.
- The present invention relates to network communications and more particularly to network communications to a cluster of data processing systems.
- The Internet Protocol (IP) is a connectionless protocol. IP packets are routed from originator through a network of routers to the destination. All physical adapter devices in such a network, including those for client and server hosts, are identified by an IP Address which is unique within the network. One valuable feature of IP is that a failure of an intermediate router node or adapter will not prevent a packet from moving from source to destination, as.long as there is an alternate path through the network.
- In Transmission Control Protocol/Internet Protocol (TCP/IP), TCP sets up a connection between two endpoints, identified by the respective IP addresses and a port number on each. Unlike failures of an adapter in an intermediate node, if one of the endpoint adapters (or the link leading to it) fails, all connections through that adapter fail, and must be reestablished. If the failure is on a client workstation host, only the relatively few client connections are disrupted, and usually only one person is inconvenienced. However, an adapter failure on a server means that hundreds or thousands of connections may be disrupted. On a System/390 with large capacity, the number may run to tens of thousands.
- To alleviate this situation, International Business Machines Corporation introduced the concept of a Virtual IP Address, or VIPA, on its TCP/IP for 0S/390 V2R5 (and added to V2R4 as well). Examples of VIPAs and their user may be found in U.S. Pat. Nos. 5,917,997, 5,923,854, 5,935,215 and 5,951,650. A VIPA is configured the same as a normal IP address for a physical adapter, except that it is not associated with any particular device. To an attached router, the TCP/IP stack on System/390 simply looks like another router. When the TCP/IP stack receives a packet destined for one of its VIPAs, the inbound IP function of the TCP/IP stack notes that the IP address of the packet is in the TCP/IP stack's Home list of IP addresses and forwards the packet up the TCP/IP stack. The “home list” of a TCP/IP stack is the list of IP addresses which are “owned” by the TCP/IP stack. Assuming the TCP/IP stack has multiple adapters or paths to it (including a Cross Coupling Facility (XCF) path from other TCP/IP stacks in a Sysplex), if a particular physical adapter fails, the attached routing network will route VIPA-targeted packets to the TCP/IP stack via an alternate route. The VIPA may, thus, be thought of as an address to the stack, and not to any particular adapter.
- While the use of VIPAs may remove hardware and associated transmission media as a single point of failure for large numbers of connections, the connectivity of a server can still be lost through a failure of a single stack or an MVS image. The VIPA Configuration manual for System/390 tells the customer how to configure the VIPA(s) for a failed stack on another stack, but this is a manual process. Substantial down time of a failed MVS image or TCP/IP stack may still result until operator intervention to manually reconfigure the TCP/IP stacks in a Sysplex to route around the failed TCP/IP stack or MVS image.
- While merely restarting an application with a new IP address may resolve many failures, applications use IP addresses in different ways and, therefore, such a solution may be inappropriate. The first time a client resolves a name in its local domain, the local Dynamic Name Server (DNS) will query back through the DNS hierarchy to get to the authoritative server. For a Sysplex, the authoritative server should be DNS/Workload Manager (WLM). DNS/WLM will consider relative workloads among the nodes supporting the requested application, and will return the IP address for the most appropriate available server. IP addresses for servers that are not available will not be returned. The Time to Live of the returned IP address will be zero, so that the next resolution query (on failure of the original server, for example) will go all the way back to the DNS/WLM that has the knowledge to return the IP address of an available server.
- However, in practice, things do not always work as described above. For example, some clients are configured to a specific IP address, thus requiring human intervention to go to another server. However, the person using the client may not have the knowledge to reconfigure the client for a new IP address. Additionally, some clients ignore the Time to Live, and cache the IP address as long as the client is active. Human intervention may again be required to recycle the client to obtain a new IP address. Also, DNSs are often deployed as a hierarchy to reduce network traffic, and DNSs may cache the IP address beyond the stated Time to Live even when the client behaves quite correctly. Thus, even if the client requests a new IP address, the client may receive the cached address from the DNS. Finally, some users may prefer to configure DNS/WLM to send a Time to Live that is greater than zero, in an attempt to limit network-wide traffic to resolve names. Problems arising from these various scenarios may be reduced if the IP address with which the client communicates does not change. However, as described above, to affect such a movement of VIPAs between TCP/IP stacks requires operator intervention and may result in lengthy down times for the applications associated with the VIPA.
- Previous approaches to increased availability focused on providing spare hardware. The High-Availability Coupled Multi-Processor (HACMP) design allows for taking over the MAC address of a failing adapter on a shared medium (LAN). This works both for a failing adapter (failover to a spare adapter on the same node) or for a failing node (failover to another node via spare adapter or adapters on the takeover node.) Spare adapters are not used for IP traffic, but they are used to exchange heartbeats among cluster nodes for failure detection. All of the work on a failing node goes to a single surviving node. In addition to spare adapters and access to the same application data, the designated failover node must also have sufficient spare processing capacity to handle the entire failing node workload with “acceptable” service characteristics (response and throughput).
- Automatic restart of failing applications also provides faster recovery of a failing application or node. This may be acceptable when the application can be restarted in place, but is less useful when the application is moved to another node, unless the IP address known to the clients can be moved with the application, or dynamic DNS updates with alternate IP addresses can be propagated to a DNS local to clients sufficiently quickly.
- Other attempts at error recovery have included the EDDIE system described in a paper titled “EDDIE, A Robust and Scalable Internet Server” by A. Dahlin, M. Froberg, J. Grebeno, J. Walerud, and P. Winroth, of Ericsson Telecom A B, Stockholm, Sweden, May 1998. In the EDDIE approach a-distributed application called “IP Address Migration Application” controls all IP addresses in the cluster. The cluster is connected via a shared-medium LAN. IP address aliasing is used to provide addresses to individual applications over a single adapter, and these aliases are located via Address Resolution Protocol (ARP) and ARP caches in the TCP/IPs. The application monitors all server applications and hardware, and reallocates aliased IP addresses in the event of failure to surviving adapters and nodes. This approach allows applications of a failing node to be distributed among surviving nodes, but it may require the monitoring application to have complete knowledge of the application and network adapter topology in the cluster. In this sense, it is similar to existing Systems Management applications such as those provided by International Business Machines Corporation's Tivoli® network management software, but the IP Address Migration Application has direct access to adapters and ARP caches. The application also requires a dedicated IP address for inter-application communication and coordination.
- U.S. patent application Ser. No. 09/401,419 entitled “METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS FOR AUTOMATED MOVEMENT OF IP ADDRESSES WITHIN A CLUSTER” filed Sep. 22, 1999, the disclosure of which is incorporated herein by reference as if set forth fully herein, describes dynamic virtual IP addresses (VIPA) and their use. As described in the '419 application, a dynamic VIPA may be automatically moved from protocol stack to protocol stack in a predefined manner to overcome failures of a particular protocol stack (i.e. VIPA takeover). Such a predefined movement may provide a predefined backup protocol stack for a particular VIPA. VIPA takeover was made available by International Business Machines Corporation (IBM), Armonk, N.Y., in System/390 V2R8 which had a general availability date of September, 1999.
- In addition to failure scenarios, scalability and load balancing are also issues which have received considerable attention in light of the expansion of the Internet. For example, it may be desirable to have multiple servers servicing customers. The workload of such servers may be balanced by providing a single network visible IP address which is mapped to multiple servers.
- Such a mapping process may be achieved by, for example, network address translation (NAT) facilities, dispatcher systems and IBM's Dynamic Name Server/Workload Management DNS/WLM systems. These various mechanisms for allowing multiple servers to share a single IP address are illustrated in
FIGS. 1 through 3 . -
FIG. 1 illustrates a conventional network address translation system as described above. In the system ofFIG. 1 , aclient 10 communicates over anetwork 12 to a networkaddress translation system 14. The network address translation system receives the communications from theclient 10 and converts the communications from the addressing scheme of thenetwork 12 to the addressing scheme of thenetwork 12′ and sends the messages to theservers 16. Aserver 16 may be selected frommultiple servers 16 at connect time and may be on any host, one or more hops away. All inbound and outbound traffic flows through theNAT system 14. -
FIG. 2 illustrates a conventional DNS/WLM system as described above. As mentioned above, theserver 16 is selected at name resolution time when theclient 10 resolves the name for the destination server from DNS/WLM system 17 which is connected to theservers 16 through thecoupling facility 19. As described above, the DNS/WLM system ofFIG. 2 relies on theclient 10 adhering to the zero time to live. -
FIG. 3 illustrates a conventional dispatcher system. As seen inFIG. 3 , theclient 10 communicates over thenetwork 12 with adispatcher system 18 to establish a connection. The dispatcher routes inbound packets to theservers 16 and outbound packets are sent overnetwork 12′ but may flow over any available path to theclient 10. Theservers 16 are typically on a directly connected network to thedispatcher 18 and aserver 16 is selected at connect time. - Such a dispatcher system is illustrated by the Interactive Network Dispatcher function of the IBM 2216 and AIX platforms. In these systems, the same IP address that the
Network Dispatcher node 18 advertises to therouting network 12 is activated onserver nodes 16 as loopback addresses. The node performing the distribution function connects to the endpoint stack via a single hop connection because normal routing protocols typically cannot be used to get a connection request from the endpoint to the distributing node if the endpoint uses the same IP address as the distributing node advertises. Network Dispatcher uses an application on the server to query a workload management function (such as WLM of System/390), and collects this information at intervals, e.g. 30 seconds or so. Applications running on the Network Dispatcher node can also issue “null” queries to selected application server instances as a means of determining server instance health. - In addition to the above described systems, Cisco Systems offers a Multi-Node Load Balancing function on certain of its routers that perform the distribution function. Such operations appear similar to those of the IBM 2216.
- Finally, in addition to the system described above, AceDirector from Alteon provides a virtual IP address and performs network address translation to a real address of a selected server application. AceDirector appears to observe connection request turnaround times and rejection as a mechanism for determining server load capabilities.
- Methods, systems and computer program products according to embodiments of the present invention provide communications with a plurality of application instances executing on a cluster of data processing systems having a plurality of communication protocol stacks associated therewith utilizing a single Internet Protocol (IP) address. A first of the plurality of communication protocol stacks is established as a routing communication protocol stack associated with the single IP address. Ones of the plurality of communication protocol stacks which are associated with the single IP address are defined as candidate target protocol stacks and an identification of candidate target protocol stacks and the routing communication protocol stack is distributed to the communication protocol stacks. The routing communication protocol stack is notified when an instance of the plurality of application instances associated with a candidate target communication protocol stack listens to a port of the single IP address to establish a current actual target stack. A request to establish a connection to the single IP address and the port of the single IP address is received and a routing table entry corresponding to the current actual target stack is created to provide a routing path associated with the IP address and the port from the routing communication protocol stack to the current actual target stack. Communications to the port of the IP address received by the routing communication protocol stack are routed based on the routing table.
- In further embodiments of the present invention, all of the plurality of communication protocol stacks of the cluster of data processing systems are defined as associated with the single IP address. Alternatively, enumerated ones of the plurality of communication protocol stacks of the cluster of data processing systems may be defined as associated with the single IP address.
- In still further embodiments of the present invention, a first of the plurality of. communication protocol stacks is established as a routing communication protocol stack associated with the single IP address for routing communications associated with at least one specified port associated with the IP address. Ones of the plurality of communication protocol stacks which are associated with the at least one specified port associated with the IP address are defined as candidate target communication protocol stacks and an identification of the candidate target protocol stacks and the routing communication protocol stack distributed to the plurality of communication protocol stacks. The routing communication protocol stack is notified when an instance of the plurality of application instances associated with a candidate target communication protocol stack listens to the at least one port of the single IP address.
- In such embodiments, the at least one port may be a plurality of ports identified in a port list associated with the single IP address. Furthermore, the least one port may be all ports associated with-the single IP address.
- In additional embodiments, the first communication protocol stack is notified that the instance of the plurality of application instances associated with a candidate target communication protocol stack has terminated listening to the port of the single IP address. In such a case, the routing table entry corresponding to the candidate target communication protocol stack may be removed so as to remove the routing path associated with the IP address, the port and the candidate target communication protocol stack. Such notification may be provided by sending a termination message through a cross coupling facility of the cluster of data processing systems.
- In still further embodiments of the present invention, the routing communication protocol stack is established and the plurality of communication protocol stacks defined by incorporating a VIPADISTribute statement in a VIPADynamic definition block associated with the first communication protocol stack. The VIPADISTribute statement defines a IP addressas a dynamic routable virtual IP address (VIPA), identifies ports associated with the VIPA which are routable, and identifies communication protocols stacks associated with the VIPA. In such embodiments, the distribution of an identification of protocol stacks which are associated with the single IP address and an identification of the first of the plurality of communication protocol stacks may be accomplished by broadcasting a list including the definitions of the VIPADISTRibute statement.
- In still further embodiments of the present invention, the routing communication protocol stack is notified that a connection utilizing the single IP address and the port of the singe IP address has terminated. The routing table entry corresponding to the connection to the candidate target communication protocol stack associated with the connection is then removed so as to remove the routing path associated with the IP address, the port and the second communication protocol stack.
- Furthermore, when a connection request is received by the routing communication protocol stack, one of the communication protocol stacks identified as having an application instance bound to the single IP address and listening to the port associated with the request is selected. The routing path may then be incorporated in the routing table for the selected communication protocol stack.
- In further embodiments of the present invention, systems, methods and computer program products are provided for communicating with a plurality of application instances executing on a cluster of data processing systems having a plurality of communication protocol stacks associated therewith utilizing a single IP address may be provided by a routing communication protocol stack configured to receive communications to the single IP address and to forward to the received communications to a selected communication protocol stack from the plurality of communication protocol stacks. A destination port table associated with the routing communication protocol stack which identifies communication protocol stacks having application instances bound to the single IP address and listening to a predefined port of the single IP address. A current routing table associated with the routing communication protocol stack which identifies routing paths from the routing communication protocol stack to a communication protocol stack associated with a connection utilizing the single IP address and the predefined port.
- In further embodiments, the routing communication protocol stack is further configured to build the destination port table based on status messages received from communication protocol stacks of the plurality of communication protocol stacks having application instances bound to the single IP address and listening to a predefined port of the single IP address in response to distribution by the routing communication protocol stack of an identification of the single IP address and the predefined port as a routable IP address. Furthermore, the routing communication protocol stack may be further configured to incorporate into the current routing table routing paths associated with new connection messages received by the routing communication.protocol stack from communication protocol stacks which initiate connections utilizing the single IP address and the predefined port as a source address.
- In yet additional embodiments of the present invention, the routing communication protocol stack is further configured to receive requests for connections to the single IP address and the predefined port, select communication protocol stacks for the connections from the destination port table and update the current routing table with path information associated with the connections and the selected communication protocol stacks. The routing communication protocol stack may also be further configured to receive connection termination messages from communication protocol stacks which detect termination of connections utilizing the single IP address and predefined port and to update the current routing table to remove routing paths associated with the terminated connections.
- The predefined port may be a plurality of predefined ports identified in a port list associated with the single IP address or the predefined port may be all ports associated with the single IP address.
- As will further be appreciated by those of skill in the art, the present invention may be embodied as methods, apparatus/systems and/or computer program products.
-
FIG. 1 is block diagram of a conventional network address translation system; -
FIG. 2 is block diagram of a conventional DNS/WLM system;. -
FIG. 3 is block diagram of a conventional dispatcher system; -
FIG. 4 is block diagram of a cluster of data processing systems incorporating embodiments of the present invention; -
FIG. 5 is a flowchart illustrating operations for initialization of a routing protocol stack incorporating distributable VIPAs according to embodiments of the present invention; -
FIG. 6 is a flowchart illustrating operations of a server protocol stack according to embodiments of the present invention; -
FIG. 7 is a flowchart illustrating operations for a incoming communications to the routing protocol stack according to embodiments of the present invention; -
FIG. 8 is a flowchart illustrating operations of a routing protocol stack receiving communications from another protocol stack according to embodiments of the present invention; -
FIG. 9 is a flowchart illustrating operations of protocol stacks during failure of a routing protocol stack according to embodiments of the present invention; -
FIG. 10 is a flowchart illustrating operations of protocol stacks according to embodiments of the present invention incorporating moveable VIPAs for recovery of a failed routing protocol stack; -
FIG. 11 is a flowchart illustrating operations of a communication protocol stack which owns a moveable VIPA; and -
FIG. 12 is a flowchart illustrating operations of a communications protocol stack in response to loss of ownership of a moveable VIPA. - The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout.
- As will be appreciated by those of skill in the art, the present invention can take the form of an entirely hardware embodiment, an entirely software (including firmware, resident software, micro-code, etc.) embodiment, or an embodiment containing both software and hardware aspects. Furthermore, the present invention can take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code means embodied in the medium for use by or in connection with an instruction execution system. In the context of this document,- a computer-usable or computer-readable medium can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- The computer-usable or computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection having one or more wires, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM). Note that the computer-usable or computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
- The present invention can be embodied as systems, methods, or computer program products which allow for a single IP address being associated with a plurality of communication protocol stacks in a cluster of data processing systems by providing a routing protocol stack which associates a Virtual IP Address (VIPA) and port with other communication protocol stacks in the cluster and routes communications to the VIPA and port to the appropriate communication protocol stack. VIPAs capable of being shared by a number of communication protocol stacks are referred to herein as “dynamic routable VIPAs” (DVIPA). While the present invention is described with reference to a specific embodiment in a System/390 Sysplex, as will be appreciated by those of skill in the art, the present invention may be utilized in other systems where clusters of computers utilize virtual addresses by associating an application or application group rather than a particular communications adapter with the addresses. Thus, the present invention should not be construed as limited to the particular exemplary embodiment described herein.
- A cluster of data processing systems is illustrated in
FIG. 4 as a cluster nodes inSysplex 10. As seen inFIG. 4 , severaldata processing systems Sysplex 10. Thedata processing systems FIG. 4 may be operating system images, such as MVS images, executing on one or more computer systems. While the present invention will be described primarily with respect to the MVS operating system executing in a System/390 environment, thedata processing systems - As is further illustrated in
FIG. 4 , thedata processing systems VIPA distribution function 23 as described herein for providing dynamic routable VIPAs so as to provide a single IP address for multiple communication protocol stacks. - While each of the communication protocol stacks 22, 26, 30, 34 and 38 illustrated in
FIG. 4 incorporate theVIPA distribution function 23, not all communication protocol stacks in a Sysplex need incorporate theVIPA distribution function 23. Thus, the present invention may be carried out on any system where two or more communication protocol stacks in a cluster of data processing systems support dynamic routable VIPAs. If a communication protocol stack does not support dynamic routable VIPA, then the dynamic routable VIPA messages according to the present invention would be ignored by the communication protocol stack. Thus, the present invention provides backward compatibility with existing communication protocol stacks. - As is further seen in
FIG. 4 , the communication protocol stacks 22, 26, 30, 34 and 38 may communicate with each other through acoupling facility 40 of theSysplex 10, for example, utilizing XCF messaging. Furthermore, the communication protocol stacks 22 and 38 may communicate with anexternal network 44 such as the Internet, an intranet, a Local Area Network (LAN) or Wide Area Network (WAN) utilizing the Enterprise System Connectivity (ESCON) 42. Thus, aclient 46 may,utilizenetwork 44 to communicate with an application executing on an MVS image inSysplex 10 through the communication protocol stacks 22 and 38 which may function as routing protocol stacks as described herein. - As is further illustrated in
FIG. 4 , as an example of utilization of the present invention and for illustration purposes,data processing system 20 has associated with itcommunication protocol stack 22 which is associated withMVS image MVS 1 which has application APP A executing onMVS image MVS 1 and utilizingcommunication protocol stack 22 to allow access to, for example,client 46 throughnetwork 44. Similarly,data processing system 24 has associated with itcommunication protocol stack 26 which is associated withMVS image MVS 2 which has a second instance of application APP A and an instance of application APP B executing onMVS image MVS 2 which may utilizecommunication protocol stack 26 for communications.Data processing system 28 has associated with itcommunication protocol stack 30 which is associated withMVS image MVS 3 which has a second instance of application APP B executing onMVS image MVS 3 which may utilizecommunication protocol stack 30 for communications.Data processing system 32 has associated with itcommunication protocol stack 34 which is associated withMVS image MVS 4 which has a third instance of application APP A executing onMVS image MVS 4 which may utilizecommunication protocol stack 34 for communications. Finally,data processing system 36 has associated with itcommunication protocol stack 38 which is associated withMVS image MVS 5 which has a third instance of application APP B executing onMVS image MVS 5 which may utilizecommunication protocol stack 38 for communications. - Utilizing the above described system configuration as an example, the
VIPA distribution function 23 will now be described. TheVIPA distribution function 23 allows for protocol stacks which are defined as supporting DVIPAs to share the DVIPA and communicate withnetwork 44 through a routing protocol stack such that all protocol stacks having a server application which is associated with the DVIPA will appear to thenetwork 44 as a single IP address. Such dynamically routable VIPAs may be provided by designating a protocol stack, such asprotocol stack 22, as a routing protocol stack, notifying other protocol stacks of the routing protocol stack and having other protocol stacks notify the routing protocol stack when an application which binds to the DVIPA is started. Because communications to the DVIPA are routed through the routing protocol stack, the routing protocol stack may provide work load balancing by distributing connections to the other protocol stacks on MVS images executing server applications which bind to the DVIPA to balance workload. Furthermore, in particular embodiments of the present invention, scalability and availability may be provided by allowing all protocol stacks for MVS images which execute applications which bind to the DVIPA to have communications routed through the routing protocol stack without user intervention to establish the routing path. - Further aspects of the
VIPA distribution function 23 according to embodiments of the present invention allow automated movement of a routing protocol function to a backup stack. Another aspect of theVIPA distribution function 23 allows recovery of a failed routing stack without disruption to connections through the backup stack. - The communication protocol stacks 22, 26, 30, 34 and 38 may be configured as to which stacks are routing stacks, backup routing stacks and server stacks. Different DVIPAs may have different sets of backup stacks, possibly overlapping. The definition of backup stacks may be the same as that for the VIPA takeover function described in U.S. patent application Ser. No. 09/401,419, entitled “METHODS, SYSTEMS AND COMPUTER PROGRAM PRODUCTS FOR AUTOMATED MOVEMENT OF IP ADDRESSES WITHIN A CLUSTER” which is incorporated herein by reference as if set forth fully herein.
- In providing for DVIPAs, up to five or more aspects of DVIPA operation may be addressed: 1) initialization and definition of DVIPAs and the affected protocol stacks; 2) incoming communications from
network 44 to the DVIPA; 3) connections originated by a protocol stack (i.e. outgoing to network 44); 4)failure of a routing stack; and 5) recovery of a routing stack. - Turning now to the first of these aspects, for the present example, application APP A is associated with DVIPA VA1 which may be associated with the respective first, second and third instances of APP A; and application APP B likewise has DVIPA VB1 associated with the respective first, second and third instances of APP B.
- Configuration of a dynamic routable VIPA may be provided by a new definition block established by a system administrator for each routing
communication protocol stack -
- VIPADynamic
- Dynamic Routable VIPA definitions . . .
- ENDVIPADynamic
- The definitions within the VIPADynamic block for a protocol stack supporting moveable VIPAs are:
-
- VIPADEFine MOVEable IMMEDiate netMask ipaddr . . .
where the netMask is used to determine the network prefix to be advertised to routing daemons for OSPF or RIP and ipaddr is the IP address of the DVIPA. Both network prefix (sometimes known as subnet address) and the mask will be advertised to the routing daemon. All of the VIPAs in a single VIPADEFine statement must belong to the same subnet, network, or supernet, as determined by the network class and address mask. The MOVEable IMMEDiate parameters define the VIPAs as moveable VIPAs which may be transferred from one communication protocol stack to another. As will be appreciated by those of skill in the art in light of the present disclosure, while the MOVEable IMMEDiate parameters are expressly defined in the above definition statements, these or other parameters may be the default parameters which are provided unless specified otherwise. Thus, the parameters need not be expressly called out in all instances.
- VIPADEFine MOVEable IMMEDiate netMask ipaddr . . .
- The definitions within the VIPADynamic block for backup are:
-
- VIPABackup rank ipaddr . . .
where the rank is a number between 1 and 254 used to determine relative order within the backup chain(s) for the associated dynamic routable VIPA(s). A communication protocol stack with the higher rank will take over the dynamic VIPAs before a communication protocol stack with a lower rank.
- VIPABackup rank ipaddr . . .
- The definitions in the VIPADYNamic block for defining a VIPA as a dynamic routable VIPA are:
-
- VIPADISTribute ipaddr PORT portlist DESTIP ipaddrlist
where ipaddr is a VIPA defined in the VIPADEFine, portlist is a list of ports for which the DVIPA will apply. If the PORT keyword is omitted, then all ports for the ipaddr will be considered as DVIPAs. The ipaddrlist is a list of protocol stacks which will be included as server stacks in routing communications directed to the DVIPA. The IP addresses in the ipaddrlist may be XCF addresses of the protocol stacks or may be designated “ALL.” If “ALL” is designated, then all stacks in the Sysplex are candidates for distribution. This may include future stacks which are not active when the routing stack is initialized. Thus, if ALL is specified, a protocol stack may be added to the DVIPA without disruption of operations and without user intervention to redefine the stack in the VIPADynamic block.
- VIPADISTribute ipaddr PORT portlist DESTIP ipaddrlist
- In addition to the above definitions, a range of IP addresses may be defined as DVIPAs utilizing the VIPARange definition. A VIPARange definition of the form:
-
- VIPARange MOVEable NONDISRUPTIVE netMASKipAddr
may designate all future VIPAs created in the range as moveable or dynamic routable VIPAs. The MOVEable NONDISRUPTIVE parameters allows future instance-specific dynamic VIPAs to participate as dynamic routable VIPAs but does not affect dynamic VIPAs created under the range before the keyword DISTribute was added (e.g. via VARY OBEY).
- VIPARange MOVEable NONDISRUPTIVE netMASKipAddr
- In the first aspect, the communication protocol stacks 22 and 38, which are designated as routing protocol stacks as they have connections to the
network 44 and include VIPADISTribute statements in the VIPADynamic block, publish the distribution information through messages broadcast by theVIPA takeover function 23 of each of the communication protocol stacks 22, 26, 30, 34 and 38 to the other communication protocol stacks 22, 26, 30, 34 and 38. At initialization or profile changes, the communication protocol stacks 22, 26, 30, 34 and 38 communicate to all partner communication protocol stacks the complete list of dynamic routable VIPAs, their associated ipAddrList and portlist and the primary and backup definitions for the communication protocol stack. - When a
communication protocol stack - Returning to the example of
FIG. 4 , for MVS1 to MVS5 the VIPADEFine statements may be: -
- MVS1:VIPADEFine MOVEable IMMEDiate DVA1
- VIPADISTribute DVA1 PORT 60 DESTIP XCF1, XCF2, XCF4
- MVS5:VIPADEFine MOVEable IMMEDiate DVB1
- VIPADISTribute DVB1 PORT 60 DESTIP ALL
- VIPADISTribute DVA1 PORT 60 DESTIP XCF2, XCF3, XCF4
For purposes of illustration the respective address masks have been omitted because they are only significant to the routing daemons. In the above illustration, XCF1 is an XCF address of the TCP/IP stack on MVS1, XCF2 is an XCF address of the TCP/IP stack on MVS2 and XCF3 is an XCF address of the TCP/IP stack on MVS4. Note that, for purposes of the present example, definitions for MVS2, MVS3, and MVS4 are not specified. Such may be the case because the protocol stacks for these MVS images are candidate target protocol stacks and are not identified as routing protocol stacks and, therefore, receive their dynamic routable VIPA definitions from the routing protocol stacks. As is further illustrated, the backup routing communication protocol stack may have a separate VIPADISTribute definition for a DVIPA than the primary routing communication protocol stack. As described in more detail below, in such a case the explicit definition of the VIPADISTribute statement for the backup routing communication protocol stack in the event of failure of the primary routing stack. Additional VIPA definitions may also be provided, however, in the interests of clarity, such definitions have been omitted.
- MVS1:VIPADEFine MOVEable IMMEDiate DVA1
- The VIPABackup statements for MVS1 and MVS5 of
FIG. 4 may be: -
- MVS1:
VIPABackup 30 DVB1 - MVS5:
VIPABackup 10 DVA1
- MVS1:
- With the above scenario in mind, embodiments of the present invention will now be described with reference to
FIGS. 5 through 12 which are flowchart illustrations of operations of protocol stacks incorporating embodiments of the present invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These program instructions may be provided to a processor to produce a machine, such that the instructions which execute on the processor create means for implementing the functions specified in the flowchart and/or block diagram block or blocks. The computer program instructions may be executed by a processor to cause a series of operational steps to be performed by the processor to produce a computer implemented process such that the instructions which execute on the processor provide steps for implementing the functions specified in the flowchart and/or block diagram block or blocks. - Accordingly, blocks of the flowchart illustrations and/or block diagrams support combinations of means for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions.
-
FIG. 5 illustrates operations of a routing communication protocol stack, such as the protocol stacks 22 and 38 inFIG. 4 in the present example. As seen inFIG. 5 , the dynamic routable VIPA is defined as described above to include the candidate target stack XCF IP addresses and the ports for the DVIPA (block 100). In the present example, theprotocol stack 22 has DVIPA DVAL identified as the dynamic routable VIPA, port 60 is routable and the candidate target stacks are communication protocol stacks corresponding to XCF addresses XCF1, XCF2, and XCF4. Theprotocol stack 38 has DVIPA DVB1 identified as the dynamic routable VIPA, port 60 is routable and the candidate target stacks are specified by the “ALL” value and may be any stack in the cluster. - The routing communication protocol stack distributes the list of DVIPAs, ports and candidate target stacks to each of the stacks in the cluster (block 102). Such a distribution may be carried out by, for example, broadcasting the information as part of a VIPA_list as is utilized in VIPA takeover. In the present example, communication protocol stacks 22 and 38 would distribute their information to the other communication protocol stacks 22, 26, 30, 34 and 38. The routing communication protocol stacks 22 and 38 also advertise their respective DVIPAs as IP addresses through the routing protocol utilized to communicate with the network 44 (block 104). Alternatively, as described below, with reference to
FIGS. 11 and 12 , ownership of the DVIPAs for communications on thenetwork 44 may be established through the IP Assist function of Queued Direct I/O for OSA Express adapters. - The routing communication protocol stacks also wait for messages from the other communication protocol stacks which identify applications which are bound to their DVIPAs and listen on an identified port (block 106). As the messages are received, the routing communication protocol stacks build a Destination Port Table (DPT) which identifies those stacks having instances of applications bound to the DVIPA and listening on an identified port (block 108). Thus, the routing communication protocol stacks, such as the communication protocol stacks 22 and 38, are notified of which communication protocol stacks have applications bound to the DVIPA and which are available to distribute connections to the DVIPA so as to balance workload between the applications.
-
FIG. 6 illustrates operations carried out by aVIPA distribution function 23 of a communication protocol stack upon receiving a message from another communication protocol stack. As seen inFIG. 6 , when a protocol stack receives a message (block 120), the protocol stack determines if the message contains a VIPA list (block 122). If not, then operations of theVIPA distribution function 23 terminate. If DVIPA information is present in the message, then theVIPA distribution function 23 determines if the communication protocol stack is identified as a candidate target stack for the DVIPA (block 124). If the communication protocol stack is a candidate target stack, either as a result of being expressly enumerated in a list or because the “ALL” parameter is specified for the DVIPA, then the protocol stack adds the DVIPA as a non-advertised or internal VIPA address (i.e. not advertised to the routing protocol), if it is not already active as such, which may be utilized by the communication protocol stack in a manner similar to a loopback address (block 126). - The communication protocol stack also monitors the addresses and ports associated with application instances utilizing the protocol stack and, if an application utilizing the protocol stack is bound or binds to the DVIPA and listens on a port identified in the VIPA list as a DVIPA port (block 128), the protocol stack sends a message to the routing communication protocol stack associated with the DVIPA to notify the routing communication protocol stack that communications may be routed to the application through the candidate target stack (block 130). Such candidate target protocol stacks which have applications bound to the DVIPA and listening on a port associated with the DVIPA may be referred to as a “current actual target” and, as described above, are identified in the DPT of the routing communication protocol stack as available for receiving connections. A message may also be sent if an application instance bound to a DVIPA and listening to a port identified in the VIPA list terminates so that the
VIPA distribution function 23 of the routing communication protocol stack may maintain an up-to-date DPT. While the sending of a message to notify the routing communication protocol stack of the existence of an application bound to the DVIPA and listening to a port of the DVIPA is illustrated inFIG. 6 as responsive to receiving a message from the routing communication protocol stack, as will be appreciated by those of skill in the art, once the DVIPA is active, such messages could be sent any time the candidate target stack detects that an application is bound to the DVIPA and listening on a DVIPA port. - Furthermore, the candidate target protocol stack may also determine if there are any active connections to the DVIPA (block 131). If so, then a connection message may be sent to the routing protocol stack (block 133) to notify it of the existence of the connection. In such a manner the routing protocol stack may incorporate the connection in its current routing table as described herein. Such a connection message may allow for movement of connections between routing protocol stacks, for example, to recover from a failure of a routing protocol stack.
- Irrespective of whether a communication- protocol stack is a candidate target stack or a current actual target stack, a communication protocol stack may be a backup for a routing communication protocol stack. Thus, as seen at
block 132, the communication protocol stack may determine if it is a backup for the routing communication protocol stack associated with the VIPA list. The backup routing communication protocol stack may also determine it if has its own VIPADISTribute statement such that it would override the VIPA list information (block 135). If so, then the communication protocol stack need not maintain the VIPA list information as it will use its own information. Otherwise, the backup communication protocol stack maintains the information from the VIPA list so as to perform backup operations in the event of failure of the primary routing stack (block 134). Thus, the backup protocol stack may utilize a different distribution pattern than the primary protocol stack. Such differences may allow for reducing the disruption of a failed stack until the failed stack may be restored by, for example, adding candidate target stacks that are only utilized when a failure of the routing stack occurs. - In the present example illustrated in
FIG. 4 , theprotocol stack 22 of MVS1 would broadcast a VIPA list (DVIPA_list—1) identifying MVS1 as the primary routing communication protocol stack, DVA1 as a dynamic routable VIPA with port 60 as an associated port and the communication protocol stacks 22, 26 and 34 as candidate target communication protocol stacks. Additionally, theprotocol stack 38 of MVS5 would broadcast a VIPA list (DVIPA_list—2) identifying MVS1 as the primary routing communication protocol stack, DVB1 as a dynamic routable VIPA with port 60 as an associated port and all of the communication protocol stacks. 22, 26 30, 34 and 38 as candidate target communication protocol stacks. Also, thecommunication protocol stack 22 would be identified as a backup to thecommunication protocol stack 38 and thecommunication protocol stack 38 would be identified as a backup to thecommunication protocol stack 22. Thecommunication protocol stack 22 would retain the information in the distributed VIPA list for DVB2 as thecommunication protocol stack 22 does not have a VIPADISTribute statement for DVB2. However, thecommunication protocol stack 38 need not retain the received VIPA list for DVA1 as it has its own, explicit, VIPADISTribute statement for DVAL. - When, for example,
communication protocol stack 26 receivesDVIPA_list —1, it examines the list and determines that it is identified as a candidate target stack. Thus, theVIPA distribution function 23 ofcommunication protocol stack 26 adds the DVIPA DVAL as a non-routable VIPA and determines if an application is executing which is bound to DVAL and listening to port 60. For purposes of the present example, APP A is bound to DVAL and listening to port 60 so thecommunication protocol stack 26 sends a SRVSTAT message tocommunication protocol stack 22 identifying itself as a current actual target. TheVIPA distribution function 23 of thecommunication protocol stack 22 incorporates the XCF address of thecommunication protocol stack 22 into its DPT. Messages to port 60 of the DVIPA may then be routed to thecommunication protocol stack 26. Because no connections exist at this time a NEWCONN message is not sent. - When the
communication protocol stack 30 receivesDVIPA_list —1, it examines the list and is not identified as a candidate target stack or as a backup to thecommunication protocol stack 22 and may disregard the list. When thecommunication protocol stack 38 receivesDVIPA_list —1, it examines the list and is not identified as a candidate target stack but is identified as a backup to thecommunication protocol stack 22. Thus, thecommunication protocol stack 38 stores the list for use in error recovery. - When any of the communication protocol stacks 22, 26, 30, 34 and 38 receive the
DVIPA_list —2, then note that the “ALL” parameter is identified and add the DVIPA DVB1 as a non-routable VIPA. These communication protocol stacks 22, 26, 30, 34 and 38 the monitor for applications bound DVB1 and listening on port 60 to determines if an application is executing which is bound to DVA1 and listening to port 60. If and when such an application binds to DVB2 and listens on port 60 a SRVSTAT message is sent to thecommunication protocol stack 38 to identify the candidate target stack as a current actual target as described above. Furthermore, if a communication protocol stack is subsequently activated, it too will identify DVB1 as a DVIPA and add DVB1 as a non-routable VIPA. -
FIG. 7 illustrates operations of a routing communication protocol stack when a communication is received from thenetwork 44. As is seen inFIG. 7 , theVIPA distribution function 23 of the communication protocol stack determines if the communication is to a DVIPA associated with the stack (block 140) by, for example, examining the IP address and port of the communication and comparing that with those of the DVIPAs for the protocol stack. If the communication is not to a DVIPA of the protocol stack, then operations of theVIPA distribution function 23 may terminate with respect to the communication. - If the communication is to a DVIPA of the protocol stack, then the
VIPA distribution function 23 determines if the communication is a SYN to establish a connection to the DVIPA (block 142). If so, then theVIPA distribution function 23 may select a current actual target for the connection (i.e. a communication protocol stack with an application bound to the DVIPA and listening to the port specified by the communication)(block 144). Such a selection may, for example, be based on predefined criteria, utilizing a predefined selection pattern, such as round-robin, weighted round-robin or the like, or may be based on dynamic criteria, policies or combinations thereof. For example, the selection may be made to distribute workload between the candidate target stacks. Thus, a workload manger and/or a service policy agent may be consulted in selecting the candidate target stack. - However the selection is made, the
VIPA distribution function 23 updates a current routing table (CRT) which defines the path from the routing communication protocol stack to the selected current actual target (block 146). Such an update may take the form of creating an entry incorporating the source IP address, DVIPA and port and the XCF address of the selected current actual target. The message is also forwarded to the selected current actual target using the XCF address of the current actual target (block 150). - Returning to block 142, if the communication is not a SYN message, then the
VIPA distribution function 23 of the routing communication protocol stack consults the CRT to route the communication and, if an appropriate destination communication protocol stack is found, routes the communication to the current actual target for the communication (block 152). The communication is then forwarded to the current actual target specified by the CRT (block 150). -
FIG. 8 illustrates operations of theVIPA distribution function 23 of the routing communication protocol stack when a message is received from another communication protocol stack. As is seen inFIG. 8 , theVIPA distribution function 23 determines the type of message received (block 160). If the message is a SRVSTAT message, then, as described above, the application and communication protocol stack entries of the DPT are updated (block 166) by theVIPA distribution function 23. As described above, the SRVSTAT message may be sent by a communication protocol stack both when an application instance binds to the DVIPA and listens to an associated port and when the application instance terminates. Thus, the SRVSTAT message may be utilized by theVIPA distribution function 23 to maintain the DPT with up-to-date information as to the current actual targets available for connections (block 166). - Returning to block 160, the
VIPA distribution function 23 may also determine if the message is a new connection message (NEWCONN). Such a message may be generated if an application bound to a DVIPA utilizing a port in the VIPA list initiates a connection or, as described above, if a communication protocol stack receives a VIPA list with a DVIPA which already have applications using the DVIPA for connections, then theVIPA distribution function 23 of the communication protocol stack sends a NEWCONN message to the routing communication protocol stack to notify the routing communication protocol stack of the connection. If the message is a NEWCONN message, then theVIPA distribution function 23 incorporates the connection into the CRT (block 164). Such an incorporation of the connection into the CRT may be carried out as described above for connections originated fromnetwork 44. - A third type of message which may be received by the
VIPA distribution function 23 is a connection termination message (TERMCONN). Such a message may be generated by aVIPA distribution function 23 when a connection terminates. When the connection terminates, theVIPA distribution function 23 of the communication protocol stack sends a TERMCONN message to the routing communication protocol stack to notify the routing communication protocol stack that the connection has ended and routing for the connection may be discontinued. Thus, if the message is a TERMCONN message (block 160), then theVIPA distribution function 23 removes the connection from the CRT (block 162). Such a removal of the connection from the CRT may be carried out by, for example, deleting or invalidating an entry in the CRT corresponding to the connection. - Returning to the example illustrated in
FIG. 4 , when a SYN message to port 60 of DVA1 is received fromnetwork 44 bycommunication protocol stack 22, theVIPA distribution function 23 determines that the SYN is to a dynamic routable VIPA for which it is the routing communication protocol stack, consults it DPT and optionally a workload management function (not shown) and selects a current actual target as a destination for the message. Thus, theVIPA distribution function 23 of thecommunication protocol stack 22 may select thecommunication protocol stack 26 as a destination. Thecommunication protocol stack 22 creates an entry for the connection in its CRT and forwards the message to thecommunication protocol stack 26. Subsequent messages from thenetwork 44 to port 60 of DVA1 from the source IP address will also be routed to thecommunication protocol stack 26 using the CRT entry. - An instance of APP A of the
communication protocol stack 26 bound to DVA1 and utilizing port 60 may also establish a connection overnetwork 44 either directly or through another protocol stack. When such occurs, theVIPA distribution function 23 ofcommunication protocol stack 26 sends a NEWCONN message to the routingcommunication protocol stack 22 identifying the new connection. TheVIPA distribution function 23 ofcommunication protocol stack 22 receives the NEWCONN message and updates its CRT to route communications from the identified new connection to port 60 of DVA1 to thecommunication protocol stack 26. Such an identification may be made by, for example, providing the source IP address of the other end of the connection, the DVIPA and port and the XCF address of thecommunication protocol stack 26 to the routingcommunication protocol stack 22. - In any event, when either of the connections ends, the
VIPA distribution function 23 of thecommunication protocol stack 26 sends a TERMCONN message to theVIPA distribution function 23 of thecommunication protocol stack 22. TheVIPA distribution function 23 of thecommunication protocol stack 22 removes the entry from the CRT corresponding to the function and, thereby, terminates routing for the connection. -
FIGS. 9 and 10 illustrate operations of theVIPA distribution function 23 during failure of a routing communication protocol stack and during recovery of a routing communication protocol stack. As seen inFIG. 9 , when a failure occurs, the other communication protocol stacks in the cluster of data processing systems are notified of the failure (block 200). The communication protocol stack identified as the backup stack for the dynamic routable VIPA takes over routing functions for that DVIPA so as to become a backup routing communication protocol stack. In addition, the backup routing communication protocol stack may broadcast the DVIPA list that it will utilize in routing connections for the DVIPA (block 201). This list may be the list provided by the primary routing communication protocol stack or a list defined explicitly for the backup routing communication protocol stack. Alternatively, the list may need only be distributed in the instance where the backup routing communication protocol stack has an explicitly defined DVIPA list. - As described above, because of the broadcast of information each of the communication protocol stacks is aware that it is a candidate target for a DVIPA and the identity of the highest ranking backup routing communication protocol stack. Therefore, the communication protocol stacks with application instances bound to the DVIPA and listening on an associated port may send a SRVSTAT message and a NEWCONN message(s) for connections to the DVIPA for the communication protocol stack to the backup routing communication protocol stack (block 202). The communication protocol stacks also associate the backup routing communication protocol stack with any connections utilizing the DVIPA so that subsequent messages for the DVIPA are sent to the backup routing communication protocol stack (block 204). The backup routing communication protocol stack utilizes the SRVSTAT messages and its information from the appropriate VIPA list to build a new DPT for the DVIPA (block 206).
- The backup routing communication protocol stack also receives NEWCONN messages from the server communication protocol stacks with existing DVIPA connections and constructs a CRT based on these messages (block 208). The routing information from the constructed CRT is incorporated into the backup routing communication protocol stack's own CRT (block 210). The backup routing communication protocol stack may also send its own DVIPA messages to the other communication protocol stacks to notify the other communication protocol stacks that it is performing the backup function (block 212). Such messages may be sent to prevent other backup communication protocol stacks in a list of backups from taking over the DVIPA. Details on the transfer of a VIPA to a backup are provided in United States Patent Application Serial No. 09/401,419 described above. Furthermore, in particular embodiments, the issuance of the SRVSTAT or the NEWCONN messages may be in response to the DVIPA messages sent by the backup routing communication protocol stack. Thus, embodiments of the present invention are not limited to the sequence illustrated in
FIGS. 9 and 10 . Operations then continue with the backup routing communication protocol stack operating as the routing communication protocol stack described above. - Turning to
FIG. 10 , when the primary communication protocol stack recovers, it again broadcasts its VIPA list which is received by the other communication protocol stacks (block 300). In response to receiving the VIPA list, the candidate target stacks send SRVSTAT messages to the recovered primary routing communication protocol stack (block 302) which identify the stacks which have application instances bound to the DVIPA and utilizing a port of the VIPA list. The recovered primary routing communication protocol stack also sends a DVIPA message to the backup communication protocol stack which receives the takeback message (block 304) and generates a NEWCONN message for all the connections which are routed through the backup communication protocol stack (block 306). The NEWCONN message is sent to the primary routing communication protocol stack (block 308)and the primary routing communication protocol stack constructs a CRT based on the information form the message. - Alternatively, the server stacks may send NEWCONN messages directly to the recovered primary routing stack either in response to receiving the VIPA list or the DVIPA message. In any case, routing of the existing connections may then be performed by the primary routing communication protocol stack. Finally, the backup routing communication protocol stack deletes the DVIPA and cleans up its routing and DPT tables (block 310). Thus, the routing of the DVIPA is returned to the recovered stack.
- In the example illustrated in
FIG. 4 , assume that the communication protocol stacks 26 and 34 have applications with connections routed through the DVIPA of the routingcommunication protocol stack 22. Furthermore, thecommunication protocol stack 38 is defined as the backup to thecommunication protocol stack 22. If thecommunication protocol stack 22 fails, then the communication protocol stacks 26 and 34 send SRVSTAT messages to the backup routingcommunication protocol stack 38 identifying them as current actual targets. The communication protocol stacks 26 and 34 also associate thecommunication protocol stack 38 with the DVIPA DVA1 and send all subsequent messages to the backup routingcommunication protocol stack 38. Thecommunication protocol stack 38 builds its DPT from the SRVSTAT messages, receives NEWCONN messages for connections through the failedcommunication protocol stack 22 and creates routing information for incorporating in its CRT. The backup routingcommunication protocol stack 38 incorporates the routing information into its routing table and begins routing messages for the connections routed through the failed communication protocol stack. The backup routingcommunication protocol stack 38 may also send a DVIPA message to the other communication protocol stacks 26, 30 and 34 to indicate that it is backing up the DVIPA DVA1. - When the primary routing
communication protocol stack 22 is recovered, it sends a VIPA list to the other communication protocol stacks 26, 30, 34 and 38. The VIPA list message signals the other communication protocol stacks 26, 30, 34 and 38 to send SRVSTAT messages to thecommunication protocol stack 22 so as to identify current actual targets for the DVIPA DVA1. Thecommunication protocol stack 22 builds a DPT from the SRVSTAT messages. The backup routingcommunication protocol stack 38 also generates a NEWCONN message for each connection to DVA1 routed through the backup routingcommunication protocol stack 38 and sends this message tot thecommunication protocol stack 22. Alternatively, the server communication stacks may send NEWCONN messages in response to the VIPA list identifying existing connections to the DVIPA. In any case, thecommunication protocol stack 22 builds a CRT from the NEWCONN message(s). The backup routingcommunication protocol stack 38 cleans up its routing tables and deletes DVA1 so that it no longer performs routing for DVA1. Thus, control of DVA1 is transferred back to thecommunication protocol stack 22 with limited disruption to the connections. - As a further example of embodiments of the present invention, operations for the movement of dynamic VIPAs, whether routable or not, will now be described. As mentioned above, a dynamic VIPA may be established by an IOCTL or bind to a VIPA defined in the VIPARange as MOVEable NONDIRUPTIVE or in a VIPADEFine statement as MOVEable IMMEDiate. In either case, the DVIPA messages described above may be utilized to move the VIPA from an existing protocol stack to a new protocol stack with the new protocol stack routing the connection to the existing protocol stack. Thus, the dynamically routable VIPA routing function may be used with non-distributed (existing) dynamic VIPAs to allow immediate movement of a dynamic VIPA from a first protocol stack to a second protocol stack without requiring that all connections to the VIPA at the first protocol stack be terminated. Also, the routing function may allow connections to the first stack to be maintained and new connections created to the second protocol stack by routing the old connections to the first protocol stack until they terminate. Thus, a gradual, non-disruptive takeback of connections may be achieved.
-
FIGS. 11 and 12 illustrate operations of a communication protocol stack to facilitate the moveable VIPA. As seen inFIG. 11 , when a dynamic VIPA is initialized, by a protocol stack it may be determined if the VIPA is a moveable nondisruptive VIPA (block 500). Initialization of the VIPA may occur, for example, by an application binding to a VIPA in a VIPA range which is defined as MOVEable NONDISRUPtive. If so, then the VIPA may be advertised to the routing protocol (block 502). Alternatively, if the IP gateway of a cluster of data processing systems is OSA Express using Queued Direct I/O, the IP Assist function may automatically update the OSA Express adapter when a “home list” of a protocol stack is updated. Thus, in such an automatic update scenario, the operations ofblock 502 may be skipped. It may also be determined if the VIPA is active on another stack (block 504). Such a determination may be made by previous messages from the other stack advertising the existence of the VIPA. If the VIPA is not active on another stack, then the current stack may establish connections using the VIPA without using the other stack as routing stack (block 510). The current stack may also create routing and destination port tables as described above in the event that the VIPA is moved to another stack or in the event that other stacks look to the current stack as a routing communication protocol stack. - If there is a VIPA on another stack (block 504), then the current stack receives connection administration messages from the other stack or stacks identifying existing connections to the stack (block 506). The current stack builds a routing table from these connection administration messages and routes information to these existing connections (block 508). Such a communication sequence may result from the advertisement that the current stack owns the VIPA. The current stack may also receive subsequent requests for connections to the VIPA and termination messages for existing connections to the VIPA so as to manage routing of information to the appropriate communication protocol stacks as described herein. Thus, ownership of the VIPA may be immediately transferred to a new communication protocol stack without disrupting existing connections.
-
FIG. 12 illustrates operations of a communication protocol stack which owns a VIPA in response to a new communication protocol stack claiming ownership. As seen inFIG. 12 , notification of the VIPA on another communication protocol stack is received (block 520) and the VIPA is associated with the new communication protocol stack (block 522) such that all messages concerning new or existing connections are sent to the new communication protocol stack. It may also be determined if there are existing connections to the VIPA (block 524). If no connections exist, then the transfer of ownership is complete merely by associating the VIPA with the new communication protocol stack and deleting the DVIPA from the current communication protocol stack (block 525). However, if connections exist, then an existing connection message is sent to the new communication protocol (block 526). As described herein, such a message may be for all connections associated with the stack or only for those which the stack is an endpoint node. Such a message may be a NEWCONN message as described above. Furthermore, if the current stack was performing any routing functions, these functions are also transferred to the new communication protocol stack and so the routing tables and destination port tables of the current communication protocol stack may be purged of the connections (block 528). - As an example of operations for a moveable VIPA according to the present invention, assume that the
communication protocol stack 38, which functions as a primary routing communication protocol stack for DVIPA DVB1, has not been started. Also assume that thecommunication protocol stack 30 is initialized and an instance APP B started and binds to a VIPA in the VIPARange statement, such as DVB1. In this case, connections to DVB1 would be established to thecommunication protocol stack 30. Subsequently, when thecommunication protocol stack 26 was initialized and an instance of APP B bound to DVB1, thecommunication protocol stack 26 would send a VIPA list identifying thecommunication protocol stack 26 as the owner of the VIPA and, in response, thecommunication protocol stack 30 would send a. NEWCONN message. Thecommunication protocol stack 26 would receive the NEWCONN message and incorporate a route for the connection to thecommunication protocol stack 30 into its routing tables. Thecommunication protocol stack 30 would send all subsequent messages regarding the connection to thecommunication protocol stack 26. When thecommunication protocol stack 38 is initialized, it would assume ownership of the VIPA as the primary routing communication protocol stack and would send a VIPA list to the other communication protocol stacks identifying itself as such. It may also send a takeback message as well to take ownership of the VIPA. The communication protocol stacks 30 and 26 which have existing connections to the VIPA would send a message identifying the existing connections to thecommunication protocol stack 38 which would incorporate routing paths for the connections into its routing table. Subsequent messages regarding the existing connections would be sent to thecommunication protocol stack 38 and subsequent connections would be made through thecommunication protocol stack 38. Thus, ownership of the VIPA was transferred nondisruptively from thecommunication protocol stack 30 to thecommunication protocol stack 26 to thecommunication protocol stack 38. - As used herein, the term “connection administration message” refers to messages between communication protocol stacks which are utilized to manage the routing of TCP/IP messages between the communication protocol stacks. Thus, for example, the NEWCONN, TERMCONN and VIPA lists may be considered connection administration messages.
- While the present invention has been described with respect to the VIPA distribution function as a part of the communication protocol stack, as will be appreciated by those of skill in the art, such functions may be provided as separate functions, objects or applications which may cooperate with the communication protocol stacks. Furthermore, the present invention has been described with reference to particular sequences of operations. However, as will be appreciated by those of skill in the art, other sequences may be utilized while still benefitting from the teachings of the present invention. Thus, while the present invention is described with respect to a particular division of functions or sequences of events, such divisions or sequences are merely illustrative of particular embodiments of the present invention and the present invention should not be construed as limited to such embodiments.
- Furthermore, while the present invention has been described with reference to particular embodiments of the present invention in a System/390 environment, as will be appreciated by those of skill in the art, the present invention may be embodied in other environments and should not be construed as limited to System/390 but may be incorporated into other systems, such as a Unix or other environments by associating applications or groups of applications with an address rather than a communications adapter. Thus, the present invention may be suitable for use in any collection of data processing systems which allow sufficient communication to all of for the use of dynamic virtual addressing. Accordingly, specific references to System/390 systems or facilities, such as the “coupling facility,” “ESCON,” “Sysplex” or the like should not be construed as limiting the present invention.
- In the drawings and specification, there have been disclosed typical preferred embodiments of the invention and, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation, the scope of the invention being set forth in the following claims.
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/045,633 US7430611B2 (en) | 2000-08-17 | 2005-01-28 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/640,409 US6996631B1 (en) | 2000-08-17 | 2000-08-17 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
US11/045,633 US7430611B2 (en) | 2000-08-17 | 2005-01-28 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/640,409 Division US6996631B1 (en) | 2000-08-17 | 2000-08-17 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
Publications (2)
Publication Number | Publication Date |
---|---|
US20050141506A1 true US20050141506A1 (en) | 2005-06-30 |
US7430611B2 US7430611B2 (en) | 2008-09-30 |
Family
ID=34700274
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/640,409 Expired - Fee Related US6996631B1 (en) | 2000-08-17 | 2000-08-17 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
US11/045,633 Expired - Lifetime US7430611B2 (en) | 2000-08-17 | 2005-01-28 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/640,409 Expired - Fee Related US6996631B1 (en) | 2000-08-17 | 2000-08-17 | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
Country Status (1)
Country | Link |
---|---|
US (2) | US6996631B1 (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030002494A1 (en) * | 2001-07-02 | 2003-01-02 | Arttu Kuukankorpi | Processing of data packets within a network element cluster |
US20070094361A1 (en) * | 2005-10-25 | 2007-04-26 | Oracle International Corporation | Multipath routing process |
US20070242671A1 (en) * | 2006-04-13 | 2007-10-18 | Christenson David A | Method and Apparatus for Routing Data Packets Between Different Internet Communications Stack Instances |
US7430611B2 (en) * | 2000-08-17 | 2008-09-30 | International Business Machines Corporation | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
US20110153771A1 (en) * | 2009-12-17 | 2011-06-23 | International Business Machines Corporation | Direct memory access with minimal host interruption |
US9137093B1 (en) * | 2007-07-02 | 2015-09-15 | Comscore, Inc. | Analyzing requests for data made by users that subscribe to a provider of network connectivity |
US9544364B2 (en) | 2012-12-06 | 2017-01-10 | A10 Networks, Inc. | Forwarding policies on a virtual service network |
US9661026B2 (en) | 2006-10-17 | 2017-05-23 | A10 Networks, Inc. | Applying security policy to an application session |
US9742879B2 (en) | 2012-03-29 | 2017-08-22 | A10 Networks, Inc. | Hardware-based packet editor |
US9942152B2 (en) | 2014-03-25 | 2018-04-10 | A10 Networks, Inc. | Forwarding data packets using a service-based forwarding policy |
US9954899B2 (en) | 2006-10-17 | 2018-04-24 | A10 Networks, Inc. | Applying a network traffic policy to an application session |
US9992107B2 (en) | 2013-03-15 | 2018-06-05 | A10 Networks, Inc. | Processing data packets using a policy based network path |
US10038693B2 (en) | 2013-05-03 | 2018-07-31 | A10 Networks, Inc. | Facilitating secure network traffic by an application delivery controller |
CN109343460A (en) * | 2018-11-23 | 2019-02-15 | 无锡信捷电气股份有限公司 | For establishing and the method for plc communication |
US20190068751A1 (en) * | 2017-08-25 | 2019-02-28 | International Business Machines Corporation | Server request management |
US10268467B2 (en) | 2014-11-11 | 2019-04-23 | A10 Networks, Inc. | Policy-driven management of application traffic for providing services to cloud-based applications |
Families Citing this family (76)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080214163A1 (en) * | 2007-01-26 | 2008-09-04 | Richard Onyon | System for and method of backing up content for use on a mobile device |
US8620286B2 (en) | 2004-02-27 | 2013-12-31 | Synchronoss Technologies, Inc. | Method and system for promoting and transferring licensed content and applications |
US8156074B1 (en) | 2000-01-26 | 2012-04-10 | Synchronoss Technologies, Inc. | Data transfer and synchronization system |
US6671757B1 (en) * | 2000-01-26 | 2003-12-30 | Fusionone, Inc. | Data transfer and synchronization system |
US7895334B1 (en) | 2000-07-19 | 2011-02-22 | Fusionone, Inc. | Remote access communication architecture apparatus and method |
US8073954B1 (en) | 2000-07-19 | 2011-12-06 | Synchronoss Technologies, Inc. | Method and apparatus for a secure remote access system |
DE10049610A1 (en) * | 2000-10-05 | 2002-04-18 | Alcatel Sa | Network Management Client |
US20020042839A1 (en) * | 2000-10-10 | 2002-04-11 | Christopher Peiffer | HTTP multiplexor/demultiplexor |
US7818435B1 (en) * | 2000-12-14 | 2010-10-19 | Fusionone, Inc. | Reverse proxy mechanism for retrieving electronic content associated with a local network |
US7340530B2 (en) * | 2001-01-17 | 2008-03-04 | International Business Machines Corporation | Methods, for providing data from network secure communications in a cluster computing environment |
US7426566B2 (en) * | 2001-01-17 | 2008-09-16 | International Business Machines Corporation | Methods, systems and computer program products for security processing inbound communications in a cluster computing environment |
US8615566B1 (en) | 2001-03-23 | 2013-12-24 | Synchronoss Technologies, Inc. | Apparatus and method for operational support of remote network systems |
US20050198379A1 (en) * | 2001-06-13 | 2005-09-08 | Citrix Systems, Inc. | Automatically reconnecting a client across reliable and persistent communication sessions |
US7562146B2 (en) * | 2003-10-10 | 2009-07-14 | Citrix Systems, Inc. | Encapsulating protocol for session persistence and reliability |
US7984157B2 (en) * | 2002-02-26 | 2011-07-19 | Citrix Systems, Inc. | Persistent and reliable session securely traversing network components using an encapsulating protocol |
US7661129B2 (en) | 2002-02-26 | 2010-02-09 | Citrix Systems, Inc. | Secure traversal of network components |
US7937471B2 (en) | 2002-06-03 | 2011-05-03 | Inpro Network Facility, Llc | Creating a public identity for an entity on a network |
US20030236980A1 (en) * | 2002-06-20 | 2003-12-25 | Hsu Raymond T. | Authentication in a communication system |
US8630414B2 (en) * | 2002-06-20 | 2014-01-14 | Qualcomm Incorporated | Inter-working function for a communication system |
US8234358B2 (en) | 2002-08-30 | 2012-07-31 | Inpro Network Facility, Llc | Communicating with an entity inside a private network using an existing connection to initiate communication |
JP4786116B2 (en) | 2002-09-06 | 2011-10-05 | ソニー株式会社 | Information processing apparatus and method, and program |
US20040078772A1 (en) * | 2002-10-16 | 2004-04-22 | Cosine Communications, Inc. | Dynamic route exchange |
US20040249973A1 (en) * | 2003-03-31 | 2004-12-09 | Alkhatib Hasan S. | Group agent |
US20040249974A1 (en) * | 2003-03-31 | 2004-12-09 | Alkhatib Hasan S. | Secure virtual address realm |
US7949785B2 (en) | 2003-03-31 | 2011-05-24 | Inpro Network Facility, Llc | Secure virtual community network system |
US7561527B1 (en) * | 2003-05-02 | 2009-07-14 | David Katz | Bidirectional forwarding detection |
WO2004107131A2 (en) * | 2003-05-28 | 2004-12-09 | Caymas Systems, Inc. | Policy based network address translation |
EP1652048A4 (en) | 2003-07-21 | 2009-04-15 | Fusionone Inc | Device message management system |
US7634509B2 (en) * | 2003-11-07 | 2009-12-15 | Fusionone, Inc. | Personal information space management system and method |
US7978716B2 (en) * | 2003-11-24 | 2011-07-12 | Citrix Systems, Inc. | Systems and methods for providing a VPN solution |
US7483998B2 (en) * | 2003-11-14 | 2009-01-27 | Alcatel Lucent | Software configurable cluster-based router using heterogeneous nodes as cluster nodes |
US7827590B2 (en) | 2003-12-10 | 2010-11-02 | Aventail Llc | Controlling access to a set of resources in a network |
US8661158B2 (en) * | 2003-12-10 | 2014-02-25 | Aventail Llc | Smart tunneling to resources in a network |
US8590032B2 (en) * | 2003-12-10 | 2013-11-19 | Aventail Llc | Rule-based routing to resources through a network |
WO2005059684A2 (en) * | 2003-12-10 | 2005-06-30 | Aventail Corporation | End point control |
US20080082421A1 (en) * | 2004-05-12 | 2008-04-03 | Richard Onyon | Monetization of an advanced contact identification system |
US9542076B1 (en) | 2004-05-12 | 2017-01-10 | Synchronoss Technologies, Inc. | System for and method of updating a personal profile |
JP2008500750A (en) * | 2004-05-12 | 2008-01-10 | フュージョンワン インコーポレイテッド | Advanced contact identification system |
US7649854B2 (en) * | 2004-05-19 | 2010-01-19 | Bea Systems, Inc. | System and method for providing channels in application servers and transaction-based systems |
US8495305B2 (en) | 2004-06-30 | 2013-07-23 | Citrix Systems, Inc. | Method and device for performing caching of dynamically generated objects in a data communication network |
US8739274B2 (en) * | 2004-06-30 | 2014-05-27 | Citrix Systems, Inc. | Method and device for performing integrated caching in a data communication network |
US7526557B2 (en) * | 2004-06-30 | 2009-04-28 | Signiant, Inc. | System and method for transferring data in high latency firewalled networks |
US7757074B2 (en) | 2004-06-30 | 2010-07-13 | Citrix Application Networking, Llc | System and method for establishing a virtual private network |
CA2574776A1 (en) * | 2004-07-23 | 2006-02-02 | Citrix Systems, Inc. | Systems and methods for optimizing communications between network nodes |
CN101199187A (en) | 2004-07-23 | 2008-06-11 | 茨特里克斯系统公司 | A method and systems for securing remote access to private networks |
US8094821B2 (en) * | 2004-08-06 | 2012-01-10 | Qualcomm Incorporated | Key generation in a communication system |
WO2006020823A1 (en) * | 2004-08-13 | 2006-02-23 | Citrix Systems, Inc. | A method for maintaining transaction integrity across multiple remote access servers |
TW200608224A (en) * | 2004-08-23 | 2006-03-01 | Lite On It Corp | Player and method for processing a file with vector-based format |
WO2006044820A2 (en) | 2004-10-14 | 2006-04-27 | Aventail Corporation | Rule-based routing to resources through a network |
US8700695B2 (en) * | 2004-12-30 | 2014-04-15 | Citrix Systems, Inc. | Systems and methods for providing client-side accelerated access to remote applications via TCP pooling |
US8549149B2 (en) | 2004-12-30 | 2013-10-01 | Citrix Systems, Inc. | Systems and methods for providing client-side accelerated access to remote applications via TCP multiplexing |
US8706877B2 (en) * | 2004-12-30 | 2014-04-22 | Citrix Systems, Inc. | Systems and methods for providing client-side dynamic redirection to bypass an intermediary |
US8954595B2 (en) | 2004-12-30 | 2015-02-10 | Citrix Systems, Inc. | Systems and methods for providing client-side accelerated access to remote applications via TCP buffering |
US7810089B2 (en) | 2004-12-30 | 2010-10-05 | Citrix Systems, Inc. | Systems and methods for automatic installation and execution of a client-side acceleration program |
CN102123178B (en) * | 2005-01-24 | 2014-04-09 | 茨特里克斯系统公司 | Systems and methods for performing caching of dynamically generated objects in network |
US8255456B2 (en) | 2005-12-30 | 2012-08-28 | Citrix Systems, Inc. | System and method for performing flash caching of dynamically generated objects in a data communication network |
KR20080017313A (en) * | 2005-05-19 | 2008-02-26 | 퓨전원 인코포레이티드 | Remote cell phone auto destruct |
US8301839B2 (en) * | 2005-12-30 | 2012-10-30 | Citrix Systems, Inc. | System and method for performing granular invalidation of cached dynamically generated objects in a data communication network |
US7921184B2 (en) * | 2005-12-30 | 2011-04-05 | Citrix Systems, Inc. | System and method for performing flash crowd caching of dynamically generated objects in a data communication network |
US8160062B2 (en) * | 2006-01-31 | 2012-04-17 | Microsoft Corporation | Network connectivity determination based on passive analysis of connection-oriented path information |
US8677479B2 (en) * | 2007-04-16 | 2014-03-18 | Microsoft Corporation | Detection of adversaries through collection and correlation of assessments |
US8181111B1 (en) | 2007-12-31 | 2012-05-15 | Synchronoss Technologies, Inc. | System and method for providing social context to digital activity |
JP5222573B2 (en) * | 2008-01-29 | 2013-06-26 | 株式会社日立製作所 | Server computer and network processing method |
US7900016B2 (en) * | 2008-02-01 | 2011-03-01 | International Business Machines Corporation | Full virtualization of resources across an IP interconnect |
US7904693B2 (en) * | 2008-02-01 | 2011-03-08 | International Business Machines Corporation | Full virtualization of resources across an IP interconnect using page frame table |
US9237175B2 (en) * | 2008-12-22 | 2016-01-12 | Microsoft Technology Licensing, Llc | Internet protocol (IP) address virtualization for terminal server sessions |
US8255006B1 (en) | 2009-11-10 | 2012-08-28 | Fusionone, Inc. | Event dependent notification system and method |
US8943428B2 (en) | 2010-11-01 | 2015-01-27 | Synchronoss Technologies, Inc. | System for and method of field mapping |
CN101984623B (en) * | 2010-11-02 | 2013-09-18 | 北京天融信科技有限公司 | Firewall network address translation dynamic load balancing method and device |
US8930475B1 (en) | 2012-03-30 | 2015-01-06 | Signiant Inc. | Systems and methods for secure cloud-based media file sharing |
US9692799B2 (en) | 2012-07-30 | 2017-06-27 | Signiant Inc. | System and method for sending and/or receiving digital content based on a delivery specification |
US9917740B2 (en) | 2015-09-09 | 2018-03-13 | International Business Machines Corporation | Reducing internodal communications in a clustered system |
US10735516B1 (en) | 2019-02-15 | 2020-08-04 | Signiant Inc. | Cloud-based authority to enhance point-to-point data transfer with machine learning |
US11301279B2 (en) | 2019-02-26 | 2022-04-12 | International Business Machines Corporation | Associating virtual IP address of virtual server with appropriate operating system in server cluster |
US10735319B1 (en) | 2019-02-26 | 2020-08-04 | International Business Machines Corporation | Virtual container extended network virtualization in server cluster |
US10728146B1 (en) | 2019-02-26 | 2020-07-28 | International Business Machines Corporation | Virtual container dynamic virtual IP address |
Citations (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5341477A (en) * | 1989-02-24 | 1994-08-23 | Digital Equipment Corporation | Broker for computer network server selection |
US5537542A (en) * | 1994-04-04 | 1996-07-16 | International Business Machines Corporation | Apparatus and method for managing a server workload according to client performance goals in a client/server data processing system |
US5603029A (en) * | 1995-06-07 | 1997-02-11 | International Business Machines Corporation | System of assigning work requests based on classifying into an eligible class where the criteria is goal oriented and capacity information is available |
US5740371A (en) * | 1995-09-30 | 1998-04-14 | International Business Machines Corporation | Load balancing of connections to parallel servers |
US5751971A (en) * | 1995-07-12 | 1998-05-12 | Cabletron Systems, Inc. | Internet protocol (IP) work group routing |
US5754752A (en) * | 1996-03-28 | 1998-05-19 | Tandem Computers Incorporated | End-to-end session recovery |
US5774668A (en) * | 1995-06-07 | 1998-06-30 | Microsoft Corporation | System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing |
US5796936A (en) * | 1993-03-01 | 1998-08-18 | Hitachi, Ltd. | Distributed control system in which individual controllers executed by sharing loads |
US5812771A (en) * | 1994-01-28 | 1998-09-22 | Cabletron System, Inc. | Distributed chassis agent for distributed network management |
US5828847A (en) * | 1996-04-19 | 1998-10-27 | Storage Technology Corporation | Dynamic server switching for maximum server availability and load balancing |
US5867661A (en) * | 1996-02-15 | 1999-02-02 | International Business Machines Corporation | Method and apparatus of using virtual sockets for reducing data transmitted over a wireless communication link between a client web browser and a host web server using a standard TCP protocol |
US5867636A (en) * | 1995-06-06 | 1999-02-02 | Apple Computer, Inc. | Client server symmetric presentation-layer connection protocol for network printing systems |
US5918017A (en) * | 1996-08-23 | 1999-06-29 | Internatioinal Business Machines Corp. | System and method for providing dynamically alterable computer clusters for message routing |
US5941988A (en) * | 1997-01-27 | 1999-08-24 | International Business Machines Corporation | Session and transport layer proxies via TCP glue |
US5946686A (en) * | 1997-07-11 | 1999-08-31 | International Business Machines Corporation | Parallel file system and method with quota allocation |
US6031978A (en) * | 1996-06-28 | 2000-02-29 | International Business Machines Corporation | System, method and program for enabling a client to reconnect to a same server in a network of computer systems after the server has moved to a different network address |
US6119174A (en) * | 1998-10-13 | 2000-09-12 | Hewlett-Packard Company | Methods and apparatus for implementing quality-of-service guarantees in data storage systems |
US6128279A (en) * | 1997-10-06 | 2000-10-03 | Web Balance, Inc. | System for balancing loads among network servers |
US6141759A (en) * | 1997-12-10 | 2000-10-31 | Bmc Software, Inc. | System and architecture for distributing, monitoring, and managing information requests on a computer network |
US6223205B1 (en) * | 1997-10-20 | 2001-04-24 | Mor Harchol-Balter | Method and apparatus for assigning tasks in a distributed server system |
US6247057B1 (en) * | 1998-10-22 | 2001-06-12 | Microsoft Corporation | Network server supporting multiple instance of services to operate concurrently by having endpoint mapping subsystem for mapping virtual network names to virtual endpoint IDs |
US6252878B1 (en) * | 1997-10-30 | 2001-06-26 | Cisco Technology, Inc. | Switched architecture access server |
US6262976B1 (en) * | 1998-09-17 | 2001-07-17 | Ordered Networks, Inc. | System and method for network flow optimization using traffic classes |
US6286039B1 (en) * | 1997-08-28 | 2001-09-04 | Cisco Technology, Inc. | Automatic static to dynamic IP address and DNS address management for remote communications network access |
US6314463B1 (en) * | 1998-05-29 | 2001-11-06 | Webspective Software, Inc. | Method and system for measuring queue length and delay |
US6317786B1 (en) * | 1998-05-29 | 2001-11-13 | Webspective Software, Inc. | Web service |
US6330560B1 (en) * | 1999-09-10 | 2001-12-11 | International Business Machines Corporation | Multiple manager to multiple server IP locking mechanism in a directory-enabled network |
US20020010783A1 (en) * | 1999-12-06 | 2002-01-24 | Leonard Primak | System and method for enhancing operation of a web server cluster |
US6353614B1 (en) * | 1998-03-05 | 2002-03-05 | 3Com Corporation | Method and protocol for distributed network address translation |
US6363081B1 (en) * | 1998-03-04 | 2002-03-26 | Hewlett-Packard Company | System and method for sharing a network port among multiple applications |
US6374300B2 (en) * | 1999-07-15 | 2002-04-16 | F5 Networks, Inc. | Method and system for storing load balancing information with an HTTP cookie |
US20020091831A1 (en) * | 2000-11-10 | 2002-07-11 | Michael Johnson | Internet modem streaming socket method |
US6430622B1 (en) * | 1999-09-22 | 2002-08-06 | International Business Machines Corporation | Methods, systems and computer program products for automated movement of IP addresses within a cluster |
US6445704B1 (en) * | 1997-05-02 | 2002-09-03 | Cisco Technology, Inc. | Method and apparatus for virtualizing a locally initiated outbound connection from a connection manager |
US6446225B1 (en) * | 1998-04-23 | 2002-09-03 | Microsoft Corporation | Server system with scalable session timeout mechanism |
US20020124089A1 (en) * | 2000-08-17 | 2002-09-05 | Aiken John Andrew | Methods, systems and computer program products for cluster workload distribution without preconfigured port identification |
US20020143954A1 (en) * | 2001-04-03 | 2002-10-03 | Aiken John Andrew | Methods, systems and computer program products for content-based routing via active TCP connection transfer |
US20020143953A1 (en) * | 2001-04-03 | 2002-10-03 | International Business Machines Corporation | Automatic affinity within networks performing workload balancing |
US20020178268A1 (en) * | 2001-05-22 | 2002-11-28 | Aiken John Andrew | Methods, systems and computer program products for port assignments of multiple application instances using the same source IP address |
US20020178265A1 (en) * | 2001-05-22 | 2002-11-28 | Aiken John Andrew | Methods systems and computer program products for source address selection |
US20020199000A1 (en) * | 2001-06-26 | 2002-12-26 | International Business Machines Corporation | Method and system for managing parallel data transfer through multiple sockets to provide scalability to a computer network |
US20030061402A1 (en) * | 2001-09-26 | 2003-03-27 | Satyendra Yadav | Method and apparatus enabling both legacy and new applications to access an InfiniBand fabric via a socket API |
US6542926B2 (en) * | 1998-06-10 | 2003-04-01 | Compaq Information Technologies Group, L.P. | Software partitioned multi-processor system with flexible resource sharing levels |
US20030079146A1 (en) * | 2001-10-24 | 2003-04-24 | Microsoft Corporation | Method and apparatus for regulating access to a computer via a computer network |
US6578066B1 (en) * | 1999-09-17 | 2003-06-10 | Alteon Websystems | Distributed load-balancing internet servers |
US6587866B1 (en) * | 2000-01-10 | 2003-07-01 | Sun Microsystems, Inc. | Method for distributing packets to server nodes using network client affinity and packet distribution table |
US6591262B1 (en) * | 2000-08-01 | 2003-07-08 | International Business Machines Corporation | Collaborative workload management incorporating work unit attributes in resource allocation |
US6594268B1 (en) * | 1999-03-11 | 2003-07-15 | Lucent Technologies Inc. | Adaptive routing system and method for QOS packet networks |
US20030202536A1 (en) * | 2001-04-27 | 2003-10-30 | Foster Michael S. | Integrated analysis of incoming data transmissions |
US6701377B2 (en) * | 1997-09-08 | 2004-03-02 | Phoenix Contact Gmbh & Co. Kg | Automation system and connecting apparatus for communication between two networks that use two different protocols with conversion between TCP/IP and PCP |
US6704317B1 (en) * | 1998-05-27 | 2004-03-09 | 3Com Corporation | Multi-carrier LAN modem server |
US6718383B1 (en) * | 2000-06-02 | 2004-04-06 | Sun Microsystems, Inc. | High availability networking with virtual IP address failover |
US6748414B1 (en) * | 1999-11-15 | 2004-06-08 | International Business Machines Corporation | Method and apparatus for the load balancing of non-identical servers in a network environment |
US6748413B1 (en) * | 1999-11-15 | 2004-06-08 | International Business Machines Corporation | Method and apparatus for load balancing of parallel servers in a network environment |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4495570A (en) | 1981-01-14 | 1985-01-22 | Hitachi, Ltd. | Processing request allocator for assignment of loads in a distributed processing system |
US4403286A (en) | 1981-03-06 | 1983-09-06 | International Business Machines Corporation | Balancing data-processing work loads |
US4577272A (en) | 1983-06-27 | 1986-03-18 | E-Systems, Inc. | Fault tolerant and load sharing processing system |
US5031089A (en) | 1988-12-30 | 1991-07-09 | United States Of America As Represented By The Administrator, National Aeronautics And Space Administration | Dynamic resource allocation scheme for distributed heterogeneous computer systems |
GB2281793A (en) | 1993-09-11 | 1995-03-15 | Ibm | A data processing system for providing user load levelling in a network |
US5563878A (en) | 1995-01-05 | 1996-10-08 | International Business Machines Corporation | Transaction message routing in digital communication networks |
US5675739A (en) | 1995-02-03 | 1997-10-07 | International Business Machines Corporation | Apparatus and method for managing a distributed data processing system workload according to a plurality of distinct processing goal types |
US5774660A (en) | 1996-08-05 | 1998-06-30 | Resonate, Inc. | World-wide-web server with delayed resource-binding for resource-based load balancing on a distributed resource multi-node network |
US5923854A (en) | 1996-11-22 | 1999-07-13 | International Business Machines Corporation | Virtual internet protocol (IP) addressing |
US5917997A (en) | 1996-12-06 | 1999-06-29 | International Business Machines Corporation | Host identity takeover using virtual internet protocol (IP) addressing |
US5951650A (en) | 1997-01-31 | 1999-09-14 | International Business Machines Corporation | Session traffic splitting using virtual internet protocol addresses associated with distinct categories of application programs irrespective of destination IP address |
US5935215A (en) | 1997-03-21 | 1999-08-10 | International Business Machines Corporation | Methods and systems for actively updating routing in TCP/IP connections using TCP/IP messages |
US6650641B1 (en) | 1999-07-02 | 2003-11-18 | Cisco Technology, Inc. | Network address translation using a forwarding agent |
US6606315B1 (en) | 1999-07-02 | 2003-08-12 | Cisco Technology, Inc. | Synchronizing service instructions among forwarding agents using a service manager |
US6657974B1 (en) | 2000-04-14 | 2003-12-02 | International Business Machines Corporation | Method and apparatus for generating replies to address resolution protocol requests |
US6996631B1 (en) * | 2000-08-17 | 2006-02-07 | International Business Machines Corporation | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
-
2000
- 2000-08-17 US US09/640,409 patent/US6996631B1/en not_active Expired - Fee Related
-
2005
- 2005-01-28 US US11/045,633 patent/US7430611B2/en not_active Expired - Lifetime
Patent Citations (59)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5341477A (en) * | 1989-02-24 | 1994-08-23 | Digital Equipment Corporation | Broker for computer network server selection |
US5796936A (en) * | 1993-03-01 | 1998-08-18 | Hitachi, Ltd. | Distributed control system in which individual controllers executed by sharing loads |
US5812771A (en) * | 1994-01-28 | 1998-09-22 | Cabletron System, Inc. | Distributed chassis agent for distributed network management |
US5537542A (en) * | 1994-04-04 | 1996-07-16 | International Business Machines Corporation | Apparatus and method for managing a server workload according to client performance goals in a client/server data processing system |
US5867636A (en) * | 1995-06-06 | 1999-02-02 | Apple Computer, Inc. | Client server symmetric presentation-layer connection protocol for network printing systems |
US5603029A (en) * | 1995-06-07 | 1997-02-11 | International Business Machines Corporation | System of assigning work requests based on classifying into an eligible class where the criteria is goal oriented and capacity information is available |
US5951694A (en) * | 1995-06-07 | 1999-09-14 | Microsoft Corporation | Method of redirecting a client service session to a second application server without interrupting the session by forwarding service-specific information to the second server |
US5774668A (en) * | 1995-06-07 | 1998-06-30 | Microsoft Corporation | System for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing |
US5751971A (en) * | 1995-07-12 | 1998-05-12 | Cabletron Systems, Inc. | Internet protocol (IP) work group routing |
US6249820B1 (en) * | 1995-07-12 | 2001-06-19 | Cabletron Systems, Inc. | Internet protocol (IP) work group routing |
US5740371A (en) * | 1995-09-30 | 1998-04-14 | International Business Machines Corporation | Load balancing of connections to parallel servers |
US5867661A (en) * | 1996-02-15 | 1999-02-02 | International Business Machines Corporation | Method and apparatus of using virtual sockets for reducing data transmitted over a wireless communication link between a client web browser and a host web server using a standard TCP protocol |
US5754752A (en) * | 1996-03-28 | 1998-05-19 | Tandem Computers Incorporated | End-to-end session recovery |
US5828847A (en) * | 1996-04-19 | 1998-10-27 | Storage Technology Corporation | Dynamic server switching for maximum server availability and load balancing |
US6031978A (en) * | 1996-06-28 | 2000-02-29 | International Business Machines Corporation | System, method and program for enabling a client to reconnect to a same server in a network of computer systems after the server has moved to a different network address |
US5918017A (en) * | 1996-08-23 | 1999-06-29 | Internatioinal Business Machines Corp. | System and method for providing dynamically alterable computer clusters for message routing |
US6496866B2 (en) * | 1996-08-23 | 2002-12-17 | International Business Machines Corporation | System and method for providing dynamically alterable computer clusters for message routing |
US20020166080A1 (en) * | 1996-08-23 | 2002-11-07 | Clement Richard Attanasio | System and method for providing dynamically alterable computer clusters for message routing |
US5941988A (en) * | 1997-01-27 | 1999-08-24 | International Business Machines Corporation | Session and transport layer proxies via TCP glue |
US6445704B1 (en) * | 1997-05-02 | 2002-09-03 | Cisco Technology, Inc. | Method and apparatus for virtualizing a locally initiated outbound connection from a connection manager |
US5946686A (en) * | 1997-07-11 | 1999-08-31 | International Business Machines Corporation | Parallel file system and method with quota allocation |
US6286039B1 (en) * | 1997-08-28 | 2001-09-04 | Cisco Technology, Inc. | Automatic static to dynamic IP address and DNS address management for remote communications network access |
US6701377B2 (en) * | 1997-09-08 | 2004-03-02 | Phoenix Contact Gmbh & Co. Kg | Automation system and connecting apparatus for communication between two networks that use two different protocols with conversion between TCP/IP and PCP |
US6128279A (en) * | 1997-10-06 | 2000-10-03 | Web Balance, Inc. | System for balancing loads among network servers |
US6223205B1 (en) * | 1997-10-20 | 2001-04-24 | Mor Harchol-Balter | Method and apparatus for assigning tasks in a distributed server system |
US6252878B1 (en) * | 1997-10-30 | 2001-06-26 | Cisco Technology, Inc. | Switched architecture access server |
US6141759A (en) * | 1997-12-10 | 2000-10-31 | Bmc Software, Inc. | System and architecture for distributing, monitoring, and managing information requests on a computer network |
US6363081B1 (en) * | 1998-03-04 | 2002-03-26 | Hewlett-Packard Company | System and method for sharing a network port among multiple applications |
US6353614B1 (en) * | 1998-03-05 | 2002-03-05 | 3Com Corporation | Method and protocol for distributed network address translation |
US6697354B1 (en) * | 1998-03-05 | 2004-02-24 | 3Com Corporation | Method and system for distributed network address translation for mobile network devices |
US6446225B1 (en) * | 1998-04-23 | 2002-09-03 | Microsoft Corporation | Server system with scalable session timeout mechanism |
US6704317B1 (en) * | 1998-05-27 | 2004-03-09 | 3Com Corporation | Multi-carrier LAN modem server |
US6317786B1 (en) * | 1998-05-29 | 2001-11-13 | Webspective Software, Inc. | Web service |
US6314463B1 (en) * | 1998-05-29 | 2001-11-06 | Webspective Software, Inc. | Method and system for measuring queue length and delay |
US6542926B2 (en) * | 1998-06-10 | 2003-04-01 | Compaq Information Technologies Group, L.P. | Software partitioned multi-processor system with flexible resource sharing levels |
US6262976B1 (en) * | 1998-09-17 | 2001-07-17 | Ordered Networks, Inc. | System and method for network flow optimization using traffic classes |
US6119174A (en) * | 1998-10-13 | 2000-09-12 | Hewlett-Packard Company | Methods and apparatus for implementing quality-of-service guarantees in data storage systems |
US6247057B1 (en) * | 1998-10-22 | 2001-06-12 | Microsoft Corporation | Network server supporting multiple instance of services to operate concurrently by having endpoint mapping subsystem for mapping virtual network names to virtual endpoint IDs |
US6594268B1 (en) * | 1999-03-11 | 2003-07-15 | Lucent Technologies Inc. | Adaptive routing system and method for QOS packet networks |
US6374300B2 (en) * | 1999-07-15 | 2002-04-16 | F5 Networks, Inc. | Method and system for storing load balancing information with an HTTP cookie |
US6330560B1 (en) * | 1999-09-10 | 2001-12-11 | International Business Machines Corporation | Multiple manager to multiple server IP locking mechanism in a directory-enabled network |
US6578066B1 (en) * | 1999-09-17 | 2003-06-10 | Alteon Websystems | Distributed load-balancing internet servers |
US6430622B1 (en) * | 1999-09-22 | 2002-08-06 | International Business Machines Corporation | Methods, systems and computer program products for automated movement of IP addresses within a cluster |
US6748413B1 (en) * | 1999-11-15 | 2004-06-08 | International Business Machines Corporation | Method and apparatus for load balancing of parallel servers in a network environment |
US6748414B1 (en) * | 1999-11-15 | 2004-06-08 | International Business Machines Corporation | Method and apparatus for the load balancing of non-identical servers in a network environment |
US20020010783A1 (en) * | 1999-12-06 | 2002-01-24 | Leonard Primak | System and method for enhancing operation of a web server cluster |
US6587866B1 (en) * | 2000-01-10 | 2003-07-01 | Sun Microsystems, Inc. | Method for distributing packets to server nodes using network client affinity and packet distribution table |
US6718383B1 (en) * | 2000-06-02 | 2004-04-06 | Sun Microsystems, Inc. | High availability networking with virtual IP address failover |
US6591262B1 (en) * | 2000-08-01 | 2003-07-08 | International Business Machines Corporation | Collaborative workload management incorporating work unit attributes in resource allocation |
US20020124089A1 (en) * | 2000-08-17 | 2002-09-05 | Aiken John Andrew | Methods, systems and computer program products for cluster workload distribution without preconfigured port identification |
US20020091831A1 (en) * | 2000-11-10 | 2002-07-11 | Michael Johnson | Internet modem streaming socket method |
US20020143954A1 (en) * | 2001-04-03 | 2002-10-03 | Aiken John Andrew | Methods, systems and computer program products for content-based routing via active TCP connection transfer |
US20020143953A1 (en) * | 2001-04-03 | 2002-10-03 | International Business Machines Corporation | Automatic affinity within networks performing workload balancing |
US20030202536A1 (en) * | 2001-04-27 | 2003-10-30 | Foster Michael S. | Integrated analysis of incoming data transmissions |
US20020178268A1 (en) * | 2001-05-22 | 2002-11-28 | Aiken John Andrew | Methods, systems and computer program products for port assignments of multiple application instances using the same source IP address |
US20020178265A1 (en) * | 2001-05-22 | 2002-11-28 | Aiken John Andrew | Methods systems and computer program products for source address selection |
US20020199000A1 (en) * | 2001-06-26 | 2002-12-26 | International Business Machines Corporation | Method and system for managing parallel data transfer through multiple sockets to provide scalability to a computer network |
US20030061402A1 (en) * | 2001-09-26 | 2003-03-27 | Satyendra Yadav | Method and apparatus enabling both legacy and new applications to access an InfiniBand fabric via a socket API |
US20030079146A1 (en) * | 2001-10-24 | 2003-04-24 | Microsoft Corporation | Method and apparatus for regulating access to a computer via a computer network |
Cited By (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7430611B2 (en) * | 2000-08-17 | 2008-09-30 | International Business Machines Corporation | System having a single IP address associated with communication protocol stacks in a cluster of processing systems |
US7130305B2 (en) * | 2001-07-02 | 2006-10-31 | Stonesoft Oy | Processing of data packets within a network element cluster |
US20030002494A1 (en) * | 2001-07-02 | 2003-01-02 | Arttu Kuukankorpi | Processing of data packets within a network element cluster |
US20070094361A1 (en) * | 2005-10-25 | 2007-04-26 | Oracle International Corporation | Multipath routing process |
US8166197B2 (en) * | 2005-10-25 | 2012-04-24 | Oracle International Corporation | Multipath routing process |
US8706906B2 (en) | 2005-10-25 | 2014-04-22 | Oracle International Corporation | Multipath routing process |
US20070242671A1 (en) * | 2006-04-13 | 2007-10-18 | Christenson David A | Method and Apparatus for Routing Data Packets Between Different Internet Communications Stack Instances |
US9954899B2 (en) | 2006-10-17 | 2018-04-24 | A10 Networks, Inc. | Applying a network traffic policy to an application session |
US10305859B2 (en) | 2006-10-17 | 2019-05-28 | A10 Networks, Inc. | Applying security policy to an application session |
US9661026B2 (en) | 2006-10-17 | 2017-05-23 | A10 Networks, Inc. | Applying security policy to an application session |
US9137093B1 (en) * | 2007-07-02 | 2015-09-15 | Comscore, Inc. | Analyzing requests for data made by users that subscribe to a provider of network connectivity |
US10063636B2 (en) | 2007-07-02 | 2018-08-28 | Comscore, Inc. | Analyzing requests for data made by users that subscribe to a provider of network connectivity |
US8527664B2 (en) * | 2009-12-17 | 2013-09-03 | International Business Machines Corporation | Direct memory access with minimal host interruption |
US20110153771A1 (en) * | 2009-12-17 | 2011-06-23 | International Business Machines Corporation | Direct memory access with minimal host interruption |
US9742879B2 (en) | 2012-03-29 | 2017-08-22 | A10 Networks, Inc. | Hardware-based packet editor |
US10069946B2 (en) | 2012-03-29 | 2018-09-04 | A10 Networks, Inc. | Hardware-based packet editor |
US9544364B2 (en) | 2012-12-06 | 2017-01-10 | A10 Networks, Inc. | Forwarding policies on a virtual service network |
US10341427B2 (en) | 2012-12-06 | 2019-07-02 | A10 Networks, Inc. | Forwarding policies on a virtual service network |
US9992107B2 (en) | 2013-03-15 | 2018-06-05 | A10 Networks, Inc. | Processing data packets using a policy based network path |
US10659354B2 (en) | 2013-03-15 | 2020-05-19 | A10 Networks, Inc. | Processing data packets using a policy based network path |
US10305904B2 (en) | 2013-05-03 | 2019-05-28 | A10 Networks, Inc. | Facilitating secure network traffic by an application delivery controller |
US10038693B2 (en) | 2013-05-03 | 2018-07-31 | A10 Networks, Inc. | Facilitating secure network traffic by an application delivery controller |
US9942152B2 (en) | 2014-03-25 | 2018-04-10 | A10 Networks, Inc. | Forwarding data packets using a service-based forwarding policy |
US10268467B2 (en) | 2014-11-11 | 2019-04-23 | A10 Networks, Inc. | Policy-driven management of application traffic for providing services to cloud-based applications |
US20190068751A1 (en) * | 2017-08-25 | 2019-02-28 | International Business Machines Corporation | Server request management |
US10749983B2 (en) | 2017-08-25 | 2020-08-18 | International Business Machines Corporation | Server request management |
US10834230B2 (en) * | 2017-08-25 | 2020-11-10 | International Business Machines Corporation | Server request management |
CN109343460A (en) * | 2018-11-23 | 2019-02-15 | 无锡信捷电气股份有限公司 | For establishing and the method for plc communication |
Also Published As
Publication number | Publication date |
---|---|
US6996631B1 (en) | 2006-02-07 |
US7430611B2 (en) | 2008-09-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7430611B2 (en) | System having a single IP address associated with communication protocol stacks in a cluster of processing systems | |
US6996617B1 (en) | Methods, systems and computer program products for non-disruptively transferring a virtual internet protocol address between communication protocol stacks | |
US6941384B1 (en) | Methods, systems and computer program products for failure recovery for routed virtual internet protocol addresses | |
US6954784B2 (en) | Systems, method and computer program products for cluster workload distribution without preconfigured port identification by utilizing a port of multiple ports associated with a single IP address | |
US6430622B1 (en) | Methods, systems and computer program products for automated movement of IP addresses within a cluster | |
US6941366B2 (en) | Methods, systems and computer program products for transferring security processing between processors in a cluster computing environment | |
US7146432B2 (en) | Methods, systems and computer program products for providing failure recovery of network secure communications in a cluster computing environment | |
US8972475B2 (en) | Network secure communications in a cluster computing environment | |
US9276834B2 (en) | Load sharing and redundancy scheme | |
US7859992B2 (en) | Router redundancy in data communication networks | |
US7609619B2 (en) | Active-active data center using RHI, BGP, and IGP anycast for disaster recovery and load distribution | |
US7519721B2 (en) | Computer program products for security processing inbound communications in a cluster computing environment | |
US6397260B1 (en) | Automatic load sharing for network routers | |
EP1653711B1 (en) | Fault tolerant network architecture | |
KR100984384B1 (en) | System, network device, method, and computer program product for active load balancing using clustered nodes as authoritative domain name servers | |
US7711831B2 (en) | Methods, systems and computer program products for source address selection | |
US7903543B2 (en) | Method, apparatus and program storage device for providing mutual failover and load-balancing between interfaces in a network | |
JP3583049B2 (en) | Router monitoring system in data transmission system using network dispatcher for host cluster | |
US8891358B2 (en) | Method for application broadcast forwarding for routers running redundancy protocols | |
JPH10198642A (en) | Server device | |
US7107350B2 (en) | Methods, systems and computer program products for security processing outbound communications in a cluster computing environment | |
JP3974925B2 (en) | Server device | |
Cisco | Designing APPN Internetworks | |
Cisco | Designing APPN Internetworks | |
Cisco | Designing APPN Internetworks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
REMI | Maintenance fee reminder mailed | ||
FPAY | Fee payment |
Year of fee payment: 4 |
|
SULP | Surcharge for late payment | ||
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AIKEN, JOHN ANDREW, JR.;DEVINE, WESLEY MCMILLAN;HERR, DAVID ANTHONY;AND OTHERS;SIGNING DATES FROM 20010518 TO 20010521;REEL/FRAME:039936/0504 |
|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY DATA AND ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED ON REEL 039936 FRAME 0504. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS' INTEREST;ASSIGNORS:AIKEN, JOHN ANDREW;CHRISTENSEN, ALFRED BUNDGAARD;DEVINE, WESLEY MCMILLAN;AND OTHERS;SIGNING DATES FROM 20000811 TO 20000817;REEL/FRAME:040286/0945 Owner name: A10 NETWORKS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:039968/0597 Effective date: 20160927 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |