WO2003090106A1

WO2003090106A1 - Method, apparatus, and computer program product for redundant network

Info

Publication number: WO2003090106A1
Application number: PCT/US2003/011682
Authority: WO
Inventors: Christopher K. Neitzert
Original assignee: Redundant Networks Inc.
Priority date: 2002-04-15
Filing date: 2003-04-15
Publication date: 2003-10-30
Also published as: AU2003228546A1

Abstract

A computer network comprising a plurality of data centers and redundant connections is disclosed. The network comprises a first public Internet service connecting the data centers a second public Internet service connecting the data centers and a virtual private network connection connecting the data centers. The computer network also comprises a first wireless communication connection between a first data center and the first public Internet service and a second wireless communication connection between a second data center and the first public Internet service. The computer network further comprises a first sonet connecting the first data center to the first public Internet service, a second sonet connecting the first data center to the second public Internet service a third sonet connecting the second data center to a third public Internet service and a fourth sonet connecting the second data center to a fourth public Internet service. Each data center comprises a plurality of routers that direct data across the data center to customer computers connected to the data center. The plurality of routers comprises a first edge router connecting the data center to a first public Internet connection, a second edge router connecting the data center to a second public Internet connection, a first router/switch connected to the first router, a second router/switch connected to the second first router, a private network edge router, and a terminal server router, wherein the first and second router/switches implement a core that is connected to and passes data between customer equipment maintaining data on the data centers.

Description

METHOD, APPARATUS, AND COMPUTER PROGRAM PRODUCT FOR REDUNDANT NETWORK

FIELD

The present invention relates broadly to the field of telecommunication networks. Specifically, the present invention relates to data centers connected by a telecommunication network having redundancy and load balancing capabilities.

BACKGROUND

With the proliferation of computing resources in business organizations, reliable data transport between geographically diverse locations requires significant efforts and facilities. Information infrastructure vulnerability encompasses problems in telecommunication networks such as backup and disaster recovery and uptime for a functioning network that transmits data between multiple machines or data centers. Often, information infrastructure vulnerability exists in such networks because the underlying combination of subsystems is fragile and functions with a low degree of fault tolerance.

Existing solutions to the problems of information infrastructure vulnerability that are implemented by large data hosting entities can fail for several reasons. Data hosting companies often do not utilize redundant network systems and tend to be focused on application performance and response time instead. For example, in a situation where two data centers operate at a great distance from each other, and load balancing between the two data centers is required, adding a second load balancer that operates in parallel with a first load balancer can decrease performance of the load balancing operation.

Significant costs involved in redesigning or retrofitting a network to be redundant often makes doing so an impractical solution to the problem of information infrastructure vulnerability. Retrofitting is not only costly but often very disruptive technically. A cross- functional view of redundancy, wherein redundancy is built into all aspects of a network, requires significant investments in facilities as well as equipment. SUMMARY OF THE INVENTION

In one aspect, the present invention provides a computer network connecting at least two data centers, comprising at least first and second data centers; a first public Internet service connecting the first and second data centers; a second public Internet service connecting the first and second data centers; and a virtual private network connection connecting the first and second data centers. The computer network further comprises a first wireless communication connection between the first data center and the first public Internet service and a second wireless communication connection between the second data center and the first public Internet service. The computer network further comprises a first sonet connecting the first data center to the first public Internet service; a second sonet connecting the first data center to the second public Internet service; a third sonet connecting the second data center to a third public Internet service; and a fourth sonet connecting the second data center to a fourth public Internet service.

Each data center comprises a plurality of routers that direct data across the data center to customer computers connected to the data center. The plurality of routers comprises a first edge router connecting the data center to a first public Internet connection; a second edge router connecting the data center to a second public Internet connection; a first router/switch connected to the first router; a second router/switch connected to the second first router; a private network edge router; and a terminal server router, wherein the first and second router/switches implement a core that is connected to and passes data between customer equipment maintaining data on the data centers.

In another aspect, the present invention provides a method of administering multiple data centers that provide colocation of customer data, the method comprising: providing a plurality of geographically distinct data centers on which customers maintain data, the data centers mirroring each other in terms of data; implementing monitoring functions on the data centers to detect communication failure; and routing data communication between the data centers in response to detected failure within the data centers. The method also performs security scans across the data centers to detect failure and intrusion detection. Performing security scans comprises performing a network scan of the data centers from an external source. The results from the network scan are used from an external source to probe hosts and ports to determine listening applications connected to the ports. Performing a security scan comprises an internal host-based scan running a series of command prompt-level, internal type attacks on the data centers. Other features and advantages of the present invention will be realized from the following detailed description and figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the basic topology of an embodiment of the present invention;

FIG. 2 is a block diagram illustrating the logical hardware architecture of the present invention;

FIG. 3 is a block diagram illustrating sonets between data centers and public data transports according to an embodiment of the present invention;

FIG. 4 is a block diagram illustrating a switching environment failover within each data center;

FIG. 5 illustrates the internal organization of the data centers;

FIG. 6 illustrates monitoring and management functions within and between data centers;

FIG. 7 illustrates a multi router traffic graphic index page used to manage traffic between data sites;

FIG. 8 illustrates a detailed traffic analysis page the provides additional traffic information for specific items;

FIG. 9 illustrates a tactical overview page for managing the data sites;

FIG. 10 illustrates a service details page that displays status of monitored services;

FIG. 11 illustrates an openNMS page used to manage network performance; and

FIG. 12 is a block diagram illustrating the major components of a work station that can be used to perform monitoring and management of the data centers in an embodiment of the present invention.

DETAILED DESCRIPTION Directing attention to FIG. 1 the basic topology of network 10 of an embodiment of the present invention includes data centers 12 and 14 connected to each other via point to point private network 16, public Internet connections 18, 20 and wireless connection to public Internet connection 22. Network 10 provides a redundant hosting network for customers having large data transmission requirements. Point to point private network 16 is dedicated to management of network 10. Customer data is transmitted across public Internet connections 18, 20, 22. While topology 10 illustrates two data centers connected to each other, those skilled in the art will readily appreciate that more than two data centers can be connected in the manner illustrated. The point of that sentence was to provide support for broad claims that will cover more than two data centers. Data centers 12, 14 include edge routers at each data connection 16, 18, 20, and 22. Point to point private network 16 is connected to edge routers 24-1 and 24-2 at data centers 12, 14, respectively. Public Internet 18 is connected to edge routers 26-1 and 26-2 at data centers 12, 14, respectively. Public Internet 20 is connected to edge routers 28-1 and 28-2 at data centers 12, 14, respectively. Wireless connection 22 is in communication with edge routers 30-1 and 30-2 at data centers 12, 14, respectively.

FIG. 2 is a block diagram illustrating in greater detail the implementation of network 10. Routers 26, 28 connect to public Internet services such as those offered by AT&T and WorldCom via OC3 poSONETS 32. In the preferred embodiment, routers 26, 28 are Cisco series 7606 routers. Routers 26, 28 are connected to each other via a communication line 34 such as gigabit capacity optical cable. Routers 26, 28 are the primary entry points to data centers 12, 14 for customer data transmitted across public Internet access provider connections 18, 20. Router 26 connects to router/switch combinations 36, 38 via communication lines 40, 42, while router 28 connects to router/switch combinations 36, 38 via communication lines 44, 46. Communication lines 40 through 46 can also be gigabit capacity optical cable. Router/switch combinations perform intrusion detection on all incoming data traffic, maintain routing tables and are managed to mirror each other to provide instant recovery in the event of failure of one of the router/switch combinations. Router/switch combinations 36, 38 connect to customer cages 48, 50, where customers securely maintain equipment and devices such as LANs that utilize network 10. In order for each router/switch 36, 38 to maintain the mirror image of the other, router/switch combinations 36, 38 are connected by two gigabit optical trunks 52, 54.

Router 26 is connected to router/switch combinations 36, 38 via communication lines 56, 58, respectively. Communication lines 56, 58 can be gigabit optical cable. Router 26-1 connects to router 26-2 via point to point private network 16, which may implemented with an OC3 PoSONET. Router 26 is responsible for routing management and monitoring data between data centers 12, 14. While FIG. 1 shows a separate router 30 supporting a wireless connection to public internet connection 22, router 26 can also support a wireless connection to public internet access providers 18, 20, by connecting to router 60, connected to wireless antenna 62 as shown in FIG. 2. Router 26 also connects to router 64, which functions as a terminal server. Router 64 is connected by serial cables to the console port of every device used to manage network 10 and allows secure telnet into each managing device, thus allowing administrative personnel to control the devices remotely. Communication line 66 connects router 26 to router 64, and in the preferred embodiment is out of band (OOB) Management 100BaseTX copper cable. OOB Management 100BaseTX copper cable is also used to connect router 64 to routers 26, 28 as a backup measure in the event that router/switches 36, 28 fail. Router 64 also connects by communication lines 68, 70, both also OOB Management 100BaseTX copper cable in the preferred embodiment, to virtual private network (VPN) system 72 and firewalls 74, respectively. VPN system 72 in the preferred embodiment utilizes a system model VPN3000 from Cisco Systems. VPN system 72 encapsulates and encrypts data for transmission to another data center, gives the network 10 its own subnet and allows customers to connect to network 10 via VPN and cleartext to the customers' networks. VPN system 72 is connected by communication lines 76, 76, both also implemented with OOB Management 100BaseTX copper cable, to service the router/switches 36, 38, respectively. Firewall 74 includes two firewalls, one for primary mode and another one for failover mode. Firewall 74 also is connected by communication line 80 to service the router/switch 36, and by communication lines 82, 84 to service the router/switch 38. In the preferred embodiment, lines 80 through 84 are implemented in OOB Management 100BaseTX copper cable.

Network 10 provides several elements of basic name and address services. High availability and resiliency of these services are vital to customers who utilize network 10. The services include publicly routable IP addresses, fast, efficient DNS management network with customer portal access, address blocks such as a class A /24 IP address block. Available IP addresses are shared between the two data centers 12, 14. Customer machines maintained in customer cages 48, 50 at data center 12 can be assigned addresses according to the cages in which they are located. In the preferred embodiment, the primary traffic path to customer addresses in customer cages 48 flows across communication line 32-1 to router 26, then across communication line 40 to router switch 36, and then to the destination customer machine in customer cages 48. Router 26 is configured for a failover mode to route data to customer machines in customer cages 48 though a secondary path across communication line 34 to router 28 and then across communication line 44 to router/switch 36. Similarly, the primary traffic path to customer addresses in customer cages 50 flows across communication line 32-2 to router 28, then across communication line 46 to router switch 38, and then to the destination customer machine in customer cages 50. Router 28 is configured for a failover mode to route data to customer machines in customer cages 50 though a secondary path across communication line 34 to router 26 and then across communication line 42 to router/switch 36. Primary and secondary paths for data sent to customer equipment maintained at data center 14 operate in similar manner to those described above for data center 12.

Customers can maintain their own networks within customer cages 48, 50. Each cage includes at least two network switches. Each switch is segmented onto a separate VLAN and has default pathways to a specific transit link. In the event of transit link failure, provider backbone failure, or inter exchange failure, network 10 switches traffic from both network switches onto a remaining network. In the preferred embodiment, customers maintain storage area networks or network attached storage configurations within cages 48, 50. Data, applications, and server configurations are replicated across multiple domains. Real-time replication of databases and content is achieved through hardware solutions, software solutions, database vendor solutions and/or operating system solutions. The present invention includes configuration management scripts to rebuild a server to rapidly recover from computer hardware failure or security breach.

FIG. 3 is a block diagram illustrating SONETs between data centers 12, 14 and public data transports according to an embodiment of the present invention. To implement redundant connections between data centers 12, 14, communication lines 32-1 through 32-4 are implemented as SONET local loops with multiple communication lines. Router 26 connects to public Internet access provider 20 via communication lines 102, 104. Both communication lines 102, 104 are active during normal operation. In the event that either of communication lines 102, 104 fail, the other communication line will carry the data that was to be transmitted over the failed line. In the preferred embodiment, communication line 102 is an OC3 IP line used to access OC3 Transit line 106 to an IP and virtual private network to send data between data centers 12, 14. In the preferred embodiment, communication line 104 implements an OC3 P2P line to access P2P OC3/Transport line 108 to send data between data centers 12, 14.

Router 28 connects to public Internet access provider 18 via communication lines 110, 112. Both communication lines 110, 112 are active during normal operation. In the event that either of communication lines 110, 112 fail, the other communication line will carry the data that was to be transmitted over the failed line. In the preferred embodiment, communication line 110 is an OC3 IP line used to access OC3/Transit line 114 to an IP and virtual private network to send data between data centers 12, 14. In the preferred embodiment, communication line 112 implements an OC3 P2P line to access P2P OC3/Transport line 116 to send data between data centers 12,14.

FIG. 4 illustrates a failover diagram for data centers 12, 14. In the event that communication lines 102 and 104 fail on both sides network 10 fails over to the other carrier (100 and 112) and vice versa. In other words, one whole circuit is a redundant failover for the other circuit. The present invention utilizes BGP on outside interfaces in data centers 12, 14 to maintain a configuration that provides the ability to announce an Autonomous System ID to other networks. This is the method by which the present invention propagates the network addresses for network 10 via any router on the edges to other networks. This method also allows the present invention to announce where network 10 and network routers are to other networks so that they may route to network 10, or allow network 10 to route over them. iBGP works in a similar manner on the inside interfaces affording network 10 route propagation and fail over between all devices.

Network 10 also runs OSPF on the interior side of routers. OSPF re/calculates routes quickly when a topology change occurs (read: failure, latency, etc). OSPF supports equal- cost multipath routing methods, so separate routes can be calculated for each device and IP type of service to and from other devices and services on the network 10. An area routing capability is provided for an additional level of routing protection by defining primary/priority routes in between certain devices in a network space. It also reduces routing protocol traffic. Also, all OSPF routing protocol exchanges are authenticated. The use of these three protocols enables each fail over point as illustrated in FIG. 4 the ability to rapidly re-route traffic between any area of the network 10.

Network 10 provides transport options built on several base-level offering components, including multiple diverse peering points, static failover routing via a private channel between locations, rapid circuit provisioning and turnup, and customer specified VLAN settings on core switches. Support for customers of network 10 includes continuous network monitoring, help desk availability, and a custom reporting portal, including near real-time access to performance and availability data from a browser.

FIG. 5 illustrates the functional organization of data centers 12, 14. Inside each data center exists cores 150, 152. These cores are implemented through router/switches 36, 38, and are the central components for customer routing and intrusion detection. Customer IP space 154 represents the blocks of IP addresses reserved for customers serviced by cores 150, 152. DMZ/public space 156 is a semisecure area for publicly facing and not privately sensitive systems, such as web servers. Corporate LAN 158 is a local area network that serves personnel who manage the business entity that administers network 10. Network Operations Center (NOC ops) 160 is a network of secure workstations that are configured to monitor, manage, service and control all operational aspects of the data centers 12, 14 and network 10. For example, workstations in the NOC ops network 160 are used to configure routers, switches, and hubs, as well as display various alerts to administrative personnel. Management channel 162 includes all communication lines implemented as Out of Band Management 100BaseTX copper cable, namely communication lines 66, 67, 68, 69, and 70, which are used for backup and management of data centers 12, 14 and network 10. Monitoring module 164 includes all monitoring functions, procedures and commands for monitoring all operations within data centers 12, 14 and generating alerts based on results of the various monitors.

Base service for customers using network 10 includes a comprehensive implementation methodology which provides a coverage model for investments in services such as fail-over and mirroring, documents a customer-specific implementation plan for creating desired resiliency by leveraging a customized network environment, and justifies the recommended actions through a specific return on investment analysis. Security services include assistance with hardware and OS hardening, firewall configuration and management, pre-launch network penetration-detection tests, a virtual private network configuration from both public and private points, and network intrusion detection sensors.

Security measures are handled at the NOC ops network 160. On a periodic basis, a security scan is performed over all devices connected within data centers 12, 14 to probe for known exploits. The security scan includes four different types of security tests. The first type test is that of a network scan from an external or nontrusted source in relation to the target network. A scanner program sends a designated protocol such as ICMP, UDP, TCP, IP, or a custom datagram to each IP address in the network address range, for example, 10.0.0.0/24 (10.0.0.0 - 10.0.0.255). If the IP address is up, the scanner then probes ports on the IP address ranging from 1 to 66657 with the designated protocol to determine if a socket on the IP address is listening. If a socket is listening on the designated port and IP, it replies back to the scanner indicating that something is there. The scanner notes the host, port, and protocol for the second test, a network penetration test. If the scanner does not receive a reply on that port, protocol or IP within a set time interval such as 90 seconds, it will note a null result in its findings and move on to the next port. The scanner is capable of scanning in sequential or random order by IP, port, and protocol. The functionality of the first type of security test can be implemented using nmap, a software product available at http://www.insecure.org/nmap/.

The second type of security test takes the results from the first scan and systematically probes each host and port by the defined protocol to determine what application is listening there. This can be performed using two methods. The first method is to read the connection message. For example, the smtp mail application qmail responds upon successful connection to port 25 via TCP with a message similar to

220.host.domain.tld 'Server Message' ESMTP (1)

The scan program then connects to its internal database and searches for a record that matches the connection message. Since the phrase "ESMTP" preceded by a "220" message is unique to qmail the scan program notes that qmail is listening on port 25 of that host. The other method is to compare the port number of the corresponding response to RFC and ITEF defined default ports for services and note that the port is the defined service. After the port and protocol have been defined, the scan program then searches through the internal database of known exploits for that program and attempts to execute those exploits on that port via the given protocol on the given host. The exploit database is populated with community driven or developed exploits, or the test administrator can create his or her own exploits based on personal knowledge and the program API. Exploit specific results are displayed to the application. Based on the exploit specific results the application notes positive or negative results in its output. A report can be generated by this application and delivered to the test administrator. One example of a scan program that can implement this second type of security test is nessus, available at http://www.nessus.org.

The third type of security test is an internal host-based scan, where a program that accesses a database of known command prompt level, non-network-based, internal type attacks is run on the system. This test catalogues every function of the system's operating system and applications and compares those applications, functions, and their versions against its database. Once comparison is completed, the program proceeds to test stored exploits against the host machine. The exploit database is populated with community driven or developed exploits, or the test administrator can create his or her own exploits based on personal knowledge and the program API. Exploit specific results are displayed to the application. Based on the exploit specific results the application notes positive or negative results in its output. A report can be generated by this application and delivered to the test administrator. One example of a scan program that can implement this third type of security test is Tiger Analytical Research Assistant (TARA), available at httpJ/www- arc.com tara/index.shtml.

The fourth type of security test is an internal host based fingerprinting scheme in which a program catalogues a predefined list of applications and files on a system and takes an MD5 checksum of the file, and writes the file fingerprints to a database. The database is then stored in a secured manner on a stand alone, non-networked machine where frequent comparisons are performed between the MD5 checksum fingerprints to determine if the binary or file has been altered. When a file is altered an email or pager alert is sent to an administrator notifying him or her of the change and/or potential breach of security. Applications to implement the fourth type of security test include veracity, available at http://veracity.com and tripwire, available at http://tripwire.org. MD5 information is available at http://linuxnewbie.org/nhf/intel/security/md5.html.

MONITORING

Directing attention to FIG. 6, Monitoring module 164 performs crucial functions for maintaining that data centers 12, 14 are both live, functioning data centers that mirror and monitor each other. While each monitoring module 164 monitors its local data center, it also monitors the other data center. Monitored items include uptime state, CPU utilization, how much traffic is passing across the various links of network 10, the state of all links, the last revision of the operating systems, etc.

Edge routers 24 through 30 are all monitored by edge monitor 170. Edge monitor 170 is a submodule of monitoring module 164 and uses simple network management protocol (SNMP) GET and IP pings to perform TCP-IP connectivity checks. All routers in the network 10 incorporate SNMP listener functions that hold traps. If a router reaches a critical or failure state the edge monitor will read the traps during the ping and send an alert to a network administrator indicating that the router has failed. Core monitor 172 is a submodule of monitor module 164 that monitors every link to customer IP space 154 and measures utilization port by port. The extent of monitoring function of core monitor 172 varies based on customer requirements. In the preferred embodiment, core monitor 172 as a mandatory function pings customers' edge routers maintained in customer cages 48, 50 to verify that the customer equipment is functioning properly, and probe any equipment that is within a customer's network socket and test to see if it is functioning properly. Customer monitor 174 listens to a customer network to detect a regular state. If customer monitor 174 detects a state change, it sends an alert to the customer and system administration. Typical items monitored by core monitor include CPU, memory and storage utilization, temp of CPUs, whatever applications are up and running, how much memory, threads, processes, child processes, etc. are used by running applications. In the case where a customer maintains a message-based system, customer monitor 174 can send message into system to traverse the customer's application. Customer monitor 174 can also perform audits of customer equipment, by injecting known exploits at router 64 against the customer's web server to stress test the customer network.

DMZ/public monitor 176, corporate LAN monitor 178, and secure NOC monitor 180 are submodules of monitoring module 164 that function in similar capacities as customer monitor 174 but monitor the DMZ/public space 156, corporate LAN 158, and NOC ops network 160, respectively.

Cross system monitoring, where data centers 12, 14 monitor each other for failure and other problems, is implemented through cross monitoring channels connected to the edge routers 24, 26, 28, 30. Through these cross monitoring channels, each monitoring module can communicate with the other data center's monitoring module. If the targeted monitoring module cannot be reached through one of the plurality of cross monitoring channels, it is assumed that the other data center is down. The local data center sends alerts to system administrators and changes to failover state to handle all data needs for customers of network 10. Routers 26 through 30 implement cross monitoring channels 182 across public internets, while router 24 implements multiple monitoring channels 184 across the point to point private network 16.

INTERFACE The present invention provides a variety of interfaces to control and manage data centers 12, 14. Directing attention to FIG. 7, the MRTG index page 200 lists brief graphical displays of recent data reflecting traffic usage on any network device that MRTG is configured to monitor and graph. MRTG polls SNMP programs on devices via UDP to collect pre-defined statistics and deliver them to a web-based graphical view that summarizes the statistics. The traffic analysis view page 210 (FIG. 8) displays more detailed traffic usage data on a given item. The tactical overview page 220 (FIG. 9) displays an overall view of all networks, hosts, services and notifications. The tactical overview page 220 monitors the network including all data centers and displays alerts that will break down into service details by network. The service details page 230 (FIG. 10) displays the status of any monitored service, and lists it by host, service, location, and also records and displays trends and histories for each host. The services and hosts are polled using Netsaint via TCP, IP, UDP, and ICMP protocols.

The openNMS view page 240 (FIG. 11) lists outages, network performance, and overall availability in a manner similar to the tactical overview page 220 and service details page 230. OpenNMS polls hosts via TCP, IP, UDP, and ICMP networking protocols and queries the host or service based on its application-specific protocol.

FIG. 12 illustrates in block diagram form the major components of a computer workstation that can be used to manage data centers 12, 14 The computer system 350 includes a processor 352 and memory 354. Processor 352 may contain a single microprocessor, or may contain a plurality of microprocessors for configuring the computer system as a multi-processor system. Memory 354 stores, in part, instructions and data for execution by processor 352. For example, the server 310 includes in memory 354 the application software for operating the website 100. The clients 314 also may include browser software in memory 354 for accessing website 100 maintained on the server 310 and using the personal evaluation system of the present invention. If the system of the present invention is wholly or partially implemented in software, including a computer program, memory 354 stores the executable code when in operation. Memory 54 may include banks of dynamic random access memory (DRAM) as well as high speed cache memory. The system 350 further includes a mass storage device 356, peripheral device(s) 358, portable storage medium drive(s) 360, input device(s) 362, a graphics subsystem 364 and a display 366. For simplicity, the components shown in FIG. 12 are depicted as being connected via a single bus 368. However, the components may be connected through one or more data transport means. For example, processor 352 and memory 354 may be connected via a local microprocessor bus, and the mass storage device 356, peripheral device(s) 358, portable storage medium drive(s) 360, and graphics subsystem 364 may be connected via one or more input/output (I/O) buses. Mass storage device 356, which is typically implemented with a magnetic disk drive or an optical disk drive, is a non-volatile storage device for storing data and instructions for use by processor 352. In another embodiment, mass storage device 356 stores software instructions for implementing monitoring module 162 for purposes of loading such computer program to memory 354.

The method of the present invention also may be stored in processor 352. Portable storage medium drive 360 operates in conjunction with a portable non-volatile storage medium, such as a floppy disk, or other computer readable medium, to input and output data and code to and from the computer system 350. In one embodiment, the method of the present invention is stored on such a portable medium, and is input to the computer system 350 via the portable storage medium drive 360. Peripheral device(s) 358 may include any type of computer support device, such as an input output (I/O) interface, to add additional functionality to the computer system 350. For example, peripheral device(s) 358 may include a network interface card for interfacing computer system 350 to a network, a modem, and the like. Input device(s) 362 provide a portion of a user interface. Input device(s) 362 may include an alphanumeric keypad for inputting alphanumeric and other key information, or a pointing device, such as a mouse, a trackball, stylus or cursor direction keys.

In order to display textual and graphical information, the computer system 350 includes graphics subsystem 364 and display 366. Display 366 may include a cathode ray tube (CRT) display, liquid crystal display (LCD), other suitable display devices, or means for displaying, that enables a user to interact with the computer program to configure the application objects and implement the workflows. Graphics subsystem 364 receives textual and graphical information and processes the information for output to display 366. Display 366 can be used to display an interface to interact with the computer program to configure the application objects and implement workflows and/or display other information that is part of a user interface. The display 366 provides a practical application of the method of automating a microelectronic manufacturing process since the method of the present invention may be directly and practically implemented through the use of the display 366. Additionally, the computer system 350 includes output devices 370. Examples of suitable output devices include speakers, printers, and the like. To connect the computer system 350 to network 312, communications device 372 controls the flow of data between the computer system 350 and computer network 312 via communication line 374.

The components illustrated in the computer system 350 are those typically found in general purpose workstations, and are intended to represent a broad category of such computer components that are well known in the art. Computer system 350 illustrates one platform that may be used for practically implementing embodiments of the present invention. Alternative embodiments of the use of the method of the present invention in conjunction with the computer system 350 further include using other display means for the monitor, such as CRT display, LCD display, projection displays, or the like. Likewise, any similar type of memory, other than memory 354, may be used. Other interface apparatus, in addition to the component interfaces, may also be used including alphanumeric keypads, other key information or any pointing devices such as a mouse, trackball, stylus, cursor or direction key.

While the preferred embodiment of the present invention has been illustrated and described in detail, it is to be understood that the figures and detailed description are merely illustrative and many modifications can be made without departing from the spirit of the invention.

Claims

What is claimed is:

1. A method of administering multiple data centers that provide colocation of customer data, the method comprising: providing a plurality of geographically distinct data centers on which customers maintain data, the data centers mirroring each other in terms of data; implementing monitoring functions on the data centers to detect communication failure; and routing data communication between the data centers in response to detected failure within the data centers.

2. The method of claim 1, further comprising the step of performing security scans across the data centers to detect failure and intrusion detection.

3. The method of claim 2, wherein the step of performing security scans comprises the step of performing a network scan of the data centers from an external source.

4. The method of claim 3, further comprising the step of using results from the network scan from an external source to probe hosts and ports to determine listening applications connected to the ports.

5. The method of claim 4, wherein the step of performing a security scan comprises an internal host-based scan runs a series of command prompt-level, internal type attacks on the data centers.