JP2000339098A - Storage domain management system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simplify the management of a storage system and also to effectively use both flexibility and capability of a storage area network(SAN) architecture by managing the storage resources in a storage network according to a storage domain. SOLUTION: A storage server 1200 in a network has the client interfaces 1210-1212 which are connected to the client servers 1201-1203 respectively. The storage interfaces 1213 and 1214 are connected to the storage devices 1205-1207 via the communication channels. These connected interfaces 1213 and 1214 are combined with some storages of the server 1200 and provide the physical storages for a storage domain which are managed in the server 1200. The server 1200 can induce a storage transaction by means of the local configuration data. Thus, the storage management is simplified for the client servers.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to the field of large-capacity storage systems, and more particularly, to the storage transaction management of intelligent storage area networks and the system configuration thereof.

[0002]

2. Description of the Related Art It is becoming common to store large amounts of data in so-called mass storage systems. Mass storage systems typically include a storage device coupled to a file server on a data network. Users in the network access the data by communicating with the file server. File servers are typically connected to specific storage devices through data channels, which typically use a point-to-point communication protocol designed for managing storage transactions.

With an increase in the amount of storage and the number of file servers in a communication network, storage area networks (S
AN) has been proposed. A storage area network connects a number of high-capacity storage systems in a communication network optimized for storage transactions, such as a fiber channel arbitrated loop (FC-AL) network.
Implemented as A SAN supports a number of point-to-point communication sessions between a user of the storage system and a particular storage system on the SAN.

[0004] File servers and other users of the storage system are configured to communicate with a particular recording medium. When the storage system is expanded or the medium is exchanged in the system, the file server and other users need to be reconfigured. Also, when so-called data movement operations require data to be moved from one device to another, access to the data must often be blocked during the migration process.
After the transfer is completed, the data cannot be used from the new system unless the user system is reconfigured.

In general, as storage systems and networks become more complex and larger, the problem of managing user configurations of data and managing the configuration of the storage system itself doubles.

[0006]

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and provides a system and a system that can simplify the management of a storage system while effectively utilizing the flexibility and capability of a SAN architecture. It aims to provide a method.

[0007]

SUMMARY OF THE INVENTION The present invention is a system, method, and server for storage domain management. Here, the storage domain management refers to a system having a centralized and secure management capability positioned at the top of the existing storage area network hardware infrastructure, and the present invention is a high-performance environment suitable for a heterogeneous environment. Provide advanced storage management with high performance and reliability. Storage domain management is at the heart of a robust storage area network fabric, integrating old and new equipment, freeing server and storage resources from network and storage management tasks. It also handles network-based applications as hosts and makes these applications available through all components of the storage area network. According to storage domain management, it is possible to construct and optimize a heterogeneous storage area network environment that cannot be achieved by conventional systems and techniques.

The present invention provides a system for managing storage resources in a storage network according to a storage domain. This system is provided with a plurality of communication interfaces that are connected to a client, a storage system, and a storage network through a communication medium. A processing unit is connected to the plurality of communication interfaces, and logic included in the processing unit is used as a storage domain corresponding to at least one client set among one or more clients in the storage network. One storage location set is composed of one or more storage systems. The system implements multi-protocol support over multiple communication interfaces, logic to route storage transactions within the storage domain in response to transaction identifiers in the protocol, management interfaces that make up the storage domain, and multiple communication interfaces Logic to convert storage transactions to be performed in a common format for routing within systems with multiple communication interfaces, and to convert from this common format to another, resources to capture data subjects of storage transactions, networks Change that consists of logic that manages the movement of data from one storage location to another Including various combinations element as possible.

In one embodiment, the system of the present invention is installed as an intermediate device in a storage area network between a client processor such as a file server and a storage system used as a storage resource in a client storage domain. Is done. Storage transactions are received by the intermediate device and managed according to the configuration of the storage domain as defined by the configuration logic of the intermediate device. Intermediate devices provide a management site within the storage area network, which can support flexible configurations, redundancy, failover, data movement, capture, and multiple protocols. Further, the intermediate device in one embodiment performs legacy system emulation, and the storage domain includes legacy storage devices for clients, thus eliminating the need for client reconfiguration.

[0010] The storage domain is managed by assigning a logical storage range to clients in the network and mapping storage resources in the network to the logical storage range of the client. Assignment of logical storage ranges to clients is accomplished by mapping logical storage ranges assigned to clients in intermediate systems or other systems that are logically independent or isolated from clients of storage resources in the network. I do. In this way, the storage domain of storage resources accessible through the storage domain manager is managed by using the storage domain manager as an intermediate device.

[0011] A storage server according to the present invention comprises a processing unit, a bus system connected to the processing unit,
It comprises a communication interface and an operating system connected to the processing unit. The bus system has a slot that can receive an interface to a data store on a server chassis or communication channel connected to the slot. The operating system provides the logic for controlling the transfer on the bus system and the logic for converting the storage transaction received from the client server on the communication interface into an internal format, and the logic for processing the internal format according to the configuration data. The configuration data maps storage transactions on the communication interface for a particular storage unit in a transaction protocol range to a virtual circuit corresponding to that range using an internal format. The virtual circuit then manages the routing of transactions to one or more physical data stores through one or more drivers in the interface. Also, because the server includes resources to emulate a physical storage device, the client server can use standard storage transactions to access virtual devices without changing the configuration of the client server for storage transactions. Protocols can be used.

According to another element of the present invention, there is provided a storage router, comprising a first communication interface, another communication interface, a processing unit and a bus system. The bus system is connected to the processing unit, the first communication interface, and another communication interface. The processing unit supports an operating system, which uses the architecture and emulation of the virtual device to direct storage transactions received on the first communication interface to another suitable communication interface depending on the configuration data.

[0013] In some embodiments, the communication interface is an interface to a fiber optic medium.
In addition, depending on the embodiment, the communication interface includes a driver that conforms to the fiber channel arbitrated loop, and the communication interface includes a driver that conforms to the standard “computer peripheral interface standard” version 3 (SCSI-3). There is also.

[0014] In some embodiments, the processing unit comprises a plurality of processing units. In some embodiments, the bus system comprises an interconnected computer bus, and in some embodiments, the computer bus conforms to a standard "peripheral component interconnect" (PCI) bus. In some embodiments, the communication interface is connected to a bus system.

[0015] In some embodiments, the storage server has a non-volatile memory, and in some embodiments, the non-volatile memory is an integrated circuit non-volatile memory, such as a flash memory.

In some embodiments, the storage server has a controller for the disk drives, and in some embodiments, the controller supports a standard "redundant array of independent disks" (RAID) protocol. In some embodiments, the disk drive is connected to the controller by fiber optic media, and in some embodiments, the disk drive has a dual interface for connecting to fiber optic media. In some embodiments, each disk drive is connected to at least two controllers.

In some embodiments, the operating system may include instructions for converting SCSI-3 instructions and data received over the communication interface into an internal format, or a logical unit number (LUN) corresponding to the SCSI-3 instructions. ), The SCSI-3 instructions and data may be associated with a virtual device that has a datastore in the storage server. Also, the initiator SCSI-3 identification number
Embodiments are also possible where (ID) and LUN are used to associate SCSI-3 instructions and data with a virtual device having a data store in the storage server.

In some embodiments, the operating system has logic to monitor the operation and status of the storage server, and in other embodiments, handles device failure and transfers control to redundant components. There is logic for it.

The present invention provides a storage server architecture that supports virtual devices and virtual circuits for recording and managing data. The storage server according to the present invention is equipped with a plurality of communication interfaces, a first set of the plurality of communication interfaces is for connecting to any kind of data user, and a second set of communication interfaces is a storage device. It is for connecting to each device in the group. The data processing resources of the storage server are connected to a plurality of communication interfaces and enable data transfer between the interfaces. The data processing resources consist of a plurality of driver modules and configurable logic that links the driver modules to the data path, which are implemented in pairs in a preferred system for redundancy. Each of the configured data paths plays a role of a virtual circuit having a driver module set selected from a plurality of driver modules. Data storage transactions received at the communication interface are mapped to one of the configured data paths.

According to another element of the invention, the plurality of driver modules have a protocol server for a protocol supported by one of the plurality of communication interfaces. The protocol server
Recognize a target identifier that identifies a particular storage range according to the protocol on that interface. Transactions addressed to a particular storage range are mapped to a particular configured data path in the server.

The data path configured as described above operates as a virtual storage device. Users of the data
It communicates with a communication interface on the storage server according to a protocol for a specific storage device. Within the server, transactions according to that protocol are mapped to virtual storage devices implemented by the driver set. The setup and modification of storage tasks performed on a particular data path and the setup and modification of storage range mapping from one data path to another are completed by configuring a driver module set in the storage server.

According to one aspect of the invention, the plurality of driver modules are one or more hardware drive modules that manage each communication interface and one that performs data path tasks independently of the plurality of communication interfaces. It has the above internal driver module.
Data path tasks include, for example, cache memory management,
Memory mirroring management, memory partition management,
There are data movement management, and other storage transaction management tasks. By providing this type of data path task in a virtual device architecture, the configuration of the storage system for managing such tasks is essentially transparent to the user.
Further, by providing a virtual device function to a storage server that is optimized to execute the above task, performance improvement and flexibility improvement are realized.

According to one aspect of the invention, the plurality of driver modules have logic for communicating data within a server environment according to an internal message format. The received storage transaction is converted to an internal message format and placed in the configured data path for the particular transaction. In a preferred embodiment, the protocol server performs protocol conversion and virtual circuit mapping.

Configurable logic includes a user interface for accepting configuration data, and memory for storing a table or list of each driver module set consisting of data paths. The configurable logic in one embodiment is implemented using a graphical user interface, for example, on a display having a touch screen that receives input signals. With a graphical user interface, flexible and easy to use configuration tools can be provided.

According to another element of the present invention, the configuration logic includes a memory for storing configuration data in the form of a table identifying a data path for the virtual circuit, wherein in one embodiment this memory comprises a storage device. This is achieved by using a persistent table storage process that maintains a table in a non-volatile memory that does not lose data even when the system is reset or the power is turned off. In addition, the configuration logic implements the data path for the virtual circuit using redundant driver modules on redundant hardware in the system, so that any failure point on the storage system will not prevent a particular storage transaction. Absent. In a preferred embodiment, the resources in the storage domain are defined using virtual circuits consisting of a plurality of driver modules and configurable logic connecting the driver modules to data paths, wherein the data paths provide redundancy to the preferred system. Is implemented as a pair. Each configured data path serves as a virtual circuit having a driver module set selected from a plurality of driver modules. Data storage transactions received at the communication interface are mapped to one of the configured data paths, and thus managed within the storage domain, which is managed and configured within the storage domain manager.

Basically, storage domain management allows a user to take full advantage of the storage area network's capabilities to address business problems. The storage domain management platform provides heterogeneous interoperability of various storage systems and protocols, reliable centralized management, scalability and superior performance,
All of the features such as reliability, availability, and maintainability can be provided on one intelligent, purpose-built platform.

[0027]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1A shows an intelligent storage area network for performing storage domain management.
1 illustrates a network having an (ISAN) server 1200. Storage area networks (SANs) can be used to provide data storage services for client computers. Storage area networks are optimized to provide high bandwidth, high throughput storage for client computers such as file servers, web servers, end user computers, and the like. The storage server 1200 according to the present invention in a preferred embodiment enables data storage, storage transaction cache service, storage routing and virtual device management on the chassis.

The storage server 1200 in the network
Are client interfaces 1210, 1211, 1212 connected to the client servers 1201, 1202, 1203, respectively.
Having. The storage interfaces 1213, 1214 are connected to the storage devices 1205, 1206, 12 through a communication channel.
07, and when these are combined with any storage on the storage server 1200, the storage server
Provides physical storage for storage domains managed within the 1200. Communication channel 1213 in this example
Are connected to devices 1205 and 1206 through a hub 1204. In operation, the client interface operates according to a protocol in which the client server requests a storage transaction with a command that includes a storage domain identifiable parameter such as one or more initiator identifiers, a logical range such as a LUN number, and an identifier of the target device. . Storage server 1200 maps the requested transaction to a virtual device and allocates physical storage for use by the virtual device for transactions between physical storage devices. Storage server
1200 also has resources to emulate the target physical device identified in the request.
The storage server 1200 can use the local configuration data to guide storage transactions, simplifying storage management for client servers.

To maximize throughput, the storage server 1200 uses a high-speed network medium such as Fiber Channel or Gigabit Ethernet.
Connected to client server 1201-1203. These client servers 1201-1203 are typically connected to end-user computers by network links.

FIG. 1A shows a management interface 108 connected to a server 1200 via a communication link 109.
The communication link that receives a signal from the station 108 and the interface in the server 1200 is, for example, an Ethernet network link, a serial cable connected to a serial port, or an Internet bus interface in various embodiments.

The server 1201-1203 and the storage device 120
Communication between 5-1207 is performed via the optical fiber channel arbitrated loop network via the storage server 1200 as an intermediate device. Channels on the FC-AL are standard computer and peripheral interface standards version 3 (SCSI-3), preferably using Fiber Channel media, also known as Fiber Channel Protocol (FCP) (e.g., SCSI B X3T10, FCP X3.269-199
This is realized using a protocol according to X). In another embodiment, a protocol, such as the Internet Protocol, may be used on a Fiber Channel fabric that performs storage transactions with various protocols. In some embodiments, storage server 1200 supports multiple protocols for data storage transactions.

FIG. 1 (b) illustrates various uses of an intelligent storage area network (ISAN) server. Storage area networks (SANs) can be used to provide data storage services for client computers and are optimized to provide high bandwidth, high throughput storage for client computers such as file servers or web servers. IS
In addition to recording and playing back data, the AN server also provides other functions such as storage routing and virtual device management.

FIG. 1B shows a server 100A-D and an ISAN server.
102A-F, thin servers 104A-C, and storage array 106. The servers 100A-D may be UNIX servers, Windows ^™ NT servers, NetWare ^™ servers, or other types of file servers.

The servers 100A-D are connected to client computers by network links. The ISAN server 102A is connected to the server 100A by a network link and provides data storage services to the server 100A by performing the requested storage transaction, so the server 100A treats the ISAN server 102A like a storage device. ISAN server
The 102A can hold more storage than a typical hard drive or hard drive array, and can be used as a storage router to provide intelligent routing between data stores connected to the ISAN server 102A. it can.

The ISAN server 102A is also capable of performing higher bandwidth, higher throughput storage transaction processing than typical hard disk drives and hard drive arrays, thus providing a large amount of multimedia data streams and other large data streams. It is possible to handle demand.

To obtain the maximum throughput, ISAN
The server 102A may be connected to the server 100A via a high-speed network medium such as an optical fiber channel. Server 100B
-D is connected to the client computer by a network link and to the storage area network by a fiber channel fabric. The storage area network includes the ISAN server 102B-D and the storage array 106, and the server 100B-D and the ISAN server 102-B
-D is an optical fiber channel arbitrated loop (FC-
AL) driver support.

Communication between the server 100B-D and the storage device on the FC-AL is preferably a standard small computer and peripheral interface standard version 3 (SCSI-3) using fiber optic channel media, also known as optical. Fiber Channel Protocol (FCP) (e.g., SCSI B X3T10, FCP X3.
269-199X). In another embodiment, a protocol, such as the Internet Protocol, may be used on the fiber channel fabric 108 that performs storage transactions with various protocols. In some embodiments, ISAN server 102A supports multiple protocols.

The thin servers 104A-C are connected to clients using network links, but do not use a storage area network to provide data storage.

The ISAN server 102E-F is directly connected to the client by a network link and there is no intermediate file server. The ISAN servers 102E-F can also provide application-specific processors that provide file server, web server, and other processing functions.

FIG. 2 shows another embodiment of the storage area network. In FIG. 2, a server 1250 having the storage director logic and the cache memory as described above is connected to clients on various platforms. These platforms are, for example, Hewl
ett-Packard server 1255, Sun server 1256, SGI server 125
7, each executing a different protocol for storage transaction management. A plurality of physical storage devices that make up the physical resources used as storage domains are also connected to the server 1250 and managed by the storage director according to the virtual device architecture described above. Multiple physical storage devices in this example include storage on Hewlett-Packard platform 1251, Sun platform 1252
On storage, storage on EMC platform 1253. In this manner, a server that includes storage director logic can create a shared storage pool that supports conventional servers and storage in a heterogeneous environment. Incompatibilities between multiple storage devices and servers can be masked or imitated as needed using virtual device architecture. In this way, the true storage area network environment can be used to manage all interoperability issues between hosts, fabrics, and storage at the storage server level.

The storage director logic using a virtual device architecture provides one intelligent coordination point for the configuration in which client servers access storage using a storage domain configuration. Adding new devices or changing the management of existing devices requires little or no hardware reconfiguration. The configuration of the storage server can provide accurate configuration information and control by automatically maintaining the mapping of datasets in physical storage to the server. Always accurate mapping of physical storage greatly simplifies the management of storage area networks. In addition, the storage director of the server can move data from the old storage device to the new storage device while the device remains online. Further, the size of the recorded object is no longer limited by the size of the largest object that can be created in one array. Multiple arrays can be linked to a single storage object, independent of the host operating system running on the client server. The Storage Director can also manage backup and test operations, such as taking snapshots of data in non-volatile cache memory, and manage data backups, for example, by copying data from disk to tape without routing through a client server. You can also. Furthermore, using a local cache,
It is possible to move data from an array with loss redundancy, repair redundant storage during repair and rebuilding of the array, and make the data fully available. For applications with multiple servers accessing a common data set, locking logic can be placed in the storage server to provide a simple solution that can be scaled using virtual device architecture.

The storage director logic in the storage server integrates cache demand from both the server and the storage, so that the number of cache memories required for the storage area network is small. This system can allocate more cache to either the client server or the storage system than each can effectively provide as internal memory. Caches are dynamically or statically allocated according to the definition for the application using the system.

FIG. 3 shows an example of a more congested storage area network using a plurality of interconnected storage servers according to the present invention. Storage server 1300,
1301 and 1302 are interconnected using communication channels 1350 and 1351 and mounted using a high-speed protocol such as, for example, optical fiber channel, gigabit Ethernet, and asynchronous transfer mode (ATM). In this embodiment, each storage server has storage director logic and a non-volatile cache. Storage server 1300, 13
01, 1302 are connected to a plurality of client servers 1310-1318 in this example, and client servers 1313 and 1313
14 is connected to a storage server 1301 through a hub 1320. Similarly, client servers 136-1318 are connected to hub 1321, and hub 1321 is connected to storage server 1302. The client servers 1310-1318 communicate with the storage server using a storage channel protocol such as FCP as described in detail above. According to these protocols, storage transactions are requested,
The initiator identifier, logical unit number (LUN), and target storage device identifier of the request are communicated, and the storage director logic uses these parameters to map the storage transaction to a virtual device in the storage domain. The server also includes resources to emulate the target storage device, allowing the client server to smoothly interoperate with multiple storage devices within its storage area network.

In FIG. 3, a plurality of storage devices
1330-1339 is connected to the storage server 1300-1302. Various symbols in the figure indicate storage devices, the network is heterogeneous, the server 1301,
It can be seen at 1302 that various devices managed by the virtual device interface are available. Also, the communication channel can be changed. Therefore, the hubs 1340, 1341, and 1342 are provided in the network, and various communication protocols can be used between the storage device and the storage server. [Intelligent Storage Area Network Server] FIG. 4 is a block diagram of a storage server in one preferred embodiment, including storage system management resources according to the present invention.

The storage server 102 has a connection option 130 having a communication interface set for users and other data processing functions and a storage option 128 having a communication interface set for storage devices. Operating system 124, block storage interface 118, management interface 12
0, having a protocol interface 122. Connection options 130 include a serial connection 140, a front panel connection 14 that supports configuration management routines in one embodiment.
2.Has an Ethernet connection 144 to support communication with a remote management station, a network interface 145, and a storage option 128
2. It has a solid state drive (SSD) 134, a SCSI interface 136, and a network interface 138. The SCSI interface 136 is connected to the DVD / CD-R 148, and the network interface 138 is connected to the storage server 102G and / or the storage 150.

The connection options 130 are various methods for connecting the server and the client to the storage server.
Serial connection 140 provides network management, remote management modem, uninterrupted power supply messages, front panel connection 142 provides management connection to storage server 102 front panel display, and Ethernet connection
144 each support a management protocol and possibly an Ethernet interface for data transfer.
Network interface 146 is one of many high speed interfaces that may be installed on a server. In some embodiments, network interface 146 is a fiber optic channel interface with a driver for fiber optic arbitrated loop (FC-AL). Network interface 146
It is possible to provide a driver for SCSI-3 on Fiber Channel media using the Fiber Channel Protocol (FCP).

The hardware interface 126 interfaces specific hardware components. For example, the network interface 146 includes a set of software modules for a specific network interface to support configuration, diagnostics, operational monitoring, health and status monitoring.

Operating system 124, table 1
16, interface 118-122, storage server 102
Support virtual device and storage routing function. These components of the storage server 102 use a configured set of driver modules in the system to route storage transactions between the appropriate storage options 128 and connection options 130.

The operating system 124 provides a message routing and transfer function in addition to the fail-safe function. The message routing and transfer function by the operating system 124 transfers messages such as storage transactions between components of the storage server 102. Used to route. These messages include messages in the internal format between components of the virtual circuit, as well as control messages in other formats.

Block storage interface 118
Provides a software module that supports the transfer of block data. Interface 118 also supports striped data storage, mirrored data storage, partition data storage, memory cache storage, and RAID storage. Various supported storage types can be concatenated to create various combinations, for example, memory cache and mirrored data storage.

The protocol interface 122 converts requests according to various protocols and provides software modules corresponding to the requests. One set of modules is installed at the layer of Ethernet connection, and includes hardware driver, data link driver, Internet protocol (IP) driver, transfer control protocol (TCP) driver, and user datagram protocol (UDD).
P) Driver and other drivers. Another set of modules provides drivers for FCP.

The management interface 120 provides a software module for managing the storage server 102. In addition to an interface for managing access to the table 116, scheduling, process coordination, system monitoring, management of informed consent,
Contains interfaces for rules-based system management, such as managing system processes and events. The informed consent management module is based on the premise that a management measure based on rules for configuring and maintaining the storage server 102 is taken.

[Operation of Storage Transaction] A storage transaction is received by any of the connection options 130. Storage transactions include read, write requests and status queries. Requests may be block-oriented.

A typical read storage transaction is composed of a read command and addressing information. A write storage transaction is similar to a read storage transaction, except that it contains information about the amount of data to which the request is sent, followed by the data to be written. More specifically, each device uses an identifier (I
D). The machine that issues the request is called the initiator, and the machine that responds to the request is called the target. In this example, server 100A is the initiator and has ID7, and storage server 102 is the target and has ID7.
With 6. The SCSI-3 protocol provides two or more addressing components, logical unit numbers (LUNs), and addresses.

The LUN specifies a sub-component of the target ID. For example, in a hybrid hard disk / tape drive enclosure, two devices may share one ID but have different LUNs. The third addressing component is an address where device data is read from and stored. Since the storage server 102A provides virtual LUNs on a per initiator basis, one storage server 102A can support, for example, 10,000 or more virtual LUNs.

The storage server 102A maps the SCSI-3 storage transaction request to a virtual circuit corresponding to one virtual LUN. A virtual circuit is a sequence of one or more virtual devices. A virtual device is made up of one or more devices, such as software modules or hardware components. For example, two network interface devices can be combined to form a virtual device, and similarly, two cache devices can be combined to form one virtual device. With this design, even if a component fails, the storage transaction processing function of the storage server 102 is not affected.

The virtual circuit is composed of virtual devices necessary to support a storage transaction.
Typically, the first component in the virtual circuit is a driver that converts storage transactions from the storage transaction communication channel format, which in this example is FCP, into an internal format. One such internal format is intelligent input and output
It can be similar to the (I ₂ O) Block Storage Architecture (BSA) message format. The internal format is storage medium and communication channel neutral in the preferred system.

The intermediate virtual device of the virtual circuit also provides other functions such as caching, mirroring, and RAID. Because the internal format is storage medium neutral, the intermediate virtual device is designed to operate in the internal format and can interoperate with other virtual devices in the circuit.

The last virtual device in a virtual circuit is usually
It is a format conversion and communication channel driver for managing storage. For example, drive array 13
2 are controlled by redundant hardware driver modules (HDMs) grouped to form virtual devices. HDM supplies BSA for SCSI conversion, and HDM handles the interface with the drives that make up the drive array 132. Similarly, if the virtual circuit is a link to a different storage on network interface 138,
A virtual device is provided that supports converting the BSA to a storage device communication channel protocol.

[0061] The storage server also includes resources within the operating system and at the interface to the client server that emulates a physical storage device. This emulation makes the virtual device appear to the client server accessing its storage as a physical device. Thus, the client server can be configured to communicate with a standard protocol such as FCP using SCSI commands for storage transactions. In embodiments utilizing SCSI commands, emulation involves responding to query commands in response to a SCSI protocol having a device identifier and device capability information predicted or adapted by the initiating server. Also, the read capability command and the mode page data command according to the SCSI protocol allow the client server using the storage to rely on standard configuration information about the physical storage device,
On the other hand, the storage server is spoofed by emulating a physical storage device at the interface with the client server and is handled by the emulation resources to map the actual storage transactions to the virtual device. With emulation resources, virtual devices can be identified by a combination of initiator, logical unit number (LUN), and target device identifier, without having to connect the storage transaction to the specific physical target device identified in the request.

FIG. 5 is a block diagram showing the functional components of a server such as that shown in FIG. 4, operating as a storage management system 151 used for storage domain management. The system 151 has a storage manager operating system 152. With the storage manager operating system 152, the functional components include storage domain routing resources 153, legacy device emulation resources 154, data migration resources 155, and redundancy, hot swap and failover resources 156. The storage manager operating system
The communication between these resources, the on-chassis cache 157, the management interface 158, and, in this embodiment, the on-chassis storage array 159 is coordinated.

The cache 157 comprises, in one embodiment of the present invention, a solid state non-volatile memory array for securely supporting storage transactions. In another embodiment, cache 157 has a redundant array to further increase fault tolerance.

The system 151 is provided with a plurality of communication interfaces 160-165, and in this example, the interface 160
The interface X is a protocol Y between the client and the storage management system 151.
The interface 162 indicates the protocol Z between the storage device and the storage management system 151, the interface 163 indicates the protocol A between the storage device and the storage management system, and the interface 164 indicates the protocol B between the storage device and the storage management system 151. The interface 165 is configured to execute a protocol C between the storage manager system 151 and another storage management system on the network, respectively.

In the illustrated example, protocols XZ and protocol AC are supported by storage management system 151, which may be a plurality of different protocols, a variation of one protocol, or a specific storage system in which the system is utilized. Any of the same protocols suitable for the area network may be used.

The storage transactions are performed from the respective communication media to the internal resources of the storage management system 151 via the interfaces 160-165. In the preferred system, storage transactions are translated into a common messaging format within the system for routing among various interfaces independent of the protocols performed by those interfaces. Storage domain routing resource 153 maps transactions within the storage domain using virtual circuits configured for specific client devices and storage devices. With legacy emulation resources 154 and data movement resources 155, when new devices are added to or removed from the network, the storage domain
Reconstructed in 1. For example, new storage devices can be added to the network, datasets in existing storage devices can be moved to new storage devices, and storage transactions from clients using that dataset can provide in-transit and target emulation By doing so, it looks as if it remains in the existing storage device even after the move is completed. Fault tolerance is maintained by the redundancy, hot swap, and failover resources 156, and the storage management system 151 can operate continuously in a high-throughput data storage network. [Overview of Hardware Architecture] FIG.
Is an intelligent storage area network
FIG. 2 is a block diagram illustrating an example of a hardware architecture suitable for a (storage) server. The hardware architecture takes advantage of redundancy and supports distributed software systems so that a single point of failure does not interfere with a particular storage transaction.

In FIG. 6, a storage server 102A is mounted. Storage servers are designed for high redundancy while using standard components and standards-based devices. For example, storage server 10
2A employs a standard peripheral component interconnect (a high-speed version of PCI and a standard Fiber Channel Arbitrated Loop (FC-AL) interface).
Other embodiments use various other protocols and interfaces.

The storage server 102A has four separate 6
It has a 4-bit 66 MHz PCI bus 200A-D. There are many possible configurations of the network interface in the storage device and the slot of the PCI bus. In one embodiment, the PCI bus is divided into two groups: SSD PCI bus 200A-B and interface PCI bus 200C-D, each group having two buses designated as upper and lower.
The upper and lower buses in each group can be configured to provide redundant services. For example, the lower SSD PC
The I bus 200B may have the same configuration as the upper SSD PCI bus 200A.

The PCI bus 200A-D is connected to a host bridge controller (HBC) module 202A-B. The HBC module 202A-B straddles the PCI bus 200A-D and forms a redundant bridge path.

The SSD PCI bus 200A-B supports solid state drive (SSD) modules 204A-G, and these SSD modules 204A-G become solid state storage devices such as flash memory stores.

The interface PCI bus is a network interface controller (NIC) module 206A-
B, redundant array of independent disks (RAID) controller (RAD)
Interconnects modules 212A-B and application specific processing (ASP) modules 208A-D to HBC modules 202A-B.

In addition to connecting the storage server 102A to the external FC-AL, NIC 206A-B is a fiber optic channel hub.
(FCH) modules 214A-D. The FCH modules 214A-D are each connected to both NIC modules 206A-B, provide 10 FC-AL ports, and are cascaded from the NIC modules 206A-B to provide a 20 station FC-AL hub.

The disk drive hub (DDH) module 216
AD provides a redundant FC-AL fabric and connects disk drives to RAC modules 212A-B. DDH module 21
The FC-AL fabric in each of 6A-D is made up of two redundant loops, which connect all drives connected to the DDH module to both RAC modules 212A-B. The RAC module manages the loop between all DH modules 216A-D, and each DDH module 216A-D supports five dual port disk drives, such as disk drive 218.

The system midplane (SMP) is not shown in FIG. SMP is a passive midplane, which is shown in FIG.
HBC module 202A-B, SSD module 20
4A-H, RAC module 212A-B, NIC module 206A-B, FC
The interconnection between the H module 214A-D, the DDH module 216A-D, and the ASP module 208A-D is realized. The SMP is compact PCI based and has four custom compact PCI buses 200A-
It has a control bus consisting of D, RAC-DDH interconnects, NIC-FCH interconnects and other midplane signals. further,
The SMP distributes power from the power subsystem (not shown in FIG. 6) to the modules at voltages of 48V, 12V, 5V and 3.3V.

The front panel display (FPD) 220
Provides a user interface for the storage server 102A. The FPD includes a display device and an input device. In one embodiment, a touch-sensitive liquid crystal display (LCD) may be used as a touch screen having an input function. The FPD 220 is connected to the HBC modules 202A-B and supports status display, configuration display and management, and other management functions.

Although not shown in FIG. 6, a redundant AC-DC power supply, a redundant DC-
Battery backup and redundant push-pull fan subsystems for DC power conversion, power down are provided.
These components support the features of high availability and low downtime that are important when utilizing storage area networks.

[0077] The storage server 102A can be connected to another storage server, such as one network port in a storage area network or a network installed in a storage device.
This connection is made to the F connected to each of the HBC modules 202A-B.
Performed on the C-AL extension port. In addition, HBC Module 2
02A-B uses RS232 serial port and 10/1 for out-of-band management
Provide 00 Ethernet port.

The bus system includes the storage server 102A
All buses within are included. In this example, the bus system has four PCI buses interconnected by a host bridge controller, and also has a PCI bus inside the HBC module that provides another interface. Slots include all locations that can receive an interface on the bus system. In this example, the four PCI buses outside the HBC module can each correspond to four interfaces.

An interface is a card or other device that goes into a slot and supports drivers and hardware for data stores connected to the interface. [Redundancy and Failover] The storage server 102A provides high redundancy.
In one embodiment, there is a redundant NIC, RAC, HBC module. SSD modules and drives support mirror rung. The drive also supports parity and dual channel access. Each DDH module includes a fully redundant FC-AL fabric for making connections to the RAC module. Failover is handled by the HBC module, which controls other modules in the storage server. This control is of a multi-layer type.

The first layer of control over the HBC module is power supply control, where each module has an individual power supply enable signal controlled by the CMB controller on the module. Although the HBC module is redundant, only one HBC module operates as a master HC module to guide and control the system and
HBC is the slave. When a module is plugged into a slot, its power supply is initially disabled,
Only the master HBC module can enable power supply. If the module begins to malfunction and does not respond to commands, the HBC module disables power to that module. The second control layer for the HBC module is the card management bus (CMB). Each module is CMB
The HBC module itself has an AVR microcontroller connected to the CMB acting as a master or a slave. The CMB microcontroller is powered by a connection to the midplane, independent of the power supplied to the main processor on the module.
The CMB allows the master HBC to read the card type, determine the presence or absence of the card, send a non-maskable interrupt to the card, or perform a hard reset of the card. The module processor and the master HBC module can also communicate through the serial port of the AVR microcontroller on the module. This communication path is a backup for controlling communication when the PCI fails.

The third control layer for the HBC module is the PCI bus. If the module does not respond using the control process on the PCI bus, you can ask questions through the CMB. If the module still does not respond, send a non-maskable interrupt using the CMB; if it still does not respond, reset through the CMB. If the module still does not respond after the reset, the power can be turned off and a warning to replace the module can be issued. [HBC
Module Redundancy] HBC Module Redundancy and Failover support system redundancy. While it is possible to operate both HBC modules 202A-B simultaneously, only one can be designated as the master by the HOST_SEL signal. The master HBC module supplies PCI bus arbitration to all of the PCI buses, controls power enable to other modules, and is recognized as a master on the CMB device. The PCI bus arbitration signal and power enable of the backup HBC module are disabled by the HOST_SEL signal. The CMB is switched by the HOST_SEL signal at each of the card's slave CMBs or FCB devices, and the HOST_SEL signal is pulled down on the system midplane (SMP) by a resistor, making the HBC module 202A the default master. HBC module 202B is HOST_
The SEL signal can be used as the master, but this usually occurs only at the time of failover or startup when the HBC module 202A is not present.

To eliminate the possibility of error occurrence, FVC is set to H
Drives the OST_SEL signal, requesting that a particular pattern be written to two separate memory locations. This prevents the malfunctioning HBC module itself from becoming the master. Both power enable signals of the HBC module are pulled down on the SMP so that both cards are powered up at boot time. HBC module 202A
Controls the power supply enable to the HBC module 202B, and similarly, the HBC module 202B
Controls power supply enable to 2A. Again, driving the power supply enable signal of the HBC module requires writing a specific pattern to two separate memory locations to eliminate the possibility of an error. PCI bridges do not support dual hosts. By specially configuring the PCI bridge, both HBC modules can be configured on the system PCI bus. The PCI bridges on both HBC modules are configured such that the address space controlled by one HBC module is mapped as a memory space local to the entire system PCI bus on the ridge by the PCI of the other HBC module. An error will occur if one HBC module attempts to read and write to another PCI address space.
Also, an error occurs when the four bridges to the system PCI bus recognize a transaction that causes a serious error. Therefore, one HBC module should not attempt to access another HBC module on the system bus.

Although the HBC module should not communicate on the PCI bus, the HBC module has two separate communication buses, a dedicated serial port bus. The dedicated serial port becomes the primary path for communication, transmits messages, and performs sanity checks on other HBC modules. If a serial port fails, you can use the CMB as a backup to determine which HBC module has failed. [HBC module start-up sequence] Both HBC modules are powered up by the EVC when the system power is turned on.
You need to determine if you have a module, but this is done through the CMB. If there are other modules, the HBC module 202A becomes the master by default. When the HBC module 202A determines that there is no HBC module 202B at the time of power-up, the power supply to the card slot of the HBC module 202B can be disabled. This adds a second HBC module,
Powered up under control of the master HBC module. When the HBC module 202A determines that the HBC module exists, communication through the serial port is performed. If the HBC module 202B determines that there is no HBC module 202A at power-up, the HBC module 202B sets the HOST_SEL signal and disables power supply to the card slot of the HBC module 202A, thereby causing the master to fail.
HBC module. When the HBC module 202B determines that the HBC module 202A is present, it must wait until HBC 0 performs communication through the serial port. If no communication is performed after a predetermined time has elapsed, the HBC module 202B starts a failover sequence. [HBC
Module Failover Sequence] The HBC modules should communicate with each other at specific intervals during the serial interface. When the backup HBC loses serial communication with the master HBC, it should attempt to establish communication with the master HBC module on its CMB. If communication is established on the CMB and both hosts are healthy, the serial communication link is abnormal. Both cards must perform diagnostics to determine where the fault is and trigger an alarm if the fault is on a backup HBC module or cannot be isolated. Fault is on master HBC module or CMB
If communication cannot be established, the backup HBC module powers down the master HBC module and becomes the master. Software Architecture Overview Storage servers are supported by operating systems designed to support unmatched high bandwidth, high throughput and storage server demands. The operating system schedules and controls data transfers on the bus system and manages the system. While many different operating systems and software component configurations are available, some embodiments use a highly modular operating system designed for storage servers.

FIG. 7 is a block diagram showing an operating system for a storage server and software modules of a supporting program.

FIG. 7 shows a hardware interface module 900, Accelerated Techn, Mobile, Alabama.
Nucleus Plus ^™ real-time kernel module 902, ISOS protocol management module 904, and storage service module 906 manufactured by ologies. The hardware interface module 900 allows software components of the storage server to communicate with hardware components of the storage server.

The Nucleus Plus ^™ real-time kernel module 902 is used to provide basic operating system functions such as tasks, queues, signals, timers, and critical section support. Exported to software module.

With the ISOS module 904, the storage server supports a messaging architecture for input and output. All hardware modules such as RAID controller (RAC) module, network interface controller (NIC) module, solid state drive (SSD) module, disk drive hub (DDH) module, fiber optic channel hub (FCH) module, etc. are input / output processors (IOP). The master host bridge processor (HBC) module is the host.

The storage service module 906 supports reliable message transfer between components using messaging classes, and supports operation of device driver modules and support for virtual devices. The device driver module (DDM) and the virtual device (VD) are constituent blocks of a storage server storage system. The storage service module 906
It is configured to support requests for storage transactions.

In some applications,
A single storage server, such as storage server 102A, has hundreds of DDMs that work with operating system modules 900-906 to service storage server requests. In another application,
Use various combinations of DDM.

The software component is implemented as a device driver module (DDM). DDMs that mainly send requests to hardware devices are called intermediate driver modules (HDMs), and DDMs that function as internal intermediate programs are intermediate service modules (ISMs).
Called. For example, DDMs that work on SSD modules are called HDMs, and DDMs that provide caching, mirroring, and other services not directly connected to hardware devices are called ISMs.

A single DDM may have multiple instances on a single storage server. For example, in FIG. 7, there are four examples of operation, health, and status PHS monitors 908A-D, which are NIC 910, RAC 920, and HBC 9 respectively.
Compatible with one of four major software subsystems: 30, SSD 940. Each DDM has its own message queue and individual identifier. For example, PHS on NIC 910
The monitor 908A becomes the device ID (DID) 0. Each DDM
Lists the classes of storage requests handled by the DDM, and the operating system module routes the requests to the DDM based on the class of the storage request. Requests can be routed by request code or virtual device number.

The NIC software subsystem 910 has three DDMs, the processor support HDM 912A, input / output conversion ISM 914A and the PHS monitor 908A, and the RAC software subsystem 920 has the processor support HDM 912B, input / output conversion ISM 914B and Three DDMs called PHS monitor 908B, HBC software subsystem 930 has processor support HDM 913C, input / output conversion ISM 914C, card management HDM 916, system monitor DDM 918, internet protocol DDM 921, front panel display DDM
922, application specific processor support DDM 924, PHS monitor 908C. SSD software subsystem 926
Has a solid state drive management HDM926 and a PHS monitor 908. Front panel display 950 is a hypertext markup language (HTML) client 928
Support.

FIGS. 8-10 show various hardware driver modules (HDMs), and FIGS. 11-14 show various internal intermediate service modules (ISMs) according to the preferred architecture of the present invention. FIG. 15 is a simplified diagram of a driver module set configured in a data path serving as a virtual circuit.

FIG. 8 shows a network interface card 520 having an HDM 524. Card 520 is the physical interface 52 to the Fiber Channel network.
With one. In this example, the network interface chip 522 is a Qlogic device such as ISP 2200A manufactured by Qlogic Corporation of Costa Mesa, California.
Is connected to physical interface 521 and line 523
Which is processed in the HDM 524. The HDM 504 conditions the communication for use by other driver modules in the system, and
The communication represented by has the SCSI format.
The communication indicated by line 526 has a message format, such as the BSA format, and the communication indicated by line 527 has an Internet Protocol (IP) format. HDM
Is an example of a driver class indicated as "Qlogic driver" in the figure. In this example, a device identifier DID 401 is given. The physical interface is identified as NIC # 1.

FIG. 9 shows a storage device 720 implemented in an array of non-volatile integrated circuit memory devices. H
The DM 722 is connected to the array 721 and converts the block storage architecture communication on line 723 to a format for recording and playback from the array 721. In this example, HDM 722
Is given a device identifier 1130 and the physical interface is identified as SSD # 4.

FIG. 10 shows a configuration of the disk drive array 820 installed in the storage server chassis of the optical fiber channel arbitrated loop architecture in the preferred embodiment shown in FIG. Optical fiber channel disk hub # 0 216A, channel disk hub # 1 216B, optical fiber channel disk hub # 2 216C, optical fiber channel disk hub # 3 216D shown in FIG.
Are connected to a redundant hub control HDM 821,822.

The HDMs 821, 822 are connected to physical fiber channel arbitrated loop connections 823, 824, respectively. The HDM 821 has a device identifier 1612, and the HDM 822 has a device identifier 1613. Connection 823 is connected to Fiber Channel interface 825, interface 825 is connected to physical interface 840 and HDM8
Network interface chip 82 connected to 27
With 6. ISM 828 is connected to HDM 827 and internal communication path 829. ISM 808 translates block storage architecture communication on line 829 into IOCB communication for HDM 827. HDM 827 network interface chip 826
The chip 826 drives the fiber optic channel 823. The device identifier 1210 is assigned to the ISM 828, and the device identifier 1110 is assigned to the HDM 827. Physical interface
825 is labeled as RAC # 0.

The fiber optic channel connection 824 is connected to the interface 830, which has a similar configuration to the interface 825, and has a physical fiber optic channel interface 831 driven by a network interface chip 832. Network interface 832 communicates with HDM 834 on the channel indicated by line 833. HDM 834 communicates with ISM 835 via channel 816, which manages the interface to BSA format messages on channel 837. In this example, ISM 835 has device identifier 121
1. The HDM 834 is provided with a device identifier 1111 and the interface 830 is identified as RAC # 1.

FIGS. 11-14 introduce some examples of ISMs according to the present invention that can be configured in a data path.

FIG. 11 shows a SCSI target server 550 which is an example of the protocol server module according to the present invention. Similar protocol server modules can be used for specific storage channels or network protocols utilized by users of data managed through the storage server of the present invention. Target server
Reference numeral 550 includes a message interface 551 for receiving a message coming from the HDM, such as the HDM in FIG. 8, connected to a communication interface for connection with a user.
In this example, the message on interface 551 has the SCSI format, and in another example, the message may already have the BSA architecture or another architecture suitable for the protocol on the communication interface currently in use. Server 550 is a SCSI-BSA
It has a switch function 550 for translating messages entering the translator 553 or answer local function 554.
Typically, translator 553 sends the message as an outgoing message on line 555. The incoming message on line 555 is supplied to translator 556, which uses the incoming BSA message on line 551.
Convert to SCSI format.

In many instances, a SCSI target device can respond to a SCSI message using the local answer service 554 without further routing the message. Many status messages not related to reading or writing from the storage itself are handled by the local answer service 554.

The target server 550 in this example is:
This is an example of a class SCSI target server, to which a device identifier 500 is assigned. One of the functions of a protocol server, such as SCSI target server 550, is to identify a storage range that is the subject of a storage transaction on the associated interface. Storage ranges are mapped to virtual circuits using configuration logic in the storage server, which is described in detail below.

FIG. 12 shows an ISM 650 that performs a mirror management data path task. ISM 650 has an interface 652 that connects to an internal communication channel on the device. Logical processor 652 receives incoming communications and data and manages the mirroring function. Logic 65
2 is primary drive 653, secondary drive 654,
It communicates with a plurality of drive interfaces including a third drive 655 and a spare drive 656. Although three-way mirroring is shown in the figure, a virtual circuit can be used to create a desired number of mirror paths to perform “N-direction” mirroring. Although the term "drive interface" is used, other storage devices can be used for the mirroring function. Drive interface
The 653-656 uses an internal communication channel to communicate with the HDM module associated with the target storage device used in the mirroring function, or with other ISM modules suitable for the particular virtual circuit. In this example, the mirror ISM 650 is implemented as an example of a class called "mirror" and is given a device identifier 10200.

FIG. 13 shows the partition ISM 750.
Partition ISM 750 includes an interface 751 that receives internal communication from other driver modules and an interface 752 that also communicates with other driver modules.
Logic process 753, base address 7
It has a data structure for storing 54 and limit address 755, and a drive interface 756. The partition logic process 753 uses a logical partitioning function that is useful for various storage management techniques, and configures the subject storage device identified by the drive process 756, so that the physical device looks like multiple logical devices in a virtual circuit. In this example, the partition ISM 750 is an example of a class “partition”, and has a device identifier 10400.

FIG. 14 shows the cache ISM 850. Cache ISM 850 has a logical processor 853 that communicates with an interface 851 to an internal message transfer structure on the storage server. The data structures in the cache ISM 850 include local cache memory allocation 85
4. It has a cache table 855 for identifying data stored in the cache 854, and a drive interface 856. The drive interface communicates on channel 857 with the HDM associated with the particular virtual circuit being used by the cache. In one embodiment, the cache memory 854 is managed in a storage server, and in another embodiment, the cache is stored in a fast non-volatile memory, such as a solid state memory module having an architecture as described with respect to FIG. Can be. In a preferred embodiment, the cache ISM 850 is implemented as an example of a class called "cache" and is given a device identifier of 10300.

FIG. 15 is a heuristic diagram of a redundant virtual circuit implemented by a data path having a plurality of driver modules according to the present invention. The virtual circuit has an external interface for communicating with the user of the data, a protocol translator for converting the communication with the user into the communication format of the driver module, and a storage object including a communication interface with the storage device. A storage operator performing the data path task can be located between the translator and the storage object. The array of driver modules acting as storage operators, such as caches, mirrors, partitions, etc., is optimized by the system designer using configured logic provided by the storage server.

In the example shown in FIG. 15, the external interface is provided by NIC # 0 and its associated HDM
Is indicated by block 1010. Protocol translator is SC
SI target server ISM 1011, cache function is ISM 10
12. The mirror function is ISM 1013. The storage object is accessed from the mirror function 1013, and in this example, a physical storage interface set selected from the basic Fiber Channel daisy chain interface and its associated HDM or external LUN interface, shown in block 1014, block 1015 and redundant Disk drives in the Fiber Channel Arbitrated Loop accessed through the ISN / HDM pair indicated by block 1016, solid state storage devices and associated HDMs indicated by block 1017, interfaces with external disk drives indicated by block 1018 and so on. It consists of an ISM / HDM pair related to. disk
Separate HDM modules on (01), (02), (03), (04) manage communication with interfaces 1015 and 1016 through fiber optic channel arbitrated loops.

In this embodiment, the mirror module
1013 accesses the disks (01), (02), and (03) as primary, secondary, and spare drives, respectively, and performs a mirror function. Although the mirror module shown in FIG. 12 includes a third drive interface, FIG.
No. 5 system does not use this third drive.

In this figure, ISM modules 1020 and 1021 are also drawn, which are not connected to the data path of the virtual circuit in the figure. These blocks show that using virtual circuit structures, new modules such as partitioning can be added by simply configuring the storage server.

The redundant data path is the interface NIC # 1 shown in block 1025 and its associated HDM, block 102
It comprises a SCSI target server ISM indicated by 6, a cache ISM indicated by block 1027, and a mirror ISM indicated by block 1028. Data storage device redundancy is:
This is achieved using the mirror function. Redundant driver modules are, in a preferred embodiment, distributed over individual IOPs in the storage server.

As shown in FIG. 15, each driver module has an individual driver identifier in parentheses in the block in FIG. 15, and is managed by the storage server and controlled by configurable logic in the storage server. This device identifier is used to support configuration logic based on tables in the configuration database.

In a preferred system, the configuration table is managed by a persistent table driver as depicted in FIGS. Returning to FIG. 4, storage server 102 stores management and routing information in a table such as table 116. Table 116
Can be accessed from the management interface 120. Table 116 is typically stored in persistent memory, such as non-volatile memory, and is maintained redundantly to support failsafe.

FIG. 16 shows a persistent table module 14 implemented as an example of a class called “persistent table” according to the basic architecture of the driver module configuration.
00 is shown. The persistent table module 1400 includes a table access logical processor 1401, various support functions including a table data access manager 1402, a persistent image manager 1403, and a persistent table instance synchronization module 1404. In this embodiment, the table data access manager 1402 is connected to a table class manager 1405, and the table class manager
06, LUN export table 1407, configuration template table 1408, DDM role call table 1409, virtual device table 1410, storage role call table
It manages a plurality of configuration tables including an 1411, an optical fiber channel disk roll call table 1412, an external LUN table 1413, and a solid state storage table 1414. The specific configuration of the set of tables managed by the persistent table module 1400 can be changed to suit a particular implementation,
It can be optimal for a class of devices.

The persistent image manager 1403 and the table instance synchronization manager 1404 communicate with a persistent data storage driver 1420 as shown in FIG. 11 and a second persistent storage driver not shown.
The persistent data storage driver 1420 is implemented as an HDM, which is an example of a class called “persistent storage”, and is provided with a device identifier according to the driver module model described above. In a preferred system, the persistent data storage HDM 1420 communicates with a solid state storage device in a storage server and provides fast access to data used in virtual circuits.

The persistent data storage holds various configuration information about the system. The DDM role call table 1409 includes a list of all device driver module examples and their unique device IDs. The storage roll call table 1411 contains a list of all active storage devices detected by the storage server, and is used by the virtual device table 1410 and configuration tools to create virtual circuits. The LUN export table 1407 allows the specified storage range in the storage channel transaction to be mapped to a virtual circuit. The external LUN table 1413 identifies logical units of storage held in other storage servers connected through an external storage interface on the storage server.

The two primary tables are the storage export to the client and the storage server 10
Support 2A storage routing function. These tables are an export table 1407 and a virtual device configuration table 1410. [Export table 1407] Export table 14
07 maps the addressing usage received with the storage transaction to a virtual circuit or storage option. The addressing information used for SCSI-3 on fiber optic channel interfaces is:
Initiator ID, target LUN, and target address.

Since all initiators or clients share many LUNs, it is not necessary to use all of this information to resolve each request, and most LUNs will select different virtual circuits. Rather, the target address, eg, an offset on the storage device, is used to address within the virtual circuit. In this exemplary embodiment, the export table 1407 is configured as shown in Table 1.

[Table 1]

The export table 1407 can include another column that describes the current state of the virtual circuit, the capacity of the virtual circuit, and other information. In one embodiment, export table 1407 lists the entire virtual circuit in a column of the export table.

Table 1 shows that requests can be routed to appropriate virtual circuits using addressing information for each protocol. Thus, only those TCP sessions that are used to identify a storage range targeting port 2000 will be routed to the virtual circuit starting with the virtual device having identifier 70.

Table 1 shows one protocol for one protocol.
This indicates that the LUN can be connected to different devices depending on the initiator of the storage transaction.
In this example, LUN 1 is mapped to a different virtual circuit based on the initiator ID. Also, the virtual circuit can be mapped based on other types of identifiers, such as “World Wide Name (WWN)”.

An example of the export table is as follows: #define EXPORT_TABLE "Export_Table" struct ExportTable Entry {rowID ridThisRow; // rowID of this row in the table U32 version; // record export table version U32 size; // export table Record size (bytes) CTProtocalType ProtocolType: // FCP, IP, etc.U32 CircuitNumber; // LUN, etc.VDN vdNext; // First virtual device number in path VDN vdLegacyBsa; // Legacy BSA virtual device number VDN vdLegacyScsi; / / Legacy SCSI virtual device number U32 ExportedLUN; // Exported LUN number U32 InitiatorId; // Host ID U32 TargetId; // Our ID U32 FCInstance; // FC loop number String32 SerialNumber; // String array of serial numbers Use Long long Capacity; // Capacity of this virtual circuit U32 FailState; U32 PrimaryFCTargetOwner; U32 SecondaryFCTargetOwner; CTReadyState ReadyState; // Current state CTRe adyState DesiredReadyState; // desired preparation state String16 WWNName; // world wide name (64 or 128 bits registered in IEEE) string32 NAME; // virtual circuit name} #endif [virtual device configuration table] virtual device configuration The table connects the virtual device to a device driver that supports the virtual device. Since virtual devices are designed to support redundant designs, the table for virtual device configuration maps virtual device numbers to device modules. In one embodiment, the table
A virtual device is mapped to a device driver that supports it using a table such as 2. FIG.
3 shows a virtual circuit starting from the virtual device 12 implemented in Table 2.

[Table 2]

As shown in Table 2, for each virtual device,
Information is provided regarding the primary and alternate driver modules that support the virtual device. For example, in the second row of Table 2, the optical fiber channel disk drive is mapped to the virtual device (VD) 10.

A virtual device is composed of one or more software or hardware modules that support the virtual device. The parameter column is used to provide initialization information. For VD 10, the parameter is SO (00), which means storage option 0. Each device driver module class has parameters for each class. The storage option driver uses a parameter that specifies a specific storage unit, and the intermediate driver class such as a mirror driver or a cache driver uses a parameter that specifies the next virtual device in the virtual circuit. According to this format, one device driver module can support a plurality of devices based on parameter settings. In Table 2, the device driver 1210 is used by the virtual devices 10, 15, 16, and 17, but specifies different parameters for each driver.
The status column indicates the status of the software or hardware module supporting the virtual device. For example, in the first row of Table 2, the status is "primary", which means that the primary device driver, here 4000, is used. The status in the second row of Table 2 is "Alternate", which indicates that the primary device driver has failed or is not responding properly. In this case, 1211 is used in the alternative driver, that is, in the second row of Table 2. For devices with multiple alternatives, the status column shows the driver used.

Example) For example, consider a case where a storage transaction is performed to the storage server 102A by specifying a LUN 2 in addressing information using the SCSI protocol on any one of the connection options 130. . In this example, it is assumed that the storage server 102A is configured as shown in Tables 1 and 2.

Connection options such as the network interface 146 for receiving storage transactions are connected to a hardware device driver. The hardware device driver receives the storage transaction and, according to the protocol, sends it to the appropriate virtual device handling the protocol.

For example, a SCSI storage transaction is sent to a device driver in the SCSI target class. Similarly, IP storage transactions are
Sent to the device driver in the target class.
Here, since the storage transaction was created using the SCSI communication protocol, it is routed to the SCSI target device driver (DID 500).

[0127] The SCSI target device driver further analyzes the request. The analysis first determines to which virtual circuit the request is mapped. This determination is made using information in the export table.
In this example, Table 1 shows that requests using the SCSI protocol specifying LUN2 should be routed to a virtual circuit starting at virtual device 12. In one embodiment, all SCSI target requests are routed to the same SCSI target driver for one interface, and in this embodiment, the target VD 12 parameter information is used to route messages to the second virtual device of the SCSI target. Also used to control the behavior of SCSI target devices.

Here, the SCSI with the driver number 500 is set.
The target device converts the SCSI message to an internal format. One example of such a format is I ₂ O
It is based on the Block Storage Architecture (BSA) format. This format is device and protocol neutral and can be used by intermediate device drivers. When the request is in internal format, it is sent to the next virtual device in the virtual circuit indicated by the parameter. Here, this parameter is VD (13), that is, the virtual device 13.

The message is routed to VD13,
This is a redundant caching driver, here 1030
Numbered 0 and 10301. The caching driver uses memory to store storage transactions. Based on the caching algorithm used by the driver, the driver routes storage transactions at appropriate intervals to the next virtual device in the virtual circuit. Where the next device is the parameter VD
(14), that is, indicated by the virtual device 14.

In the internal format, the message is
Routed to VD 14. The virtual device 14 has a redundant mirroring driver. In this case, the driver 1020
0 and 10201 are used. The mirroring driver is
A mirroring algorithm for holding a storage image mirrored by a plurality of volumes is implemented. This mirroring driver supports primary, secondary and tertiary stores and spare stores, but other mirroring drivers may support different algorithms. The mirroring driver also supports connecting new stores that are reliably synchronized with existing stores. Based on the mirroring algorithm used by the driver and the status of the mirrored store, the driver routes storage transactions to the appropriate virtual device in the virtual circuit. If both the primary and alternate stores are functioning, the mirror driver will
Only route this request to the primary and secondary stores by VD (15, 16, none, 17) or virtual devices 15, 16. “None” in the parameter list indicates that the third drive is not currently used for this virtual device.

The mirroring driver can route storage transaction messages to two devices either sequentially or in parallel. In this example, message transmission to the virtual device 15 is considered, but this example can be extended to the secondary store, virtual device 16. The virtual device 15 has a redundant driver for controlling the optical fiber channel drive. The driver converts the internal format into a format used by the driver, for example, from BSA to SCSI. The driver also provides the drive with addressing information. Here, the storage option, in this example, the optical fiber channel drive number 2 is selected using the parameter SO (02).

As described above, in the storage platform, all hardware functions (such as a disk or a flash memory) and software functions (such as a RAID stripe or a mirror) are almost always accessed through a software driver called a device.

[0133] These devices are paired (each device of the pair preferably operates on a separate substrate and preferably has redundancy) and is referred to as a virtual device. Next, the virtual devices are connected, and various configurations are completed. For example, a mirror device can be connected to two or three disk devices. Through such a configuration,
The virtual device chain is completed. These virtual device chains can be added as long as they are configured into BSA type devices that can be used in other configurations.

The virtual device chain is connected to the FCP / SCSI target server device and is mapped in the LUN export table related to “export” of the FCP target driver (that is, accessed from outside through the FCP protocol). At this point, the virtual device chain with the SCSI target server device at the beginning is called a virtual circuit.

The virtual circuit manager software that controls the configuration of the virtual circuit exports the virtual circuit by placing the “head” of the SCSI target server in a virtual device chain, and thereafter updating the export table of the FCP target. The software also supports delete, pause, and failover operations.

The virtual circuit manager software also has a role of holding a virtual circuit table VCT that describes all virtual devices in each virtual circuit in one place. This information is needed to perform many system operations, such as failover, hot swap, and shutdown.

When performing initialization, the virtual circuit manager software defines the VCT itself in the persistent table store and listens for VCT insertion, deletion and other changes.

To create a new virtual circuit, the information needed to map and export a new LUN must be included in the record in the VCT, instantiating a SCSI target server. The virtual circuit manager hears the insertion into the VCT and upon receiving the answer, performs the following actions: 1. Attempts to validate the information in the newly inserted record.
If a record contains invalid information, an error is displayed in its status field and no further action is taken. 2. of the virtual circuit specified by the newly inserted record
Create a new SCSI target server device for the LUN. 3. Set the status of the new record to "exemplary". 4. Storage assigned to the virtual circuit is flagged for use in the storage roll call table. 5. The export table is updated and the LUN is updated to the new SCSI
Sent to the target server. When a record in the virtual circuit is deleted, the virtual circuit manager performs the following actions: 1. Pauses the virtual circuit, if not already done, and displays Pause. 2. Remove the virtual circuit dispatch data from the export table. 3. Not used in the roll call record referenced from the virtual circuit record. 4. Excludes examples of SCSI target servers related to virtual circuits. The virtual circuit manager listens for changes in the "export" field in the VCT, and if the "export" field in any record in the VCT is set to "correct", the virtual circuit manager performs the following actions: 1 Export the virtual circuit by making the necessary changes to the export table of the .FCP target. 2. If any errors are encountered during the export operation, the status field of the VC record will be set and the "export" field will remain correct. "Exported" if virtual circuit is not exported
Is set to "wrong".

The virtual circuit manager listens for changes to the “pause” field of the virtual circuit table. If the "Pause" field in any of the VCT records is set to "Correct", the virtual circuit manager will do the following: 1.If a VC is currently exported, the export will stop and the "Export" Was set to "wrong". 2. A pause message is sent to all virtual devices in the virtual circuit. 3. If any error is encountered during the pause operation, the status field of the VC record will be set and the "Pause" field will remain in the correct state. That is, if the virtual circuit is not paused, the flag “paused” is set to “wrong”.

[User Interface] The user interface can be created by a data processing structure for displaying and using when configuring the storage server according to the present invention. The image includes a field for displaying a logo, a field for displaying basic information about the server chassis, and a window having an icon set. When these icons are selected, a management application can be started. Routines for managing hardware and software, managing user access, and monitoring long processes in the server are initiated by buttons. According to the present invention, a function of defining a host attached to a server, a function of mapping an exported LUN to a managed resource, and a function of configuring a managed storage can also be activated by a button operation.

This window also includes a user logon dialog box including a user name input field and a password input field.

[Hot Manager] The user activates the host manager by operating a button. Here, a Java-based user interface (UI) for determining a host (server) for a storage server will be described. When the management software opens a window, it displays a table where you enter the host name, port number, initiator ID and description in several columns for each host available for configuration and use. The other fields include the network interface card identifier and the individual host identifier described in another column. The individual host identifier in the preferred example is a worldwide number for a fiber channel host.

The host manager is Ja of the storage server.
is a sub-component of the va-based management application that allows users to access the NIC port and initiator I
You can assign a name and description to D and proceed with the LUN definition process. Access common functions through mouse popups, toolbar buttons, and action menus, such as using the Add New Host, Change Host, and Delete Host buttons to access existing hosts or define new hosts. Can be.

The user interface includes a menu and a table for displaying host information, or other graphics. When the user enters the host manager panel, the table is filled with all existing hosts. The user can select a row in the table. Each row contains information about one host. Next, the user selects to change or delete the host, and when the change is selected, a dialog box appears, allowing the user to change the host name and content. After changing, press the OK or Cancel button. Press OK and the changes are displayed in the table and sent to the server. If you select Delete, a dialog box will appear with a label indicating the host to be deleted.
OK and Cancel buttons are displayed. Press OK,
The row for that host is deleted from the table, and the server does this too. Selecting Add will bring up a dialog box where the user can add all information about the host. When you press OK, a new row for the new host is added to the table, and the server performs this addition. Click a column label to sort the columns.

[Storage Mapping] The user can activate a storage management routine, and the image displayed by this routine includes a window showing a display configuration based on a hierarchical tree for displaying storage elements.

The storage element is defined using this tree structure (for example, mirror → stripe → disk). This allows the user to configure the storage in a systematic way that matches the user's thinking about the storage. Typical types of storage elements are:-mirrors-stripes-external LUNs-internal disks-SSDs-storage collections-storage partitions By assembling these elements in a tree (e.g. a tree display like Microsoft Explorer) Use), the user can pre-configure the storage for use in the virtual circuit. Each element can be divided into partitions, and these partitions can be used in different ways. For example, you can split a stripeset into partitions, export one partition as one LUN, and use another partition as a member of a mirror (which can be further subdivided).

When a storage element is divided into partitions, it is stored in these storage collections, and this storage collection becomes a child of the partitioned elements. For unsplit elements, this partition collection does not exist. Each partition is identified by the type of storage it divides, that is, a mirror partition or a disk partition. Partitions of a storage element cannot be combined into one partition unless all partitions of that element are available (ie, all storage elements are unused). To this end, the user selects one of the partitioned storage elements having only unused partitions, and presses the "unpartition" button.

If there are dedicated spares, they are also stored in the storage collection, and this storage collection is a child of the element to which these spares are assigned.

Thus, each storage element can have the actual storage elements that make up the partition collection, spare collection, and parent element as children.

In a sense, the storage manager can be said to represent the contents of a storage role call table that lists all connected storages on the server. Each available storage element is seen at the top of the storage tree; for example, a mirror is shown as available, but the stripes and disks that make up the mirror branch are unavailable because they belong to that mirror. To reuse them elsewhere, you need to remove them from the mirror (and hence from the storage tree below the mirror). In one embodiment, this is similar to moving files from one directory to another in a Windows NT file explorer program, as well as dragging and
Perform by drop. The tree of all storage (used and unused) is displayed in the left half of the display in this example, and each element of the storage has an icon indicating its type, and specifies a name or ID.

Available (unused) storage is listed below the tree, on the right side of the window or in other general locations. This is a list of all storage that is not being used by another storage element or virtual circuit. It is expected that much of the storage that is not explicitly being used will be installed in the general spare pool, and this list of available (unused) storage will be the material and It is expected that it will be used as a convenience to make it easier to find unused storage elements. For example,
The partitions of the solid state storage device (SSD) are mirrored by a stripe set (RAID 0), and both this partition and the stripe set are included in the available list until they are mirrored. When a mirror is created from two members, the mirror is
Appears in the available list.

On the right side, information and parameters relating to the tree element selected by the user clicking the mouse with the mouse are displayed. When a storage element displayed in the available list is selected, it is selected in both the available list and the storage tree.

[0153] Addition and deletion functions are provided so that entries can be created and deleted. In addition, a change function enables a user to use a tool provided in the user interface to edit a storage element in the tree. Fields such as "Owner", "Last Repair Date", and "Description" can be changed. When the user specifies what he is trying to add (mirror, stripe, disk, etc.), an appropriate control set is given to this.

For the internal disk and external LUN, the user can specify items such as name, size, and manufacturer. Specifying an internal disk seems like a special case because the disk is a piece of hardware and is automatically detected. Users add disks only if they have "placeholders" for some hardware to be added later. This is also done for SSD substrates.

In the case of a RAID array, what happens?
Users can specify that they want to create an array of a specific RAID level (initially a mirror or stripe) and specify the storage elements that will be members of that array. This is done by selecting an entry from the list of available storage elements, where the array capacity is determined by the capacity of its members. The storage elements used as members of the array are then tagged as unavailable (since they are part of the array) and the array itself is added to the list of available storage. Each RAID array can also have a dedicated spare assigned to that array in case one of the members fails.

[0156] Partitioning of storage elements is also possible, by selecting the elements to be partitioned and specifying what size chunk the user wants. This creates two partitions if the element has not been split in the past. That is, the partition desired by the user and the remaining (unused) partition of the storage. Additional partitions are created from unused parts.

The detailed display for each storage element provides the maximum information available. One of the items displayed in the preferred system is what kind of partition for a particular storage element is
(Size and position).

[LUN mapping] By operating one button of the user interface, a LUN map routine is started. A LUN (logical unit number) map is essentially a list of LUNs and their associated data,
The VC (virtual circuit) associated with the LUN is shown on the display as a list of names and descriptions. When a user selects an entry from the LUN map and asks for its details, he can see it. The LUN map is
View a list of existing LUNs by name, description, and other fields. These fields include:-Name-Description-Exported state-Host-Storage element LUN map allows you to:-Sort based on various fields-Filter based on fields. This is only necessary if more than one LUN operates at a time (eg enable / disable). -Select LUNs for deletion or edit / view-define and add new LUNs-import existing LUNs (done in "learn mode" at hardware start-up)-add members and hot on LUNs Start copy mirror process-Export and unexport LUN. This basically starts and stops the flow of data from the host. A virtual circuit is defined (to the user) as a storage tree that can be activated by a button operation or other graphic configuration such as a dialog box connected to the host. The dialog box contains a field for entering the name of the LUN, a field for entering a description, and a target ID.
And a field for entering information for the exported LUN. The pop-up menu can be opened with a host button for a list of available hosts and a storage button for a list of available storage elements. The cache selection button is implemented as a check box.

A storage tree is actually a tree of storage members (eg, a mirror consisting of several stripe sets, a stripe set consisting of several disks). A host is actually a server that has a specific initiator ID and is connected to a specific port on a NIC. The user can define this by selecting a given host and a given storage tree representing the appropriate amount of available storage.

Use of the cache is limited to "on" or "off" using a check box. Solid systems provide tools for cache size and algorithm specifications. The use of a cache can be turned on and off during execution without interrupting the flow of data along the virtual circuit. The default when a LUN is created is "on".

In one embodiment of the LUN map, it has the functions required to create a virtual circuit, which consists of a multi-column table with two columns for host and storage. When you create a LUN, it is automatically exported and functions such as "add", "change", and "delete" are available.

In the LUN map display, a hot copy mirror is defined. This is usually done for existing LUNs. This is either the process of selecting a LUN and then selecting a storage tree to add to the existing storage tree through the addition of a mirror, or extending an existing mirror (for example, from two directions to three directions). .

[Support for Data Movement] FIG. 18 shows a three-stage data flow in a storage network connected to the first storage device 11 on the communication link 14 and to the second storage device 12 on the communication link 14. It is a schematic diagram. The intermediate device 10 also has a communication link
Connected to the client processor through 13, the intermediate device 10 receives a request to access the data at the logical address LUN A.

The storage server 10 includes a memory such as a nonvolatile cache memory used as a buffer,
Link the data access request received above with link 14
It has a logic engine that manages the hot copy process according to the present invention, as well as a data transfer resource for transferring to a storage device accessible at 15. This process can be understood by considering the three stages shown in FIG.

In stage 1, the storage server 10
Identifies the data set to be transferred, maps all data access requests received on link 13 to link 14 and connects to device 11, which stores the requested data set. The storage server initiates the hot copy process and receives a control signal identifying the target device, in this example device 12. This step starts stage 2 in which the dataset is transferred from the first device 11 to the storage server 10 as a background process.
To the second device 12 The parameters are stored on the storage server 10 and indicate how this data set is transferred and the relative priority of the background hot copy process for data access requests from client processors. During the hot copy process, data access requests are mapped to the first device 11 and the second device 12, depending on the progress of the process and the type of request. The storage server also includes resources to give priority to the hot copy process,
When the priority of the hot copy process is low, the client processor can immediately respond to the data access request. If the priority of the hot copy process is relatively high, the client processor will be delayed in responding to its data access request, but the hot copy process will be completed sooner.

When the transfer of the data set is completed, the stage 3 starts. In stage 3, a data access request from the client processor addressed to the data set is routed over the communication link 15 to the second device 12. The storage device 11 can be removed from the network together or used for another purpose.

The storage server 10 comprises, in a preferred embodiment, the storage domain manager described above.

Each of the storage devices 11 and 12 is an independent device or a logical partition in one storage unit. In this case, data moves from one address in the storage unit to another address by the hot copy process.

FIGS. 19, 20, 21 and 22 show some software of the hot copy process executed in the intelligent network server described above. Another storage server used for the hot copy process can exceed the configuration for a particular system. The structure of the virtual circuit, the persistent table storage, and the user interface will be described in detail with reference to the following figures.

FIG. 19 shows the basic data structure used in the hot copy process. The first structure 350 is called a "utility request structure", the second structure 351 is called a "utility structure", and the third structure 352 is called a "member structure". The member structure 352 is used to specify a specific virtual circuit and its status.
D), a logical block address (LBA) having the block number of the data block currently handled by the virtual circuit, the number of requests in the queue related to the virtual circuit, and parameters such as status parameters.

The utility structure 351 has parameters related to the currently executing utility, that is, the hot copy utility in this case, the “source ID” that is the source data set identifier, and one or more for the hot copy process. Of the destination storage device, a queue of requests to be executed with respect to the utility, and parameters such as a parameter indicating the currently handled block and its size.

The utility request structure 350 is
The request for the hot copy process is conveyed along with various parameters for this. These parameters include, for example, the "status" parameter that indicates the status of the request, various flags that support the request, a pointer to the corresponding utility structure, and the status of the request compared to input / output requests from the client processor. A parameter indicating the priority, a source mask identifying the dataset in the source,
For example, a destination mask that specifies the position of the destination device where the hot copy process copies the data set. In one embodiment, there are multiple destination masks for one hot copy request. As shown in FIG. 19, the utility request structure includes a logical block address (LB
A) is saved, which is also saved in the member structure for the data block in the dataset currently being handled.

To begin the hot copy process, it accepts user input, which creates a utility request structure. The persistent table storage in the storage server is updated with this structure, the status of the source and destination devices and the virtual circuits associated with that data are checked, the driver initiates the hot copy process, and the status parameters are stored in various data structures. Is set in The progress of the hot copy process is saved in persistent table storage in case of failure. If a failure occurs, the hot copy process can be restarted using other resources in the server and status information and data structures stored in persistent table storage. Other drivers in the system, such as the RAID monitor, are notified of the hot copy process. The request waits for its turn into the member structure.

When the setup is completed, an input / output process for supporting the hot copy process is started. The relative priority of the input and output processes that support the hot copy process determines the rate at which the hot copy process proceeds while the client processor is executing input and output requests for the same data set. In a preferred system,
Input and output requests from the client processor are executed first. If a block transfer that supports the hot copy process is being performed, upon receiving an input or output request from the client processor, the block transfer is completed as an atomic operation and the client processor's request is satisfied. In other systems, process priorities can be managed by different technologies.

FIG. 20 shows a basic process for executing a hot copy. The process begins with a hot copy request that has reached the top of the member structure queue (step 360). Next, a buffer in the storage server is allocated to support block transfer (step 361). A message is issued to move a copy of the first block in the dataset to the buffer (step 36).
2). The current block is moved to a buffer according to the priority set for the hot copy process (step 363). Block movement is performed using an appropriate memory lock transaction to control access by multiple processes within the storage server. Next, a message is issued that moves a copy of the block from the buffer to one or more destinations.
(Step 364). This block is moved to one or more destinations according to the priority for the hot copy process (step 365). As the block moves, the local data structures supporting the persistent table store and process are updated with status information indicating the progress of the hot copy (step 366). The process determines whether the last block of the dataset has been copied (step 367). If the copy has not been completed, a message is issued to move the copy of the next block to the buffer (step 368). The process loops to step 363 and continues moving blocks of the dataset to the destination. If it is determined in step 367 that the last block of the dataset has successfully moved to the destination, the process ends (step 369).

According to one embodiment of the present invention, for a hot copy process with multiple destinations,
It is possible that one or more members of the destination group being used may fail during the process. In this case, the process can continue at one or more destinations that continue to operate, and the corresponding table is updated in support of the continued process.

Thus, the hot copy function is used to copy a data set from one member that is not yet down to the replacement drive. The data set includes the entire contents of the storage device or a part of the contents of the storage device. The hot copy function can be used with any level of RAID array, while properly managing status and parameters.

The parameters of the hot copy include the priority of the process, the source member device, and the destination identifier. The hot copy request includes a source member identifier, a destination member identifier, a copy block size, a copy frequency or a priority. Hot copy is performed one block size at a time, according to priority. The current block position is stored in the array configuration data in the data structure as described above. The hot copy process is performed simultaneously with the normal input and output processes. Because writes to the hot-copied drive are made to both drives, the original source member remains valid if the hot-copy is interrupted or fails. Upon completion of the hot copy, the original source member is removed from the array and designated as unavailable by the system manager program. Similarly, in one embodiment, the virtual devices that support the dataset are updated to target a new destination.

FIGS. 21 and 22 show a process performed in the storage server to manage a data access request issued by the client processor during the execution of the hot copy process. Data access requests are read requests, write requests, etc.
One type or a variation of the same type may be used. Other requests include requests to support management of data channels and the like. FIG.
Shows one process that handles write requests.

When the write request reaches the top of the queue, the process starts (step 380). The process determines whether the write request specifies a location in the dataset that is the subject of the current hot copy process (step 381). If it is in the dataset that is being hot copied, the process determines whether the block indicated by the write request has already been copied to that destination (step 382). If so, a message is issued that writes to both the storage device that originally held the dataset and one or more destination storage devices (step 383). Next, the data is moved according to the priority of the input and output requests (step 384), and the process is completed (step 385).

At step 381, if the request is not in the dataset, a message is issued to execute writing of the dataset to the source (step 386),
At this point, the process flow moves to step 384. Similarly, if it is found in step 382 that the location to be written to has already been copied, a message to write to the source device is issued (step 3).
86).

FIG. 22 shows the handling of a read request generated during hot copy. The process begins when a read request reaches the top of the queue for a virtual device (step 390). First, it is determined whether the read is within the data set to be hot-copied (step 391), and if the read is within the data set, whether the read is within a block already copied to one or more destinations It is determined whether or not it is (step 392). If the read is in a block already copied to the destination, a message is issued to read the data from the new location (step 39).
3). In other systems, reads are made from the source device, or both the source and destination devices, depending on reliability, speed, and other factors that affect the management of data traffic in the system. After step 393, the data is returned to the requestor according to the priority of the data access request of the client processor (step 394). Here, the process ends (step 395).

If it is determined in step 391 that the read request is not in the data set to be hot-copied, a message for reading the source device is issued (step 396). Similarly, if it is determined in step 392 that the read request is addressed to a block that has not yet been copied to the destination, a message to read data from the source device is issued (step 396). After step 396, the process returns to step 394.

If a data read or write request occurs within a particular block while the block is being moved through the storage server buffer, a data lock algorithm is used to manage the handling of this request. For example, if a logical block is locked in support of a hot copy process while a read or write request is being received, the client processor may reject the read or write request because the data is locked. Receive notifications. In another system that gives the client processor a higher priority, read or write requests are continued, while blocks held in buffers that support hot copy are deleted, and the status of the hot copy is reset, It indicates that the block has not been moved. Various data lock algorithms can be used as needed for individual uses.

[Target Emulation] In the configuration shown in FIGS. 1 to 3, the storage server operates as an intermediate device between a data user and a storage device in a storage domain for storing data. In this environment, the server is provided with resources that emulate a legacy storage device to support legacy storage devices, ie, devices that existed before the server was inserted as an intermediate device. Thus, when the server is inserted between the legacy device and the user of the data, the server virtually determines the logical address of the legacy device according to the storage channel protocol used between the user and the legacy device. . Next, the storage server processes all requests addressed to the received legacy device according to the protocol. In addition, the required configuration information is played back from the legacy device and stored in local memory so that the status and configuration information expected by the user at the legacy device is provided using local resources in the server. . This can eliminate the communication between the server and the legacy system, and the server can spoof the operation of the legacy device according to the storage channel protocol, and does not require user reconfiguration when adding the server to the storage network, or It is greatly simplified.

[Summary] Storage Area Networking (SAN) is a new storage-centric computing architecture. With the availability of fiber channel-based storage subsystems and network components primarily, SANs have become faster data access and data movement, more flexible physical configurations, improved storage capacity utilization, and centralized storage. Management, use and reconfiguration of online storage resources, promise a heterogeneous environment.

In the traditional “direct attach storage” model, storage resources have a high-speed direct physical path to only one server, and all other servers access the storage resource at very low speeds only indirectly through the LAN. Had access. Storage area networks change this by providing high-speed access paths (using fiber optic channels) directly from individual servers to individual storage resources in a "networked" topology. The introduction of a network architecture also greatly enhances the flexibility of storage configurations, separating storage resources from specific servers and allowing them to be managed and configured with little impact on server-side resources. I have.

While SANs offer the right topology to meet the flexibility and data access needs of today's environment, SAN topologies themselves are not sufficiently addressing business problems.
Simply connecting servers and storage resources physically through SAN fabric components, such as switches, hubs, and routers, does not fully realize the potential of SANs, but SAN fabrics provide the required security. Certainly, it provides the hardware infrastructure to provide a centralized storage management function. The combined use of these two developments provides the flexibility required to achieve business goals in a new environment, and provides access to important data anytime, anywhere.

The management function required on the infrastructure of the SAN hardware is storage domain management. For optimal storage flexibility and high-performance access, storage domain management is most efficiently located in the SAN itself, rather than in a server or storage device. Approaches using server-based and storage-based resources are:
It is not optimal because neither the server side nor the storage side can fully support heterogeneous states.

Storage domain management is a centralized, secure management function that is installed on the existing SAN hardware infrastructure, and provides high-performance, high-availability advanced storage for heterogeneous environments. Provide management capabilities. The purpose of storage domain management is to integrate traditional and new equipment, free server and storage resources from SAN and storage management tasks, and to host SAN-based applications available through all SAN components. It is to constitute the core of a simple SAN fabric. Although SANs can be built without storage domain management, building and managing an optimized SAN environment requires this crucial capability.

The basic elements of storage domain management include:-Heterogeneous interoperability-Centralized management with security-Scalability and excellent performance-Enterprise-class reliability, availability and maintenance Gender-An intelligent, purpose built platform The field of storage domain management allows customers to leverage the full power of SAN to address business issues.

It is a matter of life and death in a corporate environment to be able to cope with a heterogeneous environment by linking servers and storage and by mergers and acquisitions that have become common in today's new business situations. In a product set that provides SAN functionality for a single manufacturer's product line, customers cannot achieve the full capacity of the SAN, and even if they add new server and storage products and take advantage of them, Storage domain managers must support at least Fiber Channel and SCSI attachments because they need to keep their investment in equipment. As storage domain managers need to evolve to adapt to new technologies that will be introduced over time, the platform will steadily grow, providing broader multi-protocol connectivity. SANs create large virtualized storage pools that can be managed centrally, reducing storage management tasks compared to traditional "direct attach" storage architectures, especially for backup / replay and disaster recovery Is done. Although SANs effectively provide a physical access path from all servers to all storage, security is not a secure method because not all storage is logically accessible to all servers. I have to deal with it. SAN
Fabric manufacturers do this by logically defining "zones" so that each server can only access data that is defined to be within that zone. Obviously, the ability to define a secure zone or storage "domain" is one element of the storage domain manager. Not at the port level
By defining domains in more detail, such as defining what is included in a zone at the LUN level, the use of storage assets can be more flexibly improved in the future.

The storage domain manager provides a comprehensive, centralized storage management function available from one management interface through all connected servers and storage, regardless of manufacturer. From a central location, a system administrator manages the movement or mirroring of data between heterogeneous storage resources and can dynamically utilize these functions over time for various heterogeneous storage resources. The result is significant cost savings and simplified management. As an intelligent, scalable platform, the storage domain manager is located in a completely central location and hosts storage management functions that are available across all connected servers and storage resources.

Looking at the expansion rate of storage, which is spurred by a new business situation, for a certain SAN environment, the storage capacity easily increases 100 times during its lifetime. The storage domain manager, which is positioned as the central intelligence of the SAN, must also be able to adapt to the rapid growth so that the performance does not deteriorate in response to the load. In order to achieve smooth and cost-effective scalability over a wide operating range, intelligence should be added as the configuration expands. The ability of intelligent platforms to store large amounts of data in cache memory optimizes SAN configurations and improves performance in application-specific environments. For example, when "hot spots" such as file system journals, database table indexes or logs are cached in high-speed storage within the storage domain manager itself, older SAN configurations built without the storage domain manager can be used. In comparison, the waiting time of the message path is greatly reduced. Given the large amount of on-board storage, the entire database and file system are effectively cached, resulting in significant performance improvements. On-board storage capacity is also important in staging data during data movement and other data migration activities. As mentioned earlier, one of the main reasons for moving to a SAN is to improve overall data accessibility. If any single point of failure occurs as a result of moving to this new storage architecture, many of the benefits cannot be realized. Therefore, not only the data itself but also the access path to the data must be always available. Failure downtime must be reduced by using relative internal components and functions, such as automatic I / O path failover, logical hot-sparing and pluggable, and hot-swappable components. Downtime needs to be further reduced using online management features such as online firmware upgrades, dynamic reconfiguration of hardware and software, and high performance background data movement.

To ensure the highest level of performance, the storage domain manager should be an intelligent, purpose-built platform specifically optimized for the storage-related tasks for which it is needed. preferable. The platform is backed by the high-speed local storage required to run data movement and storage management applications and supports critical local processing power to perform various storage management tasks.

Compared to using a general-purpose platform as an intelligent storage server, a purpose-built platform provides a much faster and more deterministic response time, a real-time operating system, message latency. To provide a more efficient I / O passcode that reduces operating system kernels optimized as data mover engines rather than application engines. This purpose-built platform supports kernel-level features such as reliable and innovative messaging that are not available with general-purpose operating systems. High availability features such as integrated path failover, online management and dynamic reconfiguration are supported by the core operating system. By having intelligence in the best places to support heterogeneous SAN environments, storage domain managers provide end users with the following business benefits:-Improve storage asset allocation and utilization-Rapid Flexibility to cost-effectively accommodate growing and fluid storage environments-High availability through online management and configuration-More efficient management that lowers the overall $ / Gigabit cost of storage management-Servers in an integrated SAN environment Ability to bring storage and storage together-JB by adding storage management and caching capabilities that can be used dynamically across all storage resources
Increase the value of OD storage.

The robust SAN hardware infrastructure employed in conjunction with storage domain management allows for rapid and unpredictable changing environments,
Provides the flexibility to access highly available data reliably and quickly. The centralized storage management paradigm implemented in this way is a more efficient and lower-cost method of managing data expansion that offers competitive advantages for companies.

The above description of the various embodiments of the present invention is provided by way of example and description, and is not intended to limit the invention to only the specific forms introduced. Various modifications and equivalent arrangements will be apparent to those skilled in the art.

[0199]

As described above, according to the present invention,
A system that simplifies storage system management while utilizing the flexibility and capabilities of the SAN architecture can be provided.

[Brief description of the drawings]

FIG. 1A shows a storage area network including a storage server according to the present invention configured as a storage router or a storage director for storage domain management, and FIG. 1B shows some intelligent storage area network servers. Diagram showing the use of

FIG. 2 shows a storage area network according to another configuration having a storage server according to the present invention configured as a storage router or storage director for storage domain management in a heterogeneous network.

FIG. 3 illustrates a more complex storage area network in which multiple storage servers according to the present invention are configured with direct communication channels between each other to support a wider storage domain or multiple storage domains.

FIG. 4 is a block diagram of a storage server that supports storage domain management according to the present invention.

FIG. 5 is a block diagram showing another example of a storage server that supports storage domain management according to the present invention.

FIG. 6 is a block diagram of a hardware architecture of an intelligent storage area network server.

FIG. 7 is a block diagram of software modules and support programs of an operating system for an intelligent storage area network server.

FIG. 8 is a simplified diagram of a hardware driver module for an optical fiber channel interface used in a system according to the present invention.

FIG. 9 is a simplified diagram of a solid state storage system including a hardware driver module according to the present invention.

FIG. 10 is a diagram of an internal array of disk drives used in one embodiment of the storage server according to the present invention.

FIG. 11 is a schematic diagram of a target server internal service module according to the present invention with a local answer function.

FIG. 12 is a diagram of an internal service module using a disk mirror;

FIG. 13 is a diagram of an internal service module using a partition function.

FIG. 14 is a diagram of an internal service module using a cache function.

FIG. 15 is a diagram showing a virtual circuit configuration according to the present invention.

FIG. 16 is a diagram of an internal service module using a persistent table store manager according to the present invention.

FIG. 17 is a schematic diagram of a persistent storage hardware driver module according to the present invention.

FIG. 18 is a simplified diagram of a network with an intermediate device having three-stage hot copy resources according to the present invention.

FIG. 19 is a diagram showing a data structure used in an example of a driver using a hot copy process according to the present invention.

FIG. 20 is a flowchart showing a hot copy process executed by the driver according to the present invention.

FIG. 21 is a flowchart showing handling of a write request during a hot copy process.

FIG. 22 is a flowchart showing handling of a read request during a hot copy process.

[Explanation of symbols]

 1201,1202,1203… Client server 1204… Hub 1205,1206,1207… Device 1210,1211,1212… Client interface 1213,1214… Storage interface

 ──────────────────────────────────────────────────続 き Continued on the front page (31) Priority claim number 455106 (32) Priority date December 6, 1999 (12.6 December 1999) (33) Priority claim country United States (US) (31) Priority Claim No. 482213 (32) Priority Date January 12, 2000 (2000.1.12) (33) Priority Country United States (US) (71) Applicant 597001637 One Dell Way, Round Rock, TX 78682-2244 , United States of America (72) Inventor Alan Earl Merrell, Cenin Blank Drive, Fremont, CA 94539, United States of America, 48835 (72) Inventor Joseph Altmeier, United States of America, 52327, Riverside, 52327 , Five Hundred For Teeth Street Essex, 3689 (72) Inventor Jerry Parker Lane Tonino Drive, San Jose, CA 95136, United States of America 4829 (72) Inventor James A. Taylor United States of America, 94550, California Livermore, Florence Lord 1033 (72) Inventor Ronald El Parks United States, California 94526, Danville, Mustang Court 55 (72) Inventor Alastair Taylor United States, California 95123, San Jose, Calero Avenue 755 (72) Inventor Shari Jay Nolan Pinnacle Drive 3470 (72), San Jose, California 95132, California Inventor Jeffrey ES Nespo United States, 94566, California, Pleasanton, Corte Vera Vera Luz 2720 (72) Inventor George W. Harris Jr., United States, 94041, California, Mountain View, View Street 327 (72) Inventor Richard A. Lego Jr. United States, New Hampshire 03051, Hudson , Pinewood Road 11

Claims (18)

[Claims] 1. One or more clients and one or more storage systems running respective storage channel protocols that convey sufficient information to identify clients in which storage transactions take place.
In a system for managing a storage domain in a storage network, a plurality of one or more clients and one or more storage systems are selected to be connected to each one via a communication medium and operate according to various communication protocols. A communication interface, coupled to the plurality of interfaces, for storing at least one storage location set from the one or more storage systems;
A processing unit including logic configured as a storage domain for one set of clients, and including logic for routing storage transactions within the storage domain in response to identified clients; and a storage transaction between the plurality of communication interfaces. And a logic to convert the transaction to a common format or to obtain a common format from the transaction; and a redundant resource including a non-volatile cache memory for routing storage transactions in a common format between communication interfaces in the storage domain; A management interface connected to the processing unit and configuring the storage domain. Di-domain management system. 2. The one or more clients execute respective storage channel protocols that convey sufficient information to identify a logical storage location and perform storage transactions within a storage domain corresponding to the logical storage location. The storage domain management system according to claim 1, further comprising logic for routing. 3. The storage domain management system according to claim 1, further comprising logic for managing data set migration from one storage location to another storage location in a network. 4. The storage domain management system according to claim 1, wherein the interface includes resources for configuring a plurality of storage domains in a network. 5. A storage resource configuration and management method in a storage network, comprising: installing an intermediate system of the network between a client and a storage resource in the network; and using a logic of the intermediate system to change a logical storage range in the network. Assigning to a client, using the logic of the intermediate system, allocating storage resources in a network to a logical storage range, and storing the data through an intermediate device according to the logical storage range assigned to the client and the storage resources assigned to the logical storage range A method characterized by routing transactions. 6. A communication interface for supporting a storage transaction communication channel, logic for converting a storage transaction received on the storage transaction channel into an internal format, and storing the storage transaction in the internal format in communication with the storage server. And a logic for routing to a virtual circuit that manages connection with the data store of the storage server. 7. The storage server of claim 6, wherein the virtual circuit comprises logic for converting an internal format into one or more communication protocols for a corresponding one or more data stores. 8. The communication system of claim 7, wherein each communication protocol for each corresponding data source comprises a protocol that conforms to a standard "intelligent input / output" (I ₂ O) message format. Storage server. 9. The logic for routing a storage transaction to a virtual circuit includes a table, the table having a plurality of entries, wherein the plurality of entries are between an address range specified by the storage communication channel and the virtual circuit. 7. The storage server according to claim 6, wherein the correspondence of the storage server is indicated. 10. The logic for routing storage transactions to a virtual device includes a table, the table having a plurality of entries, the plurality of entries indicating a correspondence between a virtual circuit and each data source. The storage server according to claim 6, wherein 11. The storage server according to claim 6, including a cache, wherein a virtual circuit communicates with the cache. 12. The storage server according to claim 6, wherein each data source has a non-volatile memory. 13. The storage server according to claim 6, wherein each data store has a hard disk array. 14. The storage server according to claim 6, further comprising a user interface that supports input of configuration data. 15. The storage server according to claim 14, wherein said user interface comprises a graphical user interface. 16. The storage server according to claim 14, wherein the user interface comprises a touch screen connected to the storage server. 17. At least one client system issuing a request for a storage transaction, one client communication channel into and out of the client system, a plurality of storage devices, and a plurality of storage devices. And a communication channel emanating from the storage system, comprising: a processor including a bus system; a client interface to the client communication channel connected to the bus system; and a communication interface connected to the bus system. A plurality of interfaces to each communication channel, a non-volatile cache memory connected to the bus system, and a storage Receiving a request for a transaction, directing the requested storage transaction to the plurality of storage devices, and controlling the processor to allocate the non-volatile cache memory for use in the storage transaction. A server characterized by the above-mentioned. 18. The server of claim 17, wherein the resources controlled by the processor include a process for authenticating and verifying access permissions for storage transactions.