WO2014189607A1

WO2014189607A1 - Systems and methods including an application server in an enclosure with a communication link to external controller

Info

Publication number: WO2014189607A1
Application number: PCT/US2014/031256
Authority: WO
Inventors: Mohamed EL-BATAL
Original assignee: NetApp Inc
Current assignee: NetApp Inc
Priority date: 2013-05-23
Filing date: 2014-03-19
Publication date: 2014-11-27
Anticipated expiration: 2015-11-23
Also published as: US9489151B2; US20140351507A1

Abstract

A system including a first array of storage drives in communication with a storage controller, the storage controller providing virtualized storage with the first array of storage drives; an application server running a storage warehouse application and communicatively coupled to a host device and the storage controller; and a communication port expander providing expansion slots to a first enclosure, wherein the first array of storage drives, the application server, and the communication port expander are contained in the first enclosure separate from the controller, further wherein the controller controls the first array of storage drives and is in communication with the first array of storage drives through the communication port expander.

Description

SYSTEMS AND METHODS INCLUDING AN APPLICATION SERVER.

IN AN ENCLOSURE WITH A COMMUNICATION LINK TO AN EXTER L CONTROLLER

TECHNICAL FIELD

The present description relates to network storage systems and, more specifically, to systems and methods in which application server .hardware and software are included with communication links to a storage array controller in a separate enciosure.

BACKGROUND

Conventional network storage systems may he implemented using the Storage Bridge Say standard. The Storage Bridge Bay (SBB) specification is a standard for storage solution providers who desire to have a standard controller and slot compatibility between respective controllers which can be manufactured by different organizations. The SBB specification describes She physical requirements for storage canisters and the id- plane connectors of a storage enclosure. The storage enclosure can house one or more storage canisters. Meanwhile, each storage canister houses at least one printed circuit board having electronics mounted thereon. A current version of the SBB specification, as of this writing, is Storage Bridge Bay Specification, Version 2,0 released Jan. 28, 2008, the entire contents of which are hereby incorporated, by reference.

Storage enclosures are conventionally designed to fit in a standard rack defined in EiA-3 10. A rack unit (or "U" ) is a measurement of height within the rack, where one rack unit equals 44,45mm or 1.75 inches, A storage or computing de vice that is embodied within an enclosure that it is one rack unit tali may be referred to as a "1U" device

Similarly, a storage or computing device that is embodied in an enciosure thai is two rack units tali may be referred to as a 2U device. A given rack can be any height, but is usually 1 or 23 inches tall,

A conventional, network applicat ion system includes two application servers in different 21 J enclosures, a RAID Bunch of Drives (RBOD) in a 2U enclosure, and an Expansion Bunch of Drives (EBOD) in a 2U enclosure, tor a total of SU taken up by four enclosures. The RBOD includes two storage controllers, and the EBOD includes two

I expansion Enclosure Service Modules (ESMs). However, conventional techniques for configuring a network storage system may not be as efficient as could be desired,

BRIEF DESCRIPTION OF I HE DRAWINGS

The present disclosure ts best understood from the following deiaiied description when read with the accompanying figures.

Figure i is a system diagram of a computing infrastructure including a network storage system according to aspects of the present disclosure.

Figure 2 is an illustration of an example circuit board of Figure i. adapted according to one embodiment.

Figure 3 is an illustration of an example .mechanical and cabling diagram, tor the system of Figure ! .

Figure 4 is a system diagram of a computing infrastructure including a network storage system according to one embodiment.

Figure 5 is an illustration of an example circuit board, adapted according to the embodiment of Figure 4 to include a dense server ASN and ESMs.

Figure 6 is a system diagram of a computing infrastructure including a network storage system according to aspects of the present disclosure .

Figure 7 is a system diagram of a computing infrastructure including a network storage system according to aspects of the present disclosure- Figure 8 is an illastration of example rack, adapted according to one embodiment.

Figure 9 is a block diagram of a method of installing a network storage system according to aspects of the present disclosure.

Figure 10 is an illustration of an example ROOD, adapted according to one embodiment.

Figures 11 and 12 are illustrations of methods to provide virtual storage to a network storage system, according to one embodiment.

.DETAILED DESCRIPTION

All examples and illustrative references are non-limiting and should not he used to limit the claims to specific implementations and embodiments described herein and their equivalents. The headings are solely for formatting and should not be used io limit the subject matter in any way, because text under one heading may cross reference or apply to text under one or more headings. Finally, in view of this disclosure, particular features described in relation, to one aspect or embodiment may be applied io other disclosed aspects or embodiments of the disclosure, even though not specifically shown in the drawings or described in the text.

The present disclosure relates to network storage systems and methods of implementing network storage systems that provide for more efficient use of^" space in a computer housing (e.g., a rack), In one example, a network storage system includes an Application Server Node (ASM) in a same enclosure with an ESM, where the ESM includes expansion ports. The enclosure also includes a first array of storage drives A different enclosure includes a storage controller providing virtuaiized storage functionality for the array of storage drives. The storage controller communicates with the array of storage drives via the expansion ports of the ES

Thus, in this example, even though the array of storage drives is located in the sanie enclosure as the ASN, it is not the processing power of the ASM that it used as a storage controller. Rather, the storage controller functionality is provided by a storage controller in a separate RBOD enclosure that takes advantage of the expansion ports of the ESM to communicate with the storage array

in one example, the enclosure that includes the storage controller also includes another array of storage drives that are also controlled by the storage controller. The first enclosure and the second enclosure each occupy 2V in the rack, thereby providing a 4U storage system. B contrast, a conventional network storage system would use separate enclosures for an ASN and an ESM, thereby using more units in the rack.

In. another example, an. enclosure includes a High Availability (HA) pair o ASNs, each of the AS including an ESM with expander ports. The enclosure may be still be embodied as a 2U enclosure. The space savings may he achieved by embodying the ESMs as Peripheral Component Interconnect Express (PCfej cards that occupy card slots of the ASN hardware. Ho wever, the scope of embodiments is n t limited to PCle cards as ESMs, as any appropriate connection between the ASN hardware and the ESM hardware may be used.

The block controllers may also be embodied as an HA pair, where each block controller is redundantly connected to each ASN and to each ESM. In tact, the architecture described herein may be adapted for use in clustered systems, as shown in the figures.

In yet another example, the ASN is embodied as a dense server, which has two or more processors and may use more room on a circuit board than a server that has only one processor. An example of a (lease server is one that uses two or four Intel™ Xeon™ processors, though the scope of embodiments is not limited to any particular type or number of processors, in some embodiments employing dense servers as ASNs, there may not be enough space within a 2^'U enclosure to house two or more redundant ASNs. Thus, some dense server embodiments may include only one ASN in a given enclosure, along with at least one ES . Redundancy can then be achieved by using an additional

ASN/^'ESM enclosure and making redundant connections between the storage controller and each of the ASNs.

The various embodiments described above are examples, and ft is understood that the scope of embodiments includes storage systems with different numbers of controllers, ASNs, and storage arrays. In fact, any of the embodiments described herein may be scaled larger or smaller depend ing on particular needs of a given implementation.

Figure i is a system d iagram of a computing infrastructure 100 including a

.network storage system according to aspects of the present disclosure, infrastructure 100 includes enclosure 1 10 and enclosure 120. which may be adapted according to the EIA-3.10 standard to fit into standard racks. However, the scope of embodiments is not limited t standard enclosures, and any type of enclosure, whether standard or otherwise, may be adapted for use tn some embodiments.

Enclosure 100 includes circuit boards 101 and 102, as well as storage array 1.03. in this example, circuit board 101 includes hardware and software to provide ASN and ESM functionality, as does circuit board 1 2, The ASNs of circuit boards 101 and 102 are configured as a. HA pair.

The ASNs of circuit boards 101 and 102 are in communication over the customer network with one or more host devices (not shown). The ASNs of circuit boards 101 and 1 2 are also in communication with a cluster interconnect, which may provide

communication with other devices in a cluster (e.g., additional ASNs, not shown).

Connections to the customer network and cluster interconnect may include Ethernet, infmiband, IEEE 802, 11, and/or any other appropriate communication link.

The ASNs of Figure .1 host storage applications that are made available to host systems that access to the storage system. For example, some ASNs run applications to pro ide; continuous data protection through automated backup procedures, database management application processes, snapshot management processes, the de-duplication management processes, and the like. Other examples o applications include a cluster- based file system program (e.g., a Lustre™ application), a high-peribrmanee computing application (a data warehousing application, a video delivery and processing application, an Oracle™ database application, etc.), a big data application (e.g., Hadoop™ or other map-reduce application), and the like.

In some examples, the AS s utilize irtual Machine Managers (commonly referred to as "hypervisors") to pro vide a virtual machine for each of the multiple

application processes, lit general a hypervisor controls the overall operation of each of a plurality of virtual machines. Each virtual machine may include its own specific operating system kernel and associated application processes such that the hypervisor hides the underlying physical hardware circuitry interfaces from the operating system and

application processes operating within a virtual machine.

The ASNs of Figure 1 may use any appropriate operating system or hypervisor, hut in this example, the ASN may include one or more general purpose processors (e.g. , Intel Xeort or other processors) running a data warehousing program (e.g., a map reduce program} on top of Linux.

The ESMs of Figure I. include hardware and software functionality to manage the physical operation of hardware within the storage system. Examples of functionality include environment reporting and setting via SCSI enclosure services, such as temperature control power management, and the like. The ESMs may include their own processing power and memory suitable to provide such functionality (e.g. , by an Application Specific integrated Circuit or Field Programmable Gate Array), and in many instances, the amount of processing power and memory used in an ES is less than that used in an application server.

Further, in this example, the ESMs of Figure I include expander ports for connecting further devices within the cluster. Before discussing the expander ports further, a discussion of the data transfer links may be instructive. System 100 may be implemented w ith, serial attached SCSI (SAS) architecture or a like architecture suitable lor supporting full duplex data transfer, SAS may refer to a point to point serial interface suitable tor providing lull duplex communication at three, six, or twelve gigabits per second (Gfa/s) or higher per direction per channel. Protocols that may communicate via a SAS interface may be small computer system interface (SCSI}, advanced technology attachment (ATA). SAS management or like protocols.

Expander devices included in the ESMs of Figure 1 may include edge expander devices, edge expander device sets, and lanout expanders. SAS devices, such as end devices and expander devices, may be interconnected to deliver scalable storage in. a SA.S environment. SAS devices are typically interconnected with a set of point-to-point links in the SAS domain. SAS devices may also include at least two connections for enhanced performance and availability. n an interconnection of SAS devices or SAS domain, each logical connection to a node is made via a port. A port may include a plurality of p int-to- point links, which may be denoted as phys. For instance, a typical media device, such as a storage drive, may comprise two ports with each port including at least one phy.

Additionally, multiple phys may be arranged together to make op a port, allowing for multiple concurrent connections to be established. In some embodiments, the expanders included in the ESMs of Figure 1 inelude edge expander devices or like expander devices suitable f r communication with up to 128 SAS addresses. An edge expander device enables communication to be established between nodes that are directly connected to the edge expander device.

Nevertheless, as mentioned above, the scope of embodiments is not limited to SAS Other suitable data transfer links may be appropriate, suc as Fibre Channel (FC), Ethernet Infmiband, IEEE 802.1 1, and the like.

Moving to enclosure 20. block controllers 121 and 122 are embodied in circuit boards housed within enclosure 1 0 along with storage drive array 123, Block controllers 121 and 1.22 in this example are configured as a HA pair, though the scope of

embodiments is not limited just to HA pairs.

Each controller 1 1 , 122 includes one or more processors or application specific logic that causes connected storage arrays to read or write data to or from persistent storage in response to requests received -from an external host. Each controller .121. 122 may also include memory that acts as a buffer or temporary storage for data being transferred between the host and persistent storage. Each controller 12 K 1 2 may further include host input output (I/O) controllers (HICs) that interface ihe storage array controller with the host, and target I/O controllers (TiCs) that interface the storage array controller with the storage drives. The block controllers 121, 122 may include processing power and memory suitable to virtuaiize storage and control reads and writes (e.g.. by an Application Specific Integrated Circuit or Field Programmable Gate Array), and in many instances, the amount of processing power and memory used in a block controller .12 , 122 is less than that used in an application server,

in one example, each block controller 121 , 122 viriuaiizes the storage within storage arrays 1.03, 123 to provide RAID storage. The acronym "RAID" is an umbrella term for computer data-storage schemes that can divide and replicate data among multiple physical storage drives When several physical storage drives are set up to use RAID technology, the storage drives are said to be in a RAID arra . The RAID array distributes data across several storage drives, but the array is exposed to the operating system ai the application server as a single logical storage drive or multiple logical storage drives, where the logical storage drives do not correspond one-to-one with the physical storage drives. Examples of storage drives that tnav be used in various embodiments include Solid State Drives (SSDs), Hard Disk Drives (HDDs), and the like.

Enclosure 120 may be referred to as a RAID Bunch of Drives or (R.BOD). In one example, block controllers 121, 122 are essentially identical storage controllers coupled with one another to provide redundancy in ease of failure of one. Figure 10 provides a more detailed illustration of RBOB 120 according to one embodiment, in Figure 10, controller 121 lias a circuit board 191 with a CPU, an SA.S I/O controller, and an AS expander. Similarly, controller 122 has a circuit board 1 2 with a CPU, an SAS I/O controller, and a SAS expander. The controllers 121. 122 are cross -coupled via their SAS I/O controllers. Further, each, of the storage drives in. array 123 is communicatively coupled to each controller 121 , 122. E ach of the RBODs shown in the various

embodiments herein may include cross-coupled controllers, as shown i Fig, 1 0. In the exemplary embodiment of FIG. 2, block controllers .121 , 122 are interconnected via a path (not shown), which, may provide any of a variety of well-known communication protocols and media including, for example, PCI (e.g., PCI Express), SAS, Fibre Channel, fnfmiband, Ethernet, IEEE 802,11, etc. This inter-controller interface and medium is utilized for exchanges between the controllers 121 , 122 within the storage enclosure .120.

The various components of system i 00 are communicatively coupled as follows. Block controller 12 1 is connected to the ASNs of circu it boards 101 and 102 via

communication link 141. Thus, block controller 121 is connected to both ASNs.

Similarly, block controller 122 is connected to the ASNs via communication links 142 Communication links 141 and 142 may include any appropriate type of communication link, including but not limited to SAS, Fibre Channel, infrniband, Ethernet, IEEE 802, 11, and the like.

Block controller 121 is connected to the ES s of circuit boards 10.1 and 1.02 via communication links 143, Thus, block controller 121 is connected to both of the redundant ESMs in enclosure 1 10. Similarly, block controller 122 is connected to the ESMs by communication links 144. Communication links 143 and 144 may include any appropriate type of communication link., including hot not limited to SAS, Fibre Channel, fnfmiband,

Ethernet, IEEE 802, i i, and the like.

Block controllers 121 and 122 communicate with, and control, storage drive array 123 through a midpiane interface (not shown) or other interlace within enclosure 1 0. Furthermore, block, controllers 121., 1.22 communicate with, and control., storage drive arra 103 (^'in enclosure 1 10) via communication links 143, 144 and the EMS, in other words, storage drive array 103 in enclosure 1 10 is in communication with controllers 121 , 122, which are external to enclosure 110, through the ESMs that are included within enclo ure 1 1 ,

The embodiment shown in Figure 1 may provide one or more advantages over conventional network storage systems. Combining AS.N and. ESM hardware into the same enclosure may provide space savings by creating higher density of^" storage drives. Thus, the ratio of number of rack units used for storage versus the number of rack units used for processing can be increased, increasing such ratio may be desirable because, as processing power increases, the ability to control storage drive arrays using a same size circuit board aiso increases; but as the number of rack units for storage increases, there is less space for processing power in the rack.

Figure 2 is an illustration of an. example circuit board .101 of Figure i, adapted according to otic embodiment, ft is understood that in various embodiments, system 500 (Figure 1) may include circuit board 102, which is substantially similar to circuit board 101. It should also be noted that the specific implementation shown in Figure 2 is an example, and other embodiments with different types and numbers of I/O ports, processing power, memory, and the like may be adapted for use. in one example, the hardware of a RAID controller may be used as the hardware of circuit board 101. For instance, if a RAID controller ixiard has sufficient processing hardware, its RAID controller software and firmware may be eplaced with application server hardware and firmware to provide application server functionality in any of the embodiments of Figures 1 , 6, or 7. in other w ords, the embodiment of Figure 2 shows a modified RAID controller board used as an ASN. However, any appropriate processing hardware may be used in any of the embodiments.

Dashed line 201 shows a division between the ASN hardware and the ESM hardware. Labels to the left side of circuit ixiard 101 show that some ports communicate with, the cluster interconnect, the host network, and the controllers 12 1 and 1.22. Among other things the ASN hardware includes CPU 210, H1C card 2 ! 1 , and SAS input/output control 212 {to communicate with controllers 121 and 122).

The ESM hardware can be embodied as a PCEe card, mounted directly on board ! 0.1 , or any other appropriate configuration.. Among other things, the ESM hardware includes expander 213, which provides ports for communication with controllers 12 S. and 122. The communication with controllers 1 1 and 122 is passed through connection 214 to midpiane connector 15 and to storage array 103. Thus, as explained above,

controllers 12 ! arid 122 control storage array 1 3 through the expander 213 provided by the ESM within enclosure 1.10. Storage array 103 in this example is not implemented on circuit board 101, but is instead implemented on a different circuit board in communication with midpiane connector 2.15. Glue logic for the ESM is provided in this example by

FPGA 216, which provides less processing power than does CPU 210.

Figure 3 is an illustration of an example mechanical and cabling diagram for the system of Figure 1. Enclosures 110 and 1.20 are shown as standard 2U rack-mounted enclosures Fig. 3 also shows coniniun.fca.tion. links 141 -1 4 with respective SAS pons, and this arrangement is consistent with the communication link layout shown in Figure 1. Ports labeled "IB" are Infmiband ports, and ports labeled 10G are Ethernet ports. The boxes labeled "AC" are example power supply units. It is noted that in practice a rack mounted, enclosure may have different types and numbers o ports than those shown in Fig. 3. Also, as noted above, the scope of embodiments is not limited io SAS communication links, as other embodiments may use Ethernet, Iniiniband, IEEE 802. 1 1 , and/or Fiber Channel. The example of Figure 3 is consistent with the example of Figure 2 in that the ASN is built using modified RAID controller hardware. However, the scope of embodiments is not limited to using RAID controller hardware in the ASNs.

Figure 4 is a system diagram of a computing infrastructure 400 including a network, storage system according to aspects of the present disclosure. The embodiment of Figure 4 utilizes dense servers for ASNs and, therefore, utilizes one dense server per enclosure. As explained above, a dense server may include more processors and memory than the ASNs shown in Figure 1 and may utilize more space within a given enciosure. For instance, enciosure 41.0 includes a circuit board, including a dense server ASN 41 1 and two ES s 412, 414, Similarly, enclosure 420 includes a dense server ASN 421 and ESMs 422, 424. ASNs 411 and 42 i are configured as a MA pair via the cluster interconnect and are both connected to hosts {.not shown) via the customer network. Each enciosure 1.0, 420 also includes a storage drive array 4.13 and. 423, respectively. ASNs 1 1 and 421 run a storage warehousing appitcatioo. or other application on. top of an operating system, similar to thai described above with respect to the. ASNs of Figure 1. However, ASNs 4i i and 421 may handle snore concurrent processes doe io their increased processing power.

Enclosure 430 includes two block controllers 431 and 432 and a storage drive array 433 , Controllers 431 and 432 o f Figure 4 operate substantially similarly to controllers 2 ! and 122 of Figure 1 by providing low-level virtual storage control to storage drives 413, 423, and 433.

AS 41 1 has redundant communication links 441 and 442 to each of block controllers 43 ! and 432, Similarly. A.SN 421 has redundant communication links 443, 444 to each of block controllers 431 and 432, Such communication links may be substantiall similar to links 141 and 142 of Figure i .

Each ESM 412, 4:14, 422, 424 has a respective communication link 445-448 to a particular block controller 431 , 432, as shown. Communication links 445-448 are

substantially similar to the communicat ion links 143, 144 of Figure S. Controller 43 1. communicates with, and controls, storage arrays 4 S.3 and 423 via respective ESMs 412 and 422, Similarly, block controller 432 communicates with, and controls, storage arrays 413 and 423 via respective ESMs 414 and 424,

Figure 5 is an illustration of two example ASNs 520 and 530, adapted, according to the embodiments herein. Each ASN 520, .530 is included in its own circuit board within its own enclosure and is in communication with midplane 550 through midplane cards 505, 505. ASNs 520 and 530 are provided in contrast to the circuit board of Figure 2, which includes RAID controller hardware. AS 520 includes CPU 502, and ASN 530 includes CPU 503, though the scope of embodiments may include any number of CPUs in each of the ASNs, Figure 5 illustrates one example configuration io accommodate ESMs 524 and 534 in a same enclosure with respective ASNs 520 and 530,

Each ASN 520. 530 includes its respective ESMs 24 and 534 as PCS.e cards, though various embodiments may implement the hardware of ESMs 524 and 534 in any appropriate manner. ESM 524 communicates with block controllers (not shown) and passes that communication to midplane card 504 and to storage array 513 Similarly, ESM 534 communicates ith block controllers (not shown) and passes that communication to midplane card 505 and on to storage array 513. Each ESM 524 and 534 has a respective expander 5 ! 2, 514 to provide communication links to the controllers. In some instances, PCI is used for providing power only, while the data connections between a server and an ESM are provided by technologies other than PCI , in one exam le, both ASNs 520, 530 may foe implemented side~hy~Sfde in 21; so thai the total rack unit space occupied thereby is 2U, though the scope of embodiment includes any sizes for enclosures AS s 520, 530.

With respect to the embodiment of Figure 4, each A.SN 41 1, 421. may be

implemented similarly to AS 520 hot with at least one more CPU and two ESMs in each ASN . With respect to the embodiments of Figures 1 , 6, and 7, each ASN enclosure there in (e .g., enclosure 1 10.) may he implemented using the system shown in Figure 5 in which the ASNs 520, 530 are in a HA pair and may foe optionally cross-coupled.

Figure 6 is a system diagram of a computing infrastructure 600 including a network storage system according to aspects of the present disclosure. Figure 6 includes many of the same features shown, in Figure 1 , and like features are shown with like reference numbers.

System 600 includes enclosure 1 1 that has ASNs configured as a HA pair. Block, controller il l. communicates with, and controls, storage array 1.03 via

communication link 623 and the ESM of circuit board 101. Similarly, block controller 1.22 communicates with, and controls, storage array 103 via communication link 624 and the ESM of circuit board 102,

System 600 also includes enclosure 610. Enclosure 610 has ESMs 6.1 1 and. 61 2 as well as storage array 61.3. Each of ESMs 61 1 and 612 has a respect ive expander that provides communication ports to provide communication with block controllers 12 1 and 122 via communication links 621 , 622.

Block controller 121 common scales with, and controls, storage array 1.3 via communication link 621 and ESM 61 1. Block controller 122 communicates with, and controls, storage array 613 v ia communication Sink 622 and ESM 612. The embodiment of Figure 6 illustrates that additional storage arrays (e.g., array 613) may be added using expanders as long as the block controllers of the system have adequate communication ports and processing power. Communication links 621 -624 may include any appropriate type of communication link, including but not iiniiied to SAS, Fibre Channel, Infrniband, Ethernet IEEE 802.1 1 , and the like.

Figure 7 is a system diagram of a computing infrastructure 700 including a network storage system according to aspects of the present disclosure. Figure 7 includes many of the same features shown i Figures 1 and 6, and like features are shown with like reference numbers. The ASNs of e nclosures S. 1.0 and 610 are connected to the customer network, as shown, and io each other via the cluster interconnect. In addition io the concepts shown in Figure 6, Figure 7 illustrates that at least tour ASNs can be accommodated in two different enclosures and that storag drives in those enclosures can be communicatively coupled to controllers via ESMs with, the ASNs. Put another way, the ESMs of Figure 7 provide communication ports so that storage arrays 103 and 613 can be included in the same enclosures as ASNs ail while using external controllers.

The embodiment shown in Figure 7 is similar to the embodiment shown in Figure 6, hot the embodiment of Figure 7 adds ASNs to enclosure 1.0. In. fact, in Figure 7, the components and functionality within enclosure 610 are substantially similar to those within enclosure ! 10. Thus, ESM 61 1 is included in circuit board 70S., and ESM 612 is included in circuit board 702 Block controllers 121 , 121 include redundant cross-coupling to the ASNs of circuit boards 701 and 70 via communication links 741 and 742. Once again, block, controllers 121 and 122 communicate with, and control, storage array 613 through respective ESMs 61 1 and 6.12.

In. an exemplary embodiment, the various enclosures discussed above may be installed in a standard rack. Figure 8 is art ii lustration of example rack 801, adapted according to one embodiment. n the example of Figure 8, the system 700 (Figure 7} has been installed in the rack 801 , and each of enclosures .1 .10, 120, 610 is embodied as 2U unit, thereby occupying 6U total space.

Figure 8 does not show other units in rack 8 1 for simplicity of illustration, but it is understood that the scope of^" embodiments includes racks with any number of enclosures therein. It is also understood that arty of the embodiments of Figures I , 3, 4, 6, and 7 may^¬ be configured as standard rack-mounted enclosure and installed in a rack, such a rack i O. However, the scope of embodiments is not limited to standard rack-mounted computing systems, as any appropriate configuration of enclosures (even, outside of racks) may be adopted.

Figure 9 is a block diagram of a method 900 of installing a network storage system according to aspects of the present disclosure, it is understood that addit ional steps can be provided before, during, and after the method 900, and some of the steps described can be replaced or eliminated for other embodiments o f the method 900.

Block 902 includes installing a first computing device i a housing with a plurality of slots. An example of a suitable housing includes a EiA-31 rack with, slots that are each an integer number of units tall. The first comput ing device is placed in a first slot of the housing, and the first computing device includes a first enclosure with an. application server, a communication port expander, and an array of storage drives in communication with the expander, in one example, the enclosures of method 900 are standard enclosures that are each an integer number of rack units tali.

Block 904 includes installing a second computing device in the housing by placing ihe second computing device in a second slot. Further, the second computing device includes a second enclosure with a storage drive controller, in other words, the storage drive controller and the storage drive are included in separate enclosures.

Block 906 includes operably coupling the storage drive controller to the array of storage drives through the communication port expander and operably coupling the application server to the storage drive controller. The coupling of block 906 may include installing cables and, additionally or alternatively, setting up wireless communication

1 inks.

The scope of embodiments is not limited to E1A-3.10 standard rack housings and enclosures Rather, various embodiments may include any appropriate housing or enclosure, whether standard or otherwise. Furthermore, the communication links among the various components may include an appropriate hardware interface and protocol, Various communication links that may be used, with the embodiments herein, include, hut are not limited to, Ethernet, AS, infiniband. Fiber Channel, IEEE 802. S. 1, and the like.

Figure i 1 is block diagram of a method 1 1 0 of providing virtual storage according to aspects of the present disclosure. Method 1 100 is from the perspective of the hardware and software within an. enclosure, such as enclosure 110 (Figure 1). t is understood that additional steps can be provided before, during, and after the method 1100, and some of ihe steps described can be replaced or eliminated lor other embodiments of the method ί 100.

Block 1 102 includes interacting with a client computer to receive write requests for a virtual storage volume, i this example, the virtual storage volume is implemented on an array of storage drives within a first enclosure thai includes the application server node.

As described above, an ASN provides higher-level storage functionality, such, as by implementing a map reduce system or other system. The ASN interfaces with the client computer to receive write requests and to return responses to read requests.

Further, as described above, the lower-level storage functionality, such as RAID control, is provided by the storage controller. Thus, in block 1. 1.04, the ASN passes the write request to the storage controller. Further, in this example, the storage controller is pro ided in an enclosure that is separate from the enclosure that includes the ASN and the storage drives.

In block. .1 .106, physi al storage drives of the array receive control signals from the storage controller to effect the write request . The control signals are received via a communication port expander l cated in the enclosure with the ASN and the storage arra .

Method 100 is offered to illustrate a data ath that exists in various systems according to one or more embodiments. Specifically, the physical storage drives of the array are included in an enclosure that is separate from the enclosure that includes the storage controller. Therefore, write requests received at the ASN during interaction with the client are passed on to the controller in the second enclosure, and. the controller then provides low-level storage control, signals to the storage drives in the first enclosure.

Block 1 108 illustrates thai read requests are effected using the same data path. There is no requirement that write requests are performed be tore read requests; rather, write and read requests are performed in any appropriate order.

Figure 12 is a biock diagram of a method 1200 of providing virtual storage according to aspects of the present disclosure. Method 1200 is from the perspecti ve of the hardware and software of the storage controller, such as in enclosure 1 20 (Figure i ). .it is understood that additional steps can be provided before, during, and after the method 1200, and some of the steps described can be replaced or eliminated for other embodiments o the method 1200.

At block 1202, the storage controller receives a write request from an ASN . In this example., the ASN performs data warehousing functionality, and the write request is in accordance w ith such functionality.

The write request is received via a communication port expander in a first enclosure with, the ASN and is received by a storage controller in a second enclosure separate from the first enclosure. At biock 1204, the block controller sends control signals from the controller to a storage array within the first enclosure to write data to a virtual volume implemented in the storage array. Such control signals are a part of the low-level storage functionality of^" the storage controller that provides, for example, RAID

functionality in the storage drive array.

At block 1206, the biock controller processes read requests in the same manner that the write request is processed in. blocks 1202 and 1 204. Method. 1200 illustrates a data path that includes a write or read request corning from a client and being received at an ASN, then passed to a storage controller in a different enclosure. The storage controller then communicates with the physical disk drives in the ASN enclosure via a

communication port expander to effect the read or write request

For both the methods 1 100 and .1200, many embodiments include storage drives in the controller's enclosure and in other enclosures as well as in the ASN enclosure. In such embodiments, the drives that are included i the same enclosure as the ASN are presented to the RAID controller via the port expander such that ihey become members of the total set of drives that are managed by the RAID controller. In other words, the RAID controller combines the drives in its enclosure with drives in an Expansion JBODs (EBOD) (e.g., 610 in figure , if applicable) and the drives in the ASN enclosure to create RAID volumes to be presented to the applicat ions running on the ASN.

The foregoing outlines features of^" several embodiments so thai those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should appreciate that they tnay readily use the present disclosure as a basts for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art should also realize that such equivalent constructions do not depart front the spirit and scope of the present disclosure, and that they may make var ious changes, substitutions, and alterations herein without departing from the spirit and scope of the present disclosure.

Claims

1. A s stem comprising:

a first array of^" storage drives in communication with a storage controller, the storage controller providing virtualixed storage with the first array of^" storage drives; an application server running a storage warehouse application and

communicatively coupled to a host device and the storage controller; and

a communication port expander providing expansion slots to a first enclosure, wherein the first array of storage drives, the application server, and the communication port expander are contained in the first enclosure separate from the controller, further wherein the controller controls the first array of storage drives and is in communication w ith, the first array of storage drives through the communication port expander.

2. The system of claim 1. wherein the communication port expander is provided by an Enclosure Service Module (ESM) ithin the first enclosure, the ESM providing power and temperature control to the application server.

3. The system of claim 2, wherein the application server and tire ESM are included on a same circuit hoard within, the enclosure, wherein the ESM comprises CPU resources separate from CPU resources of the application server,

4. The system of claim i, wherein the communication port expander is included on a peripheral interconnect card in communication with tire application server,

5. The system of claim i, wherein the expander comprises an SAS expander.

6. The system of claim 1. wherein the application server is part of a High availability (HA) pair of^" application servers.

7. The system of claim 1. wherein the storage controller is part of^" a High. Availability (MA) pair of storage controllers.

S. The system of claim 1. wherein the storage controller comprises a RAID controller.

9. The system of claim ! , wherein the first^" enclosure including the first array of storage drives, the communication port expander, and the application server comprises a 2U blade device, further wherein a second enclosure including a second array of storage drives and lire controller comprises a 2U blade device installed in a same rack with the first^" enclosure.

10. A method comprising:

installing a first computing device in a housing with a plurality of slots, the first computing device being placed in a first slot, where in the first computing device comprises a iirst enclosure with an application server, a communication port expander, and an array of storage drives in communication with the expander;

installing a second computing device in the housing, the second computing device being placed in a second slot, wherein, the second computing device comprises a second enclosure with, a storage drive controller; and

operahiy coupling the storage drive controller to the array of storage drives through the communication port expander and operahiy coupling the application server to the storage drive controlier,

11 . The method of claim 1.0, wherein the housing comprises a EIA-310 standard rack, and wherein the first and second computing devices comprise ElA-310 standard enclosures.

12. The method of claim 10. wherein operahiy coupling the storage drive controller to the array of storage drives comprising providing a Serial Attached SCSI. (SAS) communication link.

13. A system comprising,

an array of storage, drives in communication with a storage controlier, the storage controller providing virtuaiized storage wi th the array of storage drives:

an application server running a storage warehouse application and serving the storage controller; and

an Enclosure Service Module (ESM) providing expansion slots to the array of storage drives, wherein, the array of drives, the application server, and the ESM are contained in a same enclosure separate front the controller, further wherein the controlier controls the array of storage drives and is in communication with the array of^" storage drives through the ES .

14. The sy stem of claim 13 , wherein the application server and the ESM are included on a same circuit board within the enclosure, 15. The sy stem of claim 13 , wherein the ESM is included as a peripheral interconnect card in communication ^'with the applicat ion server.

1 . The system of claim 13, wherein the expander comprises an SAS expander.

17. The system of claim 13, wherein the application server is pari of a High availability (HA) pair of application servers. 18. The system of claim 13, wherein the storage controller is part of a High

Availability (HA) pair of storage controllers.

19. The system of claim 13, wherein ihe storage controller comprises a RAID controller

20. The system of claim ! 3, wherein the enclosure including the array of storage drives, the ESM, and the application server comprises a 2U blade device, further wherein another enclosure including ihe controller comprises a 2U blade device installed in a ame rack as the enclosure.

21 . A. method for providing virtual storage for a client, the method comprising: at an application server node, receiving write requests -from client computer for a virtual storage volume, the virtual storage volume being implemented on an array of storage drives accessible by a first computing system implementing the application server node, the storage drives being physically located with the first computing system;

passing a first write request from the application server node to a storage controller, the storage controller implemented by a second computing system physically separate from the first computing system; and

receiving drive access instructions from the storage controller to the array of storage drives to effect the write request, where the drive access instructions are received via a communication path between the first computing system and the seeond computing system,

22. The method of ciaim 21, wherein the storage controller comprises RAID controller hardware, and wherein the application server node comprises a CPU executing a data warehousing program. 3. The method of claim 21 wherein the communication path comprises a communication port expander that is included in an Enclosure Service Module (ESM). and wherein the communication port expander comprises a Serial Attached SCSI (SAS) expander,

24. A method for providing virtual storage in a storage network, the method comprising:

receiving a write request from an application server node, the write request being in accordance w ith data warehousing functionality of the application server node, the write request received via a communication port expander in a first enclosure with the application server node, the write request being received by a storage controller in a second enclosure separate from the first enclosure; and

in response io the write request, sending control signals from the controller to a storage array within the first enclosure to write data to a virtual volume implemented in the storage array.

25. The method of claim 24, wherein the storage controller comprises a RAID controller,

26. The method of claim 24, further comprising:

receiving a read request from the application server at the storage controller; and in response io the read request, sending control signals from the controller to the storage array to read data to the virtual volume .