WO2010097132A1

WO2010097132A1 - An apparatus and method for expanding a storage controller

Info

Publication number: WO2010097132A1
Application number: PCT/EP2009/067180
Authority: WO
Inventors: Ian David Judd; William James Scales; Thomas William Rickard
Original assignee: International Business Machines Corporation
Priority date: 2009-02-25
Filing date: 2009-12-15
Publication date: 2010-09-02
Also published as: TW201036268A

Abstract

A modified electronics package and a method of manufacture of a modified electronics package with dimensions constrained by standards, comprising an envelope and a circuit structure. The circuit structure provides a free space within the dimensions, which contains a connector for connecting a further circuit structure with either the circuit structure or with a device outside the envelope. In further embodiments of the invention, the modified electronics package meets Storage Bridge Bay Working Group (SBB) specifications. It may also comprise a further circuit structure, which may be a PCIe adapter card or a data storage device. Further embodiments include an apparatus comprising a modified electronics package in combination with controller modules or further modified electronics packages, along with a rack mounted storage system with such apparatuses.

Description

AN APPARATUS AND METHOD FOR EXPANDING A STORAGE CONTROLLER

Technical Field

The invention relates to an apparatus and method for expanding a storage controller. In particular the invention relates to an apparatus and method for expanding a standards- constrained controller.

Background of the Invention

A typical data storage system enclosure is modular, having bays at the front for receiving disk modules, and bays at the rear for receiving power supply unit (PSU) modules and various electronics modules providing functions to control the system, for example input/output connections to the enclosure, Fibre Channel (FC) or internet Small Computer System Interface (iSCSI) Redundant Array of Inexpensive Disks (RAID) functionality, enclosure management services, etc. Many layouts and configurations are possible.

A typical design of such an enclosure comprises a plurality of modules. First, bolted into the enclosure chassis is a mid-plane. Onto this mid-plane arc plugged all other modules. Modules plugged into the mid-plane are concurrently maintainable, that is, they may be unplugged, removed and replaced without interrupting system operation.

One particular type of electronics module is a controller module, used to provide functions to control the data storage system. A typical data storage system has a first enclosure which contains two storage controllers and additional expansion enclosures. The first and additional enclosures are electrically connectable with each other through industry standard connections, such as Serial Attached SCSI (SAS) and Peripheral Component Interconnect Express (PCIe). PCIe is a computer expansion card standard which provides a wide range of functionality. Two storage controller modules are provided in a data storage system enclosure to provide redundancy in the event that one or the other fails. The expansion enclosures are often called Switched Bunch of Drives (SBODs). As typical enclosure modules are manufactured by a number of independent hardware vendors and system \cndors, industry standards

been developed in order to standardise the size of such modules, and to provide common interface specifications for the electronic interconnections One such standard is Storage Bridge Bay (SBB), which is emerging as a standard for low-end and midrange storage systems SBB defines the mechanical and electrical requirements of an electronics module (called a 'canister ) and an interface to the mid-plane in the enclosure

In addition to the advantages in common design specifications for industry standard designs, there are also

capabilities of the modules For example, only a limited number of Storage Area Network (SAN) ports may be accommodated within the space a\ailable, therefore the controller function is limited This disclosure describes a flexible way of expanding the controller capabilities by utilizing space that is already available in the SBOD expansion enclosures

Summary of Invention

Viewed from a first aspect, the present invention preferably provides a modified electronics package having standards-constrained dimensions comprising an envelope, a circuit structure for interfacing, and a free space withm the dimensions The free space comprises a connector for connecting a further circuit structure for communication with the circuit structure or a device outside the envelope

Preferably, the present invention provides a modified electronics package wherein the modified electronics package is a Storage Bπdgc Bay electronics package

Preferably, the present invention provides a modified electronics package wherein the modified electronics package further comprises a further circuit structure Preferably, the further circuit structure is a PCIe adapter card Alternate ely, the further circuit structure is a data storage device Preferably, the present invention provides a modified electronics package wherein the further circuit structure is hot-pluggable or hot-swappablc

Viewed from a second aspect, the present invention provides a method of manufacture for a modified electronics package having standards-constrained dimensions comprising the steps of providing an envelope, forming a circuit structure for interfacing, wherein the circuit structure provides a free space within the dimensions, providing a connector in the free space for connecting a further circuit structure for communication with the circuit structure or a device outside the envelope

Preferably, the present invention provides a method of manufacture wherein the modified electronics package is a Storage Bridge Bay electronics package

Preferably, the steps further comprise providing a further circuit structure Preferably, the further circuit structure is a PCIc adapter card Alternatively, the further circuit structure is a data storage device Preferably, the further circuit structure is hot-pluggable or hot- swappable

Viewed from a third aspect, the present invention provides an apparatus comprising a modified electronics package of the first aspect Preferably the apparatus further comprises a second modified electronics package Alternatively the apparatus further comprises a controller

Viewed from a fourth aspect, the present invention provides a rack mounted data storage system comprising combinations of apparatuses of the third aspect

Advantageously, the present invention provides for a modified electronics package and method of manufacture of a modified electronics package operable to expand controller functionality in standards-constrained data storage systems Also, advantageously, the present invention also provides for standards-constrained apparatuses comprising controllers with expanded functionality Also, advantageously, the present invention provides for a standards-constrained rack mounted data storage system comprising combinations of apparatuses with expanded controller functionality

Brief Description of the Drawings

A preferred embodiment of the invention will now be described, by way of example only, and with reference to the following drawings:

Figure 1 depicts a diagram of a typical data storage system enclosure incorporating the present invention;

Figure 2 depicts the top view of a data storage system enclosure incorporating the present invention;

Figure 3 depicts the top view of a prior art controller module;

Figure 4 depicts a diagram of a prior art interface module;

Figure 5 depicts the top view of an enhanced interface module;

Figure 6 depicts a further top view of an enhanced interface module;

Figure 7 depicts an alternative configuration of an enhanced interface module;

Figure 8 depicts one configuration of the invention within a data storage system enclosure;

Figure 9 depicts a conventional SBB data storage system;

Figure 10 depicts how extra SAN ports may be added to a data storage system;

Figure 11 depicts how extra SAN ports may be added to a data storage system in another embodiment of the invention; Figure 12 is an exemplary schematic flow chart for a method of manufacture of an enhanced interface module.

Detailed Description of the Iirv entioii

Figure 1 is a block diagram depicting a typical data storage system enclosure 100 as viewed from the back, in which the present invention may be embodied The typical data storage system enclosure 100 may be mounted m a rack mount system The illustrated data storage system enclosure 100 comprises an outer cover 110 The rear face 120 has a plurality of slots 130, one of which is illustrated Various electronics modules 140, one of which is illustrated, may be inserted into a slot 130 Examples of electronics modules 140 comprises controller modules and power supply modules The backplate 145 of the electronics module 140 typically comprises a plurality of electronic connectors to provide connectivity to external devices Additionally a plurality of electronics modules 140 may be inserted into slots in the front face 150 Such electronics modules 140 arc not illustrated in this figure Typically the electronics modules 140 that fit into the front face comprise disk drive modules

Figure 2 depicts the top view of the data storage system enclosure 210 according to the prior art Two electronics modules 240 arc illustrated as being inserted into the rear face 220 In addition, one disk drive module 245 is illustrated as being inserted into the front face 250

The electronics modules and the disk drive modules arc electrically conncctablc to a circuit structure 270, which is also known as a mid-plane 270, through module interconnect pms 260 and mid-plane interconnect pms 265

Figure 3 depicts the top view of a controller module 300 according to the prior art It typically comprises a metal outer envelope 310 It w ill be clear to one of ordinary skill in the art that other materials, such as plastic may be used Within the envelope 310 is an electronics circuit structure comprising an electronics card 320 and circuitry The circuitry typically comprises a central processing unit (CPU) complex 330 with control circuitry, electronically connectable through a PCIe switch complex 340 to a PCIe adapter card 350

Arrowed lines in this and subsequent figures illustrate exemplary electrical connections 305 between components One such exemplary connection 305 is labelled An example of the functionality provided by a PCTc adapter card 350 is that of providing FC connection capability External connections are provided by a mid-plane connector 360, and through connectors 365, 370 on the backplatc 380 of the controller module 300 Typical types of connector 365, 370 arc SAS, FC and iSCSI connectors

Figure 4 depicts the top view of an interface module 400 according to the prior art It typically comprises a metal outer envelope 410 The dimensions of the envelope 410 arc constrained by industry standards Within the envelope 410 is an electronics circuit structure comprising an electronics card 420 and circuitry

A data storage system comprises a plurality of data storage system enclosures 110 A first data storage system enclosure 110 comprises two controller modules 300 Subsequent data storage system enclosures are provided as expansion enclosures, and are often called Switched Bunch of Drives (SBODs) An SBOD normally contains two interface modules

400 instead of the controller modules 310 These interface modules 400 provide, for example, redundant paths to each disk drive module 245 and enclosure management services An interface module 400 often requires only a single logic chip 430 (for example, a Fibre Channel Arbitrated Loop (FC-AL) switch or a SAS expander), plus a small amount of external memory 435 (for example, random access memory (RAM) and Flash memory)

Thus, an interface module typically has spare space, power and cooling External connections are provided by a mid-plane connector 460, and through connectors 465 on the backplate 480 of the interface module 400

Figure 5 depicts the top view of an enhanced interface module 500 It typically comprises a metal outer envelope 410, 510 The dimensions of the envelope 410, 510 are constrained by industry standards Withm the envelope 510 is an electronics circuit structure comprising an enhanced electronics card 520 and circuitry The enhanced interface module 500 comprises a single logic chip 430, 530 and external memory 435, 535 External connections are provided by a mid-plane connector 540, through connectors 545, 570 on the backplate 480, 580 of the enhanced interface module 500, and through a connector 550 An exemplary electrical PCIe connection 575 connects a connector 570 to the connector 550 The enhanced electronics card 520 is so shaped as to provide a free space 560 within the en\ elope 510 so as to accommodate a further electronic circuit structure (not shown)

Figure 6 depicts a further top

of an enhanced interface module 500, 600 of figure 5 with a further electronics card 690 inserted in the free space 560, 660 The further electronics card 690 is connectable to the enhanced electronics card 620 or to a device outside of the envelope through connector 650 The further electronic circuit structure 690 also has external connections through connectors 655 on the backplatc 480, 580, 680 of the enhanced interface module 500, 600 An exemplary electrical PCIe connection 675 connects a connector 670 to the connector 650 An example of a further circuit structure is an industry standard size PCIe adapter card, or it may be a customised card

Figure 7 depicts an alternative configuration of an enhanced interface module 700 It comprises a metal outer envelope 410, 510, 710 Within the envelope 710 is an electronics circuit structure comprising an enhanced electronics card 720 and circuitry The enhanced interface module 700 comprises a single logic chip 430, 530, 730 and external memory 435, 535, 735 The enhanced electronics card 720 is so shaped as to pro\ide free space 760 within the envelope 710 so as to accommodate a further circuit structure 790 External connections arc provided by a mid-plane connector 740, 770 and through connectors 745 on the backplate 780 of the enhanced interface module 700 The enhanced electronics card is connectable to the further circuit structure through a further mid-plane connector 795 The further circuit structure 790 is connectable to an external connector 770 through an exemplary electrical PCIe connection 775 and 785 that are connected through mid-plane connector 740 and further mid-plane connector 795 The further circuit structure also has external connections through connectors 755 on the backplatc 780 of the enhanced interface module 700

It will be appreciated by the person skilled in the art, that a plurality of shapes and configurations of enhanced electronics card 520, 620, 720, including positioning of connectors, and further circuit structure 690, 790 are possible Figure 8 depicts one configuration of the invention within a data storage system enclosure 100 comprising one controller module 300, 805 and one enhanced interface module 600, 895. The outer covers 110, 410, 510, 610, 710 are not shown. The controller module 300 comprises an electronics card 320, 820 and circuitry. The circuitry comprises a CPU/SAS Controller/SAS expander 810, electronically connectable through a PCIe switch complex

830 to an adapter card 850. External connections are provided by a mid-plane connector 840, by SAS connectors 865, and by a set of FC connectors 875. The enhanced interface module 895 comprises an enhanced electronics card 825. The enhanced electronics card 825 comprises a SAS expander module 815. The enhanced electronics card 825 is connectable to a PCIe adapter card 890 through PCIe connector 855. External connections of the enhanced electronics card 825 are provided by a mid-plane connector 840, and by SAS connectors 865. The PCTe adapter card 890 has external connections through a set of FC connectors 875, and by the mid-plane connector 840. Electrical connections between the modules are illustrated by exemplary arrowed lines. In particular a peer PCIe bus 835 exists between the PCIc switch complex 830 to the PCIc adapter card 890 in the enhanced interface module 895 through the mid-plane connector 840, the SBB mid-plane bus 845 and the PCIe connector 855. This figure depicts how the FC capabilities provided by the controller module 300, 805 are expanded by the provision of FC capabilities provided on the enhanced interface module by PCIc adapter card 890.

Figure 9 depicts a conventional SBB data storage system 900 comprising one data storage system enclosure 910 with dual controller modules 300, 930, 935, which communicate over a PCIe bus 920 in the mid-plane. This data storage system enclosure 910 has limited SAN ports.

Figure 10 depicts how extra SAN ports may be added to a data storage system 1000. In a preferred embodiment of the invention, the first data storage system enclosure 1010 contains dual controller modules 1030, 1040. The two controller modules 1130, 1140 may communicate through a connection 1020, for example through the SAN, or through the FC- AL or SAS drive fabric. Alternatively they may be connected via an additional PCIe cable

(not shown). Each controller module 1030, 1040 is connected to an enhanced interface module 1035, 1045 in the second data storage system enclosure 1015 by one or more additional PCIc cables 1050

Figure 1 1 depicts how extra SAN ports may be added to a data storage system 1 100 in another embodiment of the invention A first data storage system enclosure 1110 comprises a controller module 1130 connected through a PCIe bus 1120 to an enhanced interface module 1135 that provides additional SAN ports A second data storage system enclosure 1 1 15 is added to the data storage system The second data storage system enclosure 1 1 15 also comprises a second controller module 1140 connected through a PCIe bus 1120 to a second enhanced interface module 1145 that also provides additional SAN ports The two controller modules 1130, 1140 may communicate via an additional PCIe cable 1150 between the data storage system enclosures 1110, 1115 The two controller modules 1130, 1140 may communicate for example for cache mirroring

Other combinations of module within a data storage system enclosure will be apparent to storage system designers

In a preferred embodiment a data storage system may be assembled comprising a plurality of modified data storage system enclosures as described hcrcm, preferably within a rack mounted system

Figure 12 is an exemplary schematic flow chart for a method of manufacture of an enhanced interface module as described herein The schematic flow chart diagram that follows is generally set forth as a logical flow chart diagram As such, the depicted order and labelled steps arc indicative of one embodiment of the presented method Other steps and methods may be conceived that are equivalent in function, logic, or effect of one or more steps or portions thereof, of the illustrated method Additionally the format and symbols employs are provided to explain the logical steps of the method and are understood not to limit the scope of the method Although various arrow types and line types may be employed in the flow chart diagrams that are understood not to limit the scope of the corresponding method For instance an arrow may indicate a waiting or monitoring period of specified duration between enumerated steps of the depicted method Additionally the order in which a particular method occurs may or may not strictly adhere to the order of the corresponding steps shovvn. After the start 1210, the method of manufacture comprises the steps of: providing 1220 an envelope 410, 510, 610, 710; forming 1230 an enhanced electronics card 520, 620, 720, 825; providing 1240 a connector 550, 650, 740, 855 in the free space 560, 760; and, optionally, providing 1250 a further circuit structure 690, 790, 890; before the end of the method 1260.

An exemplary embodiment of the invention will satisfy the dimensions of the SBB standard. An SBB canister will accommodate a full-height, half-length PCIc adapter card. The adapter may be, for example, a standard or a customised FC or iSCSI Host Bus Adapter (HBA) to provide additional storage area network (SAN) ports. Alternatively, the further circuit structure may be a data storage device provided by a Solid-State Disk (SSD), or by a Flash memory. In a preferred embodiment the further circuit structure 690, 790 may be hot- pluggable or hot-swappable from outside of the enhanced interface module 600, 700 by a user. Other applications will be apparent to storage system designers.

For the avoidance of doubt, the term "comprising", as used herein throughout the description and claims is not to be construed as meaning "consisting only of.

Claims

1 A modified electronics package having standards-constrained dimensions comprising an envelope, a circuit structure for interfacing wherein said circuit structure being adapted to provide a free space within said dimensions, and said free space having a connector for connecting a further circuit structure for commumcdtion with said circuit structure or a device outside the envelope

2 The modified electronics package of claim 1, wherein the electronics package is a Storage Bridge Bay electronics package

1 The modified electronics package of cither of claims 1 or 2, wherein the connector is connectdble to said circuit structure

4 The modified electronics package of either of claims 1 or 2, wherein the connector is conncctablc to a device outside the envelope

5 The modified electronics package of any of claims 1 to 4 further comprising a further circuit structure

6 The modified electronics package of claim 5, wherein the further circuit structure is a PCIc adapter card

7 The modified electronics package of claim 5, wherein the further circuit structure is a data storage device

8 The modified electronics package of any of claims 5 to 7, wherein the further circuit structure is hot-pluggable or hot-swappable

9. A method of manufacture for a modified electronics package having standards- constrained dimensions comprising the steps of: providing an envelope; forming a circuit structure for interfacing: wherein said circuit structure being adapted to provide a free space within said dimensions; and providing a connector in said free space for connecting a further circuit structure for communication with said circuit structure or a device outside the envelope.

10. The method of manufacture of claim 9, wherein the modified electronics package is a

Storage Bridge Bay electronics package.

11. The method of manufacture of either of claims 9 or 10, wherein the connector is connectable to said circuit structure.

12. The method of manufacture of either of claims 9 or 10, wherein the connector is connectable to a device outside the envelope.

13. The method of manufacture of any of claims 9 to 12, wherein the steps further comprise providing a further circuit structure.

14. The method of manufacture of claim 13, wherein the further circuit structure is a PCIe adapter card.

15. The method of manufacture of claim 13, wherein the further circuit structure is a data storage device.

16. The method of manufacture of any of claims 13 to 15, wherein the further circuit structure is hot-pluggable or hot-swappable.

17. An apparatus comprising a modified electronics package as claimed in any of claims 1 to 8.

18. The apparatus of claim 17 wherein the apparatus further comprises a second modified electronics package.

19. The apparatus of claim 17 wherein the apparatus further comprises a controller module.

20. A rack mounted data storage system comprising a plurality of combinations of apparatuses as claimed in any of claims 17 to 19.