JP7315753B2 - storage device - Google Patents

Description

The present disclosure relates to storage devices.

A storage device includes a storage controller that controls data transfer between a host computer connected to the storage device and a storage medium within the storage device. The storage controller includes a front-end interface for connecting host computers, a back-end interface for connecting multiple drives as storage media, a processor for controlling the storage device, memory connected to the processor, and the like.

Also, PCI Express (hereinafter referred to as PCIe) is known as one of communication network standards for connecting processors, front-end interfaces, back-end interfaces, and the like. Furthermore, in order to match the usage scale of users who use the storage device, the storage device often includes an interconnect for interconnecting a plurality of storage devices.

Availability is one of the most important factors in storage devices used in mission-critical operations. To provide this availability, the storage device uses, for example, a dual-controller architecture in which the storage controllers described above are configured redundantly.

In the dual controller architecture, interfaces for connecting controllers include PCIe and Infiniband, for example. When one of the dual controllers becomes unusable for some reason, the normal controller on the other side detects the abnormality, and the availability is guaranteed by continuing I/O from the host computer until the unusable controller recovers.

In recent years, due to the increase in single drive capacity and progress in virtualization technology, data used by multiple host computers or applications are often mixed on a single drive. Therefore, for example, when an abnormality is detected in a single drive, it is necessary to notify both controllers of the abnormality. This is because both controllers of the storage apparatus operate in cooperation with each other, and by notifying both controllers of the occurrence of an abnormality, it is necessary to quickly and consistently perform appropriate processing.

On the other hand, the price of flash memory has been reduced and the use of flash memory has progressed, and the use of SSDs (Solid State Drives), in which flash memories are mounted in drives, in storage apparatuses is progressing. Along with the popularization of such SSDs, in recent years, NVM Express (Non-Volatile Memory Express) has been formulated as a new communication protocol for one of the purposes of eliminating the above-described increase in processing time. NVM Express is located in the application layer as a protocol layer, and uses PCIe as the physical layer.

By the way, a PCI Express domain has a structure in which switches, bridges, and endpoint devices are connected in a tree with a device called a root complex at the top. A root complex is the most rooted device in a PCI Express system.

Bridges are often used to perform protocol conversion to connect legacy devices, such as PCI or PCI-X endpoints, to PCI Express. A switch is used to increase the number of PCI Express ports. An endpoint is, for example, a device that performs I/O, such as a hard disk or SSD. According to PCI Express specifications, an endpoint can only belong to one PCI Express domain.

High performance is one of the most important factors in storage devices used in mission-critical operations. In order to provide the aforementioned availability and high performance, the storage device adopts a configuration that allows access to arbitrary drives from the storage controller. The storage controller includes multiple processors with root complexes and root ports. Furthermore, depending on the scale of the device, the storage device includes multiple storage controllers. However, according to the PCI Express specifications, an end point, an SSD supporting NVMe, can only belong to one PCI Express system, that is, under one processor. In other words, it is not possible to adopt a configuration in which any drive can be accessed from the storage controller.

A non-transparent bridge (hereinafter referred to as NTB) is a means for solving this problem. A storage device using NTB is disclosed, for example, in Patent Document 1. Patent Document 1 discloses a technology that enables efficient communication between storage control units via a backend switch that connects between the storage control unit and the storage device. A virtual switch (VS) obtained by logically dividing a switch is provided for each root port, and the NTB of the backend switch connects between them. Drives belong to either VS according to the PCI Express specification.

Here, focusing on availability, when an error is detected in the PCI port on the drive side, the drive normally notifies the root port to which the drive belongs. The drive issues a memory write command to notify a specified area of the processor main memory. At that time, according to PCI Express specifications, there is one destination for notification from the drive.

JP 2018-190368 A

By using the NTB technology and the technology disclosed in Patent Document 1, it becomes possible to configure a storage controller to access any drive. However, Patent Document 1 does not disclose the case of three or more storage controllers.

On the other hand, the virtual switch in the backend switch disclosed in Patent Document 1 is required for each root complex (for example, processor) according to the PCI Express specifications. In Patent Document 1, by applying NTB between one or more backend switches, it becomes possible to access any drive from the storage controller. In Patent Document 1, a drive is associated with one virtual switch included in the backend switch.

These include the following issues. That is, when the number of processors increases, the number of virtual switches also increases, so when a processor accesses an arbitrary drive, the number of virtual switches it passes through differs between drives. In other words, when a processor accesses an arbitrary drive, performance variations occur due to performance non-uniformity that accompanies virtual switch crossing. As a result, the performance of the entire system may be degraded due to low-performance drive access. In addition, since it passes through a number of virtual switches, the design of address translation becomes complicated when designing the storage device.

On the other hand, if the PCI port is malfunctioning, the processor cannot communicate with the drive, so a timeout occurs for the accessing processor. In this case, since it takes time to detect an abnormality, it is better to quickly notify the processor of the error, stop the drive access, and perform fault processing. As mentioned above, when multiple processors exist, it is important to notify all processors that can access the drive of the detection of an abnormality. However, the PCI Express specification can only notify a single processor.

Therefore, a storage device and an information processing method capable of performing stable processing performance and appropriate fault handling are desired.

A storage apparatus according to one aspect of the present disclosure includes a plurality of controllers, a plurality of storage drives, a plurality of controller-side ports connected to each of the plurality of controllers, and a plurality of drive-side ports connected to each of the plurality of storage drives. The switch device performs address translation between the plurality of controller-side ports and the plurality of drive-side ports.

According to one aspect of the present disclosure, it is possible to perform stable processing performance and appropriate fault handling in a storage device.

3 is a diagram showing the configuration of nodes included in the storage device; FIG. 3 is a diagram showing a configuration example of a drive box included in the storage device; FIG. It is a figure which shows an example of a storage apparatus. 4 is a flow chart showing a procedure for notifying a processor of an abnormality that has occurred on the drive side in a storage device. 4 is a flow chart showing a processing procedure of a processor for an abnormality notification received from a drive box; FIG. 10 is a diagram showing a message transmission destination management table in the storage device; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, various information may be described using expressions such as “management table”, but the various information may be expressed in a data structure other than a table. Also, the "management table" can be called "management information" to indicate that it does not depend on the data structure.

Also, the processing may be described with the subject of "program". The program is executed by a processor, for example, a CPU (Central Processing Unit), and performs prescribed processing. Note that the subject of the processing may be the processor because the processing is performed using storage resources (eg, memory) and communication interface devices (eg, communication port) as appropriate. The processor may have dedicated hardware in addition to the CPU. A computer program may be installed on each computer from a program source. The program source may be provided by, for example, a program distribution server or storage media.

Also, each element can be identified by a number or the like, but other types of identification information such as a name may be used as long as it is identifiable information. Although the same reference numerals are given to the same parts in the drawings and explanations of the present invention, the present invention is not limited to this embodiment, and all applications consistent with the idea of the present invention are included in the technical scope of the present invention. Also, unless otherwise specified, each component may be singular or plural.

A storage device and an information processing method, for example, a storage device and an information processing method suitable for application to a storage device equipped with SSDs (Solid State Drives) as storage drives (hereinafter simply referred to as drives) will be disclosed below. More specifically, in the configuration example of the storage apparatus disclosed below, a PCIe (PCI Express) switch (physical switch) in a drive box containing storage drives performs protocol conversion in communication between multiple storage controllers and multiple storage drives. As a result, stable processing performance and appropriate fault handling can be quickly implemented. The drive, for example, supports NVM Express (NVMe). A physical switch may be used instead of the virtual switch.

Details of the present embodiment will be described below with reference to the drawings. Details of the PCIe standard and the NVMe standard are publicly known, so detailed description thereof will be omitted, and only items necessary for describing the present embodiment will be described as appropriate.

FIG. 1 is a diagram showing the configuration of nodes included in a storage device in this embodiment. The storage device includes one or more drives (such as SSD) (not shown in FIG. 1) and one or more nodes. A node includes one or more storage controllers (also simply called controllers) that control data input/output to/from drives.

The node 100 of this embodiment emphasizes availability and includes two controllers 105A and 105B to continuously provide services. The two controllers 105A, 105B have identical configuration and functionality in this example. Storage controller 105A includes processor (CPU) 110A, front-end interface 115A, back-end interface 120A, interconnect interface 125A, and memory 130A. Storage controller 105B includes processor 110B, front-end interface 115B, back-end interface 120B, interconnect interface 125B, and memory 130B.

In the following, storage controller 105, processor 110, and front-end interface 115 represent any storage controller, any processor, and any front-end interface, respectively. Backend interface 120, interconnect interface 125, and memory 130 represent any backend interface, any interconnect interface, and any memory, respectively.

Processor 110 controls each component within storage controller 105 , such as front-end interface 115 . For example, processor 110 sets data transfer parameters for front-end interface 115 . The processor 110 monitors failures within the node 100 and, when detecting failures, executes processing to deal with the failures.

The front-end interface 115 includes an interface connected to a host computer (not shown), and performs predetermined protocol processing on packets received from the host computer and packets to be sent to the host computer. A host computer is a computer device having information processing resources such as a processor and memory, and is composed of, for example, an open system server or a mainframe computer. The host computer sends write commands and read commands to the storage device via the network.

For example, the front-end interface 115 acquires information such as the storage location of read data or write data in a drive, which will be described later, and the size of the packet, from the packet received from the host computer. The front-end interface 115 also identifies commands contained in the packets and converts the received packets into a form used inside the storage controller 105 .

The front-end interface 115 generates a packet that can be transmitted to the host computer by adding, to the read data, identification data of the host computer that is the transmission destination, control data related to the command to the host computer, etc., based on the communication protocol between the storage controller 105 and the host computer.

The backend interface 120 is an interface that allows the processor 110 to perform data communication with a drive box and drives in the drive box, which will be described later. If the protocol outside the backend interface 120 and the protocol on the processor 114 side are different, the backend interface 120 performs protocol conversion. For example, the protocol on the processor 114 side is PCIe, and the protocol outside the backend interface 120 is PCIe, Fiber Channel, or Ethernet.

The backend interface 120 includes a backend port (downstream port) for connecting the node 100 and the drive box. In the example configuration of FIG. 1, one backend interface 120A in storage controller 105A includes backend ports 122A-1 and 122A-2, and another backend interface 120A includes backend ports 122A-3 and 122A-4. One backend interface 120B in storage controller 105B includes backend ports 122B-1 and 122B-2, and another backend interface 120B includes backend ports 122B-3 and 122B-4.

A backend port may be part of a switch within a backend interface. Backend interface 120 may be part of the processor. In this case, a backend port for connecting with the drive box may be built into the processor.

The interconnect interface 125 is a communication interface between a plurality of (two in the configuration example of FIG. 1) storage controllers 105 . For communication between the storage controllers 105, for example, Infiniband using an interface card, Ethernet, or PCI Express to which NTB (Non Transparent Bridge) is applied may be used. Any interface capable of communicating data between processors 110 or memory 130 can be used.

FIG. 2 is a diagram showing a configuration example of a drive box 200 included in the storage device.

Drive box 200 includes PCIe switches 205A and 205B (hereinafter any PCIe switch is referred to as PCIe switch 205) and storage drives 230, 240, 250 and 260. FIG.

PCIe switch 205A includes root complex connection virtual switches 210A, 210B, address translators 225A, 225B, drive connection virtual switch 220A, and switch control module 215A. PCIe switch 205B includes root complex connection virtual switches 210C, 210D, address translators 225C, 225D, drive connection virtual switch 220B, and switch control module 215B. The PCIe switches 205A and 205B each include multiple controller-side ports (virtual ports or physical ports, respectively) to which storage controllers are connected, and multiple drive-side ports (virtual ports or physical ports, respectively) to which storage drives are connected.

In the following, PCIe switch 205, root complex connection virtual switch (controller-side switch) 210, address translator 225, drive connection virtual switch (drive-side switch) 220, and switch control module 215 represent any PCIe switch, any root complex connection virtual switch, any address translator, any drive connection virtual switch, and any switch control module, respectively.

PCIe switch 205 includes, for example, a processor and memory. The processor functions as a root complex connection virtual switch, an address translator, a drive connection virtual switch, and a switch control module by operating according to programs stored in the memory. The PCIe switch 205 can include logic circuits that perform predetermined operations in addition to processors. A virtual switch improves the scalability of a storage device.

Drive box 200 further includes connection paths 235A, 235B, 245A, 245B, 255A, 255B, 265A, 265B between the drive connection virtual switches and the drives.

The PCIe switch 205 is a device for connecting the backend interface 120 of the storage controller 105 and the drives. Drives often have two ports, so-called dual-port drives, in order to improve the availability and performance of storage devices. Therefore, the storage device includes two PCIe switches 205 to connect to each port of the dual port drive.

The root complex connection virtual switch 210 is a logical (virtual) PCIe switch configured within the PCIe switch 205 . The root complex connection virtual switch 210 connects between the root complex of the PCIe domain included in the processor and the drive connection virtual switch 220 which will be described later. More specifically, the root complex connection virtual switch 210 connects between the storage controller backend interface 120 and the drive connection virtual switch 220 . Basically, as many root complex connection virtual switches 210 as there are processors accessing each drive are implemented so that each drive can be accessed by multiple processors.

The drive connection virtual switch 220 is a logical (virtual) PCIe switch configured within the PCIe switch 205 like the root complex connection virtual switch 210 . The drive connection virtual switch 220 connects the root complex connection virtual switch 210 and the drives. The drive connection virtual switch 220 may be single within the PCIe switch 205 as shown in FIG. 2, or multiple drive connection virtual switches may be configured in the PCIe switch 205 .

The address conversion unit 225 refers to the address conversion information and converts between the communication protocol of the root complex connection virtual switch 210 and the communication protocol of the drive connection virtual switch 220 . The address converter 225 is, for example, a Non-Transparent Bridge (NTB).

The address conversion unit 225 converts the control information of the PCIe packet arriving at the root complex connection virtual switch 210 into control information adapted in the PCIe domain including the destination drive. The control information includes the header information of the PCIe packet and the address of the access destination. The address conversion unit 225 converts the control information of the PCIe packet that has arrived at the drive connection virtual switch 220 into control information adapted to the PCIe domain including the destination processor. The control information includes header information and a destination address of the PCIe packet.

As described above, PCIe switch 205 includes root complex connected virtual switch 210 , drive connected virtual switch 220 , and address translator 225 between root complex connected virtual switch 210 and drive connected virtual switch 220 . The effects of these are as follows.

From the perspective of the processor 110 of the storage controller, access to the drives from the processor 110 goes through the root complex connection virtual switch 210 , the address translator 225 and the drive connection virtual switch 220 . This is the same for all drives. That is, processor 110 can access any drive at the same distance. Therefore, by adopting the configuration shown in FIG. 2, it is possible to provide stable processing performance as a storage device.

Drives 230, 240, 250, and 260 are devices that support NVMe and store data of the host computer. The drives 230, 240, 250, and 260 include SSDs equipped with non-volatile memory such as flash memory, MRAM (Magnetoresistive Random Access Memory), phase-change memory (Phase-Change Memory), ReRAM (Resistive Random-Access Memory), FeRAM ( Various media such as Ferroelectric Random Access Memory) can also be used. A multistage connection configuration in which a plurality of PCIe switches 205 are connected and drives are further connected may be applied to the storage apparatus.

In general, a storage device may configure a RAID (Redundant Array of Inexpensive Disks) group from a plurality of drives, and store data sent from a host computer in a logical volume, which is a logical storage area configured by the RAID group. When a drive that constitutes a RAID group fails, the storage device can restore the data stored in the failed drive using the data and parity of other drives belonging to the same RAID group.

FIG. 3 is a diagram showing an example of a storage device 300 according to this embodiment. The storage device 300 includes two nodes 100-1 and 100-2 (an arbitrary node is hereinafter referred to as node 100), four drive boxes 200-1, 200-2, 200-3, and 200-4 (an arbitrary drive box is hereinafter referred to as drive box 200), and a link 310 connecting between node 100 and drive box 200.

Node 100-1 includes two storage controllers 105A-1, 150B-1. Storage controller 105A-1 includes memory 130A-1, processor 110A-1, and two backend interfaces 120A-1. Storage controller 105B-1 includes memory 130B-1, processor 110B-1, and two backend interfaces 120B-1.

Node 100-2 includes two storage controllers 105A-2, 150B-2. Storage controller 105A-2 includes memory 130A-2, processor 110A-2, and two backend interfaces 120A-2. Storage controller 105B-2 includes memory 130B-2, processor 110B-2, and two backend interfaces 120B-2.

Drive box 200-1 includes two PCIe switches 205A-1, 205B-1. Drive box 200-2 includes two PCIe switches 205A-2, 205B-2. Drive box 200-3 includes two PCIe switches 205A-3, 205B-3. Drive box 200-4 includes two PCIe switches 205A-4, 205B-4. The details of the components of the node 100 and the drive box 200 are as described with reference to FIGS. 1 and 2. FIG.

In the storage device 300 shown in FIG. 3, links 310 are connected so that all four drive boxes 200 can be accessed from a single storage controller 105 . From the standpoint of availability of the storage device, even-numbered storage controllers 105, for example, are connected to even-numbered PCIe switches 205 of drive boxes 200 so that even-numbered storage controllers 105 can be accessed continuously from normal nodes 100 when nodes 100 become unusable due to some kind of abnormality.

In this way, when the storage controller in the node or the node becomes unusable due to some abnormality, all the drives of the storage device 300 can be continuously accessed from the normal storage controller or the normal node, and the availability of the entire storage device can be maintained.

FIG. 4 is a flow chart showing the procedure for notifying the processor of an abnormality that has occurred on the drive side in the storage apparatus 300 of this embodiment. With this procedure, it is possible to promptly perform appropriate failure processing for an abnormality that has occurred on the drive side. 4 is implemented by the switch control module 215 shown in FIG. First, at step 405 , if the switch control module 215 does not detect a port failure of the connected drive ( 405 : NO), the process proceeds to step 410 . If the switch control module 215 detects a drive port abnormality (405: YES), the process proceeds to step 415.

At step 410, the switch control module 215 performs normal processing. Examples of normal processing include status monitoring of the entire PCIe switch 205, communication control with the processor of the storage controller, processing for an abnormality occurring in the PCIe switch 205, and the like. The switch control module 215 performs port abnormality detection confirmation of the drive as part of normal processing.

In step 415, the switch control module 215 creates a drive port failure message to notify the storage controller processor 110 of the drive port failure. The message contains the drive identifier and fault type information.

At step 418, switch control module 215 determines to which processor 110 the message generated at step 415 should be sent. The switch control module 215 refers to the message transmission destination management table 600 to determine the processor 110 and route for transmitting the message. The message transmission destination management table 600 will be described later.

At step 420 , the switch control module 215 checks the write destination address for each destination processor 110 of the message created at step 410 . The write destination address indicates an address within the memory 130 of the destination processor. The switch control module, for example, refers to the message address field provided in the PCIe configuration register within the PCIe switch 205 in order to confirm the write destination address.

At step 425 , the switch control module 215 issues a write command to write the message generated at step 415 to the write destination address of the memory 130 confirmed at step 420 to the processor 110 of the storage controller. Processor 110 writes the message to the specified write address according to the write instruction.

Subsequently, in step 430, the switch control module 215 issues a write command to the processor 110 of the other storage controller to write the message generated in step 415 to the write destination address confirmed in step 420, and terminates.

According to the flowchart shown in FIG. 4, any processor (controller) that can access the storage drive can be notified of an abnormality on the drive side, so that the storage apparatus can promptly perform appropriate fault processing.

FIG. 5 is a flow chart showing the processing procedure of the processor for the abnormality notification received from the drive box. By the processing of the flow chart of FIG. 5, it is possible to promptly perform appropriate failure processing for an abnormality that has occurred on the storage drive side.

The execution subject of the flow chart shown in FIG. 5 is a control program that runs on the processor 110 of the storage controller. This control program is stored in memory 130 used by processor 110 .

First, the control program performs normal processing at step 505 . The normal processing here includes processing for controlling the storage device, read processing from the host computer, write processing, destaging processing for storing data stored in the cache area provided in the memory 130 of the processor 110 in the drive, and other processing related to functions for providing business continuity.

In it, the control program confirms the abnormality notification about the storage drive received from the PCIe switch 205 . This processing may be performed at regular intervals during normal processing, or may be performed immediately in response to an interrupt.

At step 510, the control program returns to step 505 when the condition for confirming the abnormality notification is not satisfied (510: NO). The control program proceeds to step 515 when the conditions for confirming the abnormality notification from the drive are satisfied (510: YES).

At step 515, the control program checks for an area in memory 130 for storing anomaly notifications for storage drives. In step 520, the control program returns to step 505 if no anomaly notification has been stored. The control program proceeds to step 525 if an anomaly notification is stored.

At step 525, the control program refers to the stored anomaly notification and identifies the storage drive in which the anomaly occurred. At step 530 , the control program refers to management information for managing issued IO requests stored in memory 130 and checks whether there is an issued IO request for the storage drive identified at step 525 . Additionally, the control program references the IO request queue in memory 130 to see if there are any outstanding IO requests for the storage drive identified in step 525 .

At step 535 , the control program cancels (Aborts or cancels) the issued IO requests for the drive identified at step 525 and the unissued IO requests confirmed at step 535 . This avoids issuing an IO request to an abnormal storage drive, and can properly process unfinished IO requests.

By performing the processing up to step 535, all processors that can access the drive identified in step 525 can stop IO. As a result, time-out due to IO to the drive specified in step 525 can be avoided, and appropriate failure processing can be promptly performed.

At step 540 , the control program confirms whether another control program is performing failure processing for the drive notified of the abnormality identified at step 525 . Information about the control program that is executing the fault handling is stored in the shared memory area of each storage controller 105, for example. If another control program is performing fault handling for the drive notified of the abnormality specified in step 525, the fault handling in progress is left to the fault handling to complete the fault handling appropriately, and the process returns to step 505.

If another control program has not performed failure processing for the drive that notified the abnormality identified in step 525 , the control program takes charge of recovery processing 545 for the drive that notified the abnormality identified in step 525 . As a result, recovery processing is properly executed. The recovery processing here is, for example, clearing failure information of the PCIe switch, initialization of the drive port, and the like.

By implementing the flow chart shown in FIG. 5, the control program can promptly perform appropriate failure processing for an abnormality that has occurred on the drive side.

FIG. 6 is a diagram showing a message transmission destination management table in the storage device 300 of this embodiment. A node # column 605 indicates serial numbers assigned to nodes in the storage device 300 . For example, in FIG. 3, node 100-1 may be numbered node #0, and node 100-2 may be numbered node #1.

A CTL# column 610 indicates serial numbers assigned to the storage controllers constituting the nodes in the storage device 300 . For example, in FIG. 3, the number of the storage controller 105A-1 of the node 100-1 can be CTL#0, and the number of the storage controller 105B-1 of the node 100-1 can be CTL#1.

A CPU# column 615 indicates serial numbers assigned to the processors included in the storage controller in the storage device 300 . For example, in FIG. 3, the number of the processor 110A-1 of the node 100-1 can be CPU#0, and the number of the processor 110B-1 of the node 100-1 can be CPU#1.

A Port# column 620 indicates serial numbers assigned to communication ports between nodes and drive boxes in the storage device 300 . For example, in FIG. 1, the number of the backend port 122A-1 can be Port#0, the number of the backend port 122A-2 can be Port#1, the number of the backend port 122A-3 can be Port#2, and the number of the backend port 122A-4 can be Port#3.

As described above, each record indicates the processor to which the message is sent and the port through which the message passes (path for communication between the processor and the PCIe switch). One PCIe switch communicates with one processor via one or more ports.

The Send Target column 625 indicates the destination that the switch control module has selected to send the message generated in step 415 of FIG. For example, in FIG. 6, the value of the Send Target column 625 of Node#0, CTL#0, CPU#0, and Port#0 is "1". The value of the Send Target column 625 of the record of node #0, CTL #0, CPU #0, Port #2 is "0".

In the Send Target column 625 of the message destination management table 600 of FIG. 6, "1" indicates that the record is the message destination. Assume that Send Target “0” is not set as the destination of the message. That is, node #0, CTL#0, CPU#0, and Port#0 are destinations of messages, and node #0, CTL#0, CPU#0, and Port#2 are not destinations of messages.

The message transmission destination management table 600 of FIG. 6 is generated by a control program running on the processor 110 of the storage controller and stored in a memory (not shown) within the PCIe switch that can be referenced by the switch control module. Then, the processor 110 of the storage controller appropriately updates the message transmission destination management table according to the operating status of the backend interface, and stores it in the memory within the PCIe switch each time.

Effects of the message transmission destination management table 600 of FIG. 6 will be described. First, the switch control module can confirm the destination to which the message generated in step 415 of FIG. 4 should be sent by referring to this message destination management table 600 . This reduces the load on the switch control module.

For example, if one processor is equipped with multiple backend ports and the multiple backend ports are used evenly to communicate with the drives in the drive box, any one of the backend ports can be used to send a message. may be set to "0".

Also, in some cases, the backend interface 120 of the storage device 300 may become unusable for some reason. At that time, the processor 110 of the storage controller sets the value of the Send Target column 625 of the disabled backend port to "0". By doing so, the switch control module 215 can identify the backend interfaces (backend ports) that cannot be used by referring to the message destination management table, thereby avoiding unnecessary message transmission. This reduces the load on the switch control module. It also avoids errors caused by sending messages to unavailable backend interfaces.

As described above, in the storage device of this embodiment, the switch device (PCIe switch) converts between the communication protocol with the processor and the communication protocol with the storage drives. This equalizes communication performance between processors and storage drives.

Furthermore, when the switch device detects an abnormality in communication with the storage drive, it sends a message notifying the abnormality to each of the processors selected from the connected processors. This makes it possible to quickly process an abnormality. Note that the features of this embodiment can be applied to a storage device that uses a communication protocol different from the communication protocol of the above configuration example.

The above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the described configurations. In addition, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

Further, each of the configurations, functions, processing units, processing means, etc. described above may be realized by hardware, for example, by designing them in an integrated circuit. Moreover, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function.

Furthermore, information such as programs, tables, and files that implement each function may be stored in memory, storage devices such as hard disks and SSDs, or storage media such as IC cards, SD cards, and DVDs. Furthermore, control lines and information lines indicate those considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In practice, it may be considered that almost all configurations are interconnected.

Aspects of the present invention other than those described in the claims include the following. The storage apparatus includes a plurality of controllers, a plurality of storage drives, a plurality of controller-side ports connected to the plurality of controllers, respectively, and a plurality of drive-side ports respectively connected to the plurality of storage drives, wherein the switch device performs address translation between the plurality of controller-side ports and the plurality of drive-side ports.
Further aspects of the present invention other than those described in the claims are as follows.
(1)
multiple controllers;
a plurality of storage drives;
a plurality of switch devices;
each switch device of the plurality of switch devices,
a plurality of controller-side switches that respectively communicate with different controllers among the plurality of controllers via a plurality of controller-side ports;
a drive-side switch communicating with each of the plurality of storage drives via a plurality of drive-side ports;
an address conversion unit that performs address conversion between the plurality of controller-side switches and the drive-side switches;
the drive-side switch is connected to the plurality of controller-side switches;
each storage drive of the plurality of storage drives is connected to the plurality of switch devices;
wherein, when each switch device of the plurality of switch devices detects an abnormality in one of the plurality of drive-side ports, it refers to information specifying the controller to notify the abnormality, and selects the controller to notify the abnormality from the plurality of controllers.
(2)
(1) The storage device according to
The storage device, wherein the plurality of controller-side switches and the drive-side switches are virtual switches.
(3)
multiple controllers;
a plurality of storage drives;
a switch device including a plurality of controller-side ports connected to each of the plurality of controllers and a plurality of drive-side ports connected to each of the plurality of storage drives;
The switch device is
converting addresses between the plurality of controller-side ports and the plurality of drive-side ports;
A storage device that, when an abnormality is detected in one of the plurality of drive-side ports, notifies a message indicating the abnormality to the plurality of controllers via each of the plurality of controller-side ports.
(4)
The storage device according to (3),
The storage device according to claim 1, wherein the controller notified of the abnormality cancels IO requests issued and to be issued to the one port in which the abnormality is detected.
(5)
The storage device according to (3),
A storage device, wherein one controller among the plurality of controllers performs recovery processing from the abnormality when another controller does not perform the recovery processing from the abnormality.
(6)
A communication method in a storage device including a plurality of switch devices,
each switch device of the plurality of switch devices,
a plurality of controller-side switches;
a drive-side switch;
and an address translator,
The communication method includes:
wherein the plurality of controller-side switches communicate with different controllers among the plurality of controllers via a plurality of controller-side ports;
the drive-side switch communicates with a plurality of storage drives via a plurality of drive-side ports;
the address conversion unit performs address conversion between the plurality of controller-side switches and the drive-side switches;
the drive-side switch in communication with the plurality of controller-side switches;
each storage drive of the plurality of storage drives in communication with the plurality of switch devices;
A communication method, wherein, when each switch device of said plurality of switch devices detects an abnormality in one of said plurality of drive-side ports, it refers to information specifying a controller that notifies said abnormality, and selects a controller that notifies said abnormality from said plurality of controllers.
(7)
A communication method for a switch device in a storage device, comprising:
the switch device communicates with a plurality of controllers via each of a plurality of controller-side ports;
wherein the switch device communicates with a plurality of storage drives via each of a plurality of drive-side ports;
wherein the switch device translates addresses between the plurality of controller-side ports and the plurality of drive-side ports;
A communication method for a switch device, wherein when the switch device detects an abnormality in one of the plurality of drive-side ports, a message indicating the abnormality is sent to the plurality of controllers via each of the plurality of controller-side ports.
(8)
multiple controllers;
a plurality of storage drives;
a plurality of switch devices;
each switch device of the plurality of switch devices,
a plurality of controller-side switches that respectively communicate with different controllers among the plurality of controllers via a plurality of controller-side ports;
a drive-side switch communicating with each of the plurality of storage drives via a plurality of drive-side ports;
an address conversion unit that performs address conversion between the plurality of controller-side switches and the drive-side switches;
the drive-side switch is connected to the plurality of controller-side switches;
each storage drive of the plurality of storage drives is connected to the plurality of switch devices;
a drive box housing the plurality of storage drives and the plurality of switch devices;
a plurality of nodes, and
each node of the plurality of nodes includes a controller in the plurality of controllers;
The storage device, wherein the plurality of switch devices in the drive box are connected to different controllers in one of the plurality of nodes.
(9)
The storage device according to (8),
The storage device, wherein the plurality of controller-side switches of each of the plurality of switch devices are connected to controllers of nodes different from each other.

100... node, 105A, 105B... storage controller, 200... drive box, 205A, 205B... PCIe switch, 220A, 220B... switch control module, 225A, 225B... address converter

Claims (8)

multiple controllers;
a plurality of storage drives;
a switch device including a plurality of controller-side ports connected to the plurality of controllers and a plurality of drive-side ports connected to the plurality of storage drives;
a drive box containing the plurality of storage drives and the plurality of switch devices;
a plurality of nodes;
including
The switch device is
a controller-side switch that communicates with the controller through the controller-side port;
a drive-side switch communicating with each of the plurality of storage drives via the plurality of drive-side ports;
an address conversion unit that performs address conversion between the controller-side switch and the drive-side switch;
in the switch device, the drive-side switch is connected to a plurality of the controller-side switches;
the storage drive is connected to the plurality of switch devices;
each node of the plurality of nodes includes two or more controllers in the plurality of controllers;
wherein the plurality of switch devices are connected to controllers of different nodes;
storage device. The storage device according to claim 1,
The storage device, wherein the controller-side switch and the drive-side switch are virtual switches. The storage device according to claim 1,
The storage device, wherein, when detecting an abnormality in one of the plurality of drive-side ports, the switch device notifies the plurality of controllers of a message indicating the abnormality via each of the plurality of controller-side ports. The storage device according to claim 3,
The storage device according to claim 1, wherein the controller notified of the abnormality cancels IO requests issued and to be issued to the one port in which the abnormality is detected. The storage device according to claim 3,
A storage device, wherein one controller among the plurality of controllers performs recovery processing from the abnormality when another controller does not perform the recovery processing from the abnormality. The storage device according to claim 1,
wherein, upon detecting an abnormality in one of the plurality of drive-side ports, the switch device refers to information designating a controller that notifies the abnormality, and selects the controller that notifies the abnormality from the plurality of controllers. The storage device according to claim 1,
the plurality of controller-side switches of each switching device in the plurality of switching devices are connected to controllers of different nodes;
storage device. The storage device according to claim 1,
the two or more controllers in each node of the plurality of nodes access the storage drive through different switch devices;
storage device.

Images