WO2012032607A1 - Storage system, storage apparatus, and controller - Google Patents

Abstract

The reliability of data access to a storage unit can be improved. A storage system (10) comprises: a plurality of storage apparatuses (18a to 18c) each of which has a storage unit, a first relay unit connected in series to a data transmission path (13) for relaying the data access to the storage unit, and a second relay unit connected in series to a data transmission path (14) and also connected to the first relay unit via a third data transmission path, the opening/closing of which is controlled in accordance with an external instruction, for relaying the data access to the storage unit; and a controller (12) that disables a communication using the third data transmission path of each of the plurality of storage apparatuses (18a to 18c), thereby monitoring the statuses of the connections to the storage unit via the data transmission paths (13, 14) and that, when having detected any storage unit the data access to which cannot be performed via any of the data transmission paths (13, 14), transmits an instruction to enable a communication using the third data transmission path of at least one of the storage apparatuses.

Description

Storage system, storage device, and controller

The present invention relates to a storage system, a storage apparatus, and a controller that process data.

The storage system has a storage device that stores data, and a controller that controls writing of data to the storage device or reading of data from the storage device. As one of storage systems, there is a storage system connected from a controller to a storage device through a redundant path.

The storage system having a redundant path has a first data transmission path and a second data transmission path for transmitting data between the storage device and the controller. Some storage systems having redundant paths have a plurality of relay devices connected in series to each of the first data transmission path and the second data transmission path.

In a storage system having a redundant path, even if a failure or the like occurs in either the first or second data transmission path due to an abnormality in the relay device, the controller is connected via the data transmission path that has not failed. Can access data to the storage device.

Further, as one storage system, a controller and one or more switch devices are made redundant, the redundant switch devices are connected, and a request interface and a monitoring control unit are connected to each of the redundant switch devices. There is a technology with

In addition, there is a technology in which one end of a plurality of cascades is connected by an inter-cascade link, and the inter-cascade link has a logical connection state and a disconnection state.
In addition, the control unit includes an abnormality detection unit that detects that an abnormality has occurred in the first route among the plurality of routes, and a communication unit that communicates with other control units when an abnormality is detected. There is technology to prepare. The control unit further includes an access unit that performs access to the storage device using the second path among the plurality of paths based on the communication result.

JP 2008-242872 A JP 2009-211140 A JP 2009-157859 A

However, even in a storage system having the first and second data transmission paths, if a failure or the like occurs in both data transmission paths due to an abnormality in a plurality of relay devices, the memory that cannot be accessed by the controller A device can arise.

In this case, it is conceivable to open data between the first data transmission path and the second data transmission path and to try to access the storage device while avoiding a location where a failure or the like has occurred. However, simply by opening the first data transmission path and the second data transmission path, a data transmission path in a loop state is generated, and there is a possibility that data cannot be correctly accessed from the controller to the storage device. is there.

In view of the above, an object of the present invention is to provide a storage system, a storage device, and a controller that improve the reliability of data access to a storage device.

In order to achieve the above object, the following storage system is provided.
This storage system is connected to a storage device and a first data transmission path, and is connected to a first relay device that relays data access to the storage device and a second data transmission path, and receives an instruction from the outside. A plurality of storage devices each having a second relay device that is connected to the first relay device via a third data transmission path that is controlled to open and close and relays data access to the storage device; and a plurality of storage devices A first data transmission path in which the first relay devices of the respective devices are connected in series, and a second data transmission path in which the second relay devices of the plurality of storage devices are connected in series; The communication via the third data transmission path of each storage apparatus is blocked, the connection status to the storage device via the first and second data transmission paths is monitored, and the first And upon detecting a data inaccessible storage device from any of the second data transmission path includes a controller for transmitting the opening instruction communication via the third data transmission path of at least one storage device.

The reliability of data access to the storage device can be improved.
These and other objects, features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings which illustrate preferred embodiments by way of example of the present invention.

1 is a diagram illustrating an example of a storage system according to a first embodiment. FIG. It is a figure which shows an example of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the hardware of the control module which concerns on 2nd Embodiment. It is a figure which shows an example of the hardware of the drive enclosure which concerns on 2nd Embodiment. It is a figure which shows an example of the data transmission of the storage system which concerns on 2nd Embodiment. It is a figure which shows the comparative example of a storage system. It is a figure which shows an example of the control table which concerns on 2nd Embodiment. It is a flowchart which shows an example of the process sequence of the controller which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the control table which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the control table which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the control table which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a flowchart which shows an example of the process sequence of the expander which concerns on 2nd Embodiment. It is a sequence diagram which shows an example of the procedure of the opening process of the data transmission path between the expanders which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the state of the storage system which concerns on 2nd Embodiment. It is a figure which shows an example of the drive enclosure which concerns on 3rd Embodiment. It is a flowchart which shows an example of the process sequence of the expander which concerns on 3rd Embodiment.

Hereinafter, embodiments will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating an example of a storage system according to the first embodiment.

The storage system 10 includes a plurality of storage devices 18a, 18b, 18c, a controller 12, and two data transmission paths 13, 14.
The storage device 18a includes a storage device 11a, two relay devices 15a and 16a, and a data transmission path 17a. The relay device 15 a is connected to the data transmission path 13 in series. Further, the relay device 15a is connected to the storage device 11a and relays data access to the storage device 11a.

The relay device 16a is connected to the data transmission path 14 in series. Further, the relay device 16a is connected to the storage device 11a and relays data access to the storage device 11a. Further, the relay device 15a and the relay device 16a are connected by a data transmission path 17a. The data transmission path 17a is controlled to open and close according to an instruction from the outside.

The storage device 18b includes a storage device 11b, two relay devices 15b and 16b, and a data transmission path 17b. The relay device 15 b is connected in series to the data transmission path 13. Further, the relay device 15b is connected to the storage device 11b and relays data access to the storage device 11b.

The relay device 16b is connected to the data transmission path 14 in series. Further, the relay device 16b is connected to the storage device 11b and relays data access to the storage device 11b. Further, the relay device 15b and the relay device 16b are connected by a data transmission path 17b. The data transmission path 17b is controlled to open and close according to an instruction from the outside.

The storage device 18c includes a storage device 11c, two relay devices 15c and 16c, and a data transmission path 17c. The relay device 15 c is connected in series to the data transmission path 13. Further, the relay device 15c is connected to the storage device 11c and relays data access to the storage device 11c.

The relay device 16c is connected to the data transmission path 14 in series. Further, the relay device 16c is connected to the storage device 11c and relays data access to the storage device 11c. Further, the relay device 15c and the relay device 16c are connected by a data transmission path 17c. The data transmission path 17c is controlled to open and close according to an instruction from the outside.

The controller 12 is connected to the data transmission path 13 and the data transmission path 14. The controller 12 writes data to the storage devices 11a to 11c or reads data from the storage devices 11a to 11c. The controller 12 performs data access to each of the storage devices 11a to 11c via the data transmission path 13 or the data transmission path 14 when there is no failure in the data transmission path 13 and the data transmission path 14.

Furthermore, the controller 12 blocks or opens communication via the data transmission paths 17a to 17c of the storage devices 18a to 18c. Furthermore, the controller 12 monitors the connection status to the storage devices 11a to 11c via the data transmission path 13 and the data transmission path 14.

Next, a procedure for opening the data transmission paths 17a to 17c will be described.
As an initial state, the data transmission paths 17a to 17c are all blocked.
When a failure or the like occurs in the data transmission path 13 or the data transmission path 14, the controller 12 determines whether there is any storage device that cannot access data from either the data transmission path 13 or the data transmission path 14 among the storage devices 11a to 11c. Determine.

If there is a storage device inaccessible to data, the controller 12 transmits a communication opening instruction via at least one of the data transmission paths 17a to 17c. Here, the case where there is a storage device inaccessible to data means that communication with at least one of the relay devices 15a to 15c is interrupted and communication with at least one of the relay devices 16a to 16c is performed. This is the case when it is interrupted.

At this time, for example, the controller 12 may open the data transmission paths 17a to 17c one by one in order. Alternatively, after the controller 12 has opened all the data transmission paths 17a to 17c, the controller 12 may leave one and close the other.

Alternatively, the controller 12 detects the farthest end storage device that can communicate via either the data transmission path 13 or the data transmission path 14, and the data transmission of the detected storage apparatus among the data transmission paths 17a to 17c. You may make it open a path | route. Here, the farthest end storage device refers to a storage device having the largest number of stages from the controller 12.

By opening the data transmission path of the farthest end storage device among the data transmission paths 17a to 17c, the data transmission path to the communicable portion of the data transmission path 13 or the data transmission path 14 can be used to the maximum extent. This makes it possible to efficiently select a data transmission path to be opened.

Then, the controller 12 performs data access to the storage device determined to be inaccessible via the opened data transmission path among the data transmission paths 17a to 17c. That is, the controller 12 accesses the storage device through a part of the data transmission path 13, the opened data transmission path among the data transmission paths 17a to 17c, and a part of the data transmission path 14.

For example, if an error occurs in the relay device 15c and communication between the controller 12 and the relay device 15c is interrupted, an error occurs in the relay device 16a and communication between the controller 12 and the relay device 16a is interrupted, Data cannot be accessed from the controller 12 to the storage device 11c.

In this case, in the storage system 10, the data transmission path 17b of the storage device 18b is opened. Then, the controller 12 performs data access to the storage device 11c via the opened data transmission path 17b. That is, the controller 12 accesses the storage device 11c via the data transmission path 19 passing through the relay apparatuses 15a and 15b, the data transmission path 17b, and the relay apparatuses 16b and 16c.

As described above, in the storage system 10, when the controller 12 detects a storage device that is inaccessible to data from either the data transmission path 13 or the data transmission path 14, the following processing is executed. That is, the controller 12 transmits a communication opening instruction via at least one of the data transmission paths 17a to 17c.

Thus, even when a failure or the like occurs in the data transmission path 13 and the data transmission path 14 due to an abnormality in a plurality of relay apparatuses, data access from the controller 12 to each of the storage devices 11a to 11c becomes possible.

In the storage system 10, the data transmission paths 17 a to 17 c are controlled to be opened / closed according to instructions from the controller 12. Thereby, it can suppress that the data transmission path | route of a loop state arises.

For example, it is assumed that the data transmission paths 17a to 17c are not controlled to be opened but all are always open. In this case, for example, a data transmission path in a loop state via the relay apparatus 15b, the relay apparatus 15c, the data transmission path 17c, the relay apparatus 16c, the relay apparatus 16b, and the data transmission path 17b is generated.

In the storage system 10 of FIG. 1, for example, only the data transmission path 17b can be opened and the remaining data transmission paths 17a and 17c can be blocked, so that the occurrence of a looped data transmission path is suppressed. it can.

In this way, the storage system 10 can improve the reliability of data access to the storage devices 11a to 11c.
[Second Embodiment]
Next, a more specific embodiment of the storage system 10 will be described as a second embodiment.

FIG. 2 is a diagram illustrating an example of a storage system according to the second embodiment.
The storage system 100 includes a controller enclosure (CE) 110 that includes two controllers 114 and 117. The storage system 100 further includes a plurality of drive enclosures (DE) 150a, 150b, and 150c.

The drive enclosures 150a to 150c are identified in the storage system 100 by identifiers “1”, “2”, and “3”, respectively. The controller enclosure 110 and the drive enclosures 150a to 150c have storage devices 113, 155a, 155b, and 155c, respectively.

That is, each drive enclosure 150a to 150c corresponds to a storage device. The storage devices 113 and 155a to 155c are identified in the storage system 100 by identifiers “A”, “B”, “C”, and “D”, respectively.

The drive enclosures 150a to 150c are connected in series to two data transmission paths 121 and 122 connected to the controllers 114 and 117, respectively. For the data transmission paths 121 and 122, for example, SAS (Serial Attached SCSI) is used. Here, the number of drive enclosures 150a to 150c is not limited to three, and may be appropriately set according to the required storage capacity, for example.

First, the drive enclosure 150a will be described.
The drive enclosure 150a includes an expander (EXP) 151a including a memory 152a, an expander 153a including a memory 154a, and a storage device 155a that stores data. The expanders 151a and 153a are identified in the drive enclosure 150a by identifiers “0” and “1”, respectively. For example, a magnetic disk (HDD: Hard Disk Drive) or a solid state drive (SSD: Solid State Drive) is used as the storage device 155a.

The expanders 151a and 153a are each connected to the storage device 155a through a data transmission path such as SAS. The expander 151 a is connected in series to the data transmission path 121, and the expander 153 a is connected in series to the data transmission path 122. The expanders 151a and 153a relay data access from the controller 114 or the controller 117 to the storage device 155a, respectively.

A communication path 159a is provided between the expander 151a and the expander 153a. For example, I ² C (trademark), Ethernet (registered trademark), or the like is used for the communication path 159a. Further, the expander 151a is provided with an inter-expander port 156a, and the expander 153a is provided with an inter-expander port 157a.

Between the expander 151a and the expander 153a, an inter-expander data transmission path 158a that is connected to the inter-expander port 156a and the inter-expander port 157a is provided. For example, SAS or the like is used for the inter-expander data transmission path 158a.

In the initial state, the inter-expander port 156a and the inter-expander port 157a are both disabled, and the inter-expander data transmission path 158a is closed. When the inter-expander port 156a and the inter-expander port 157a are both enabled, the inter-expander data transmission path 158a is opened.

The memory 152a stores a control table in which a connection state between the expander 151a and the controller 114 or the controller 117 and a connection state between the expander 151a and the storage device 155a are stored. Further, this control table stores a connection state between the expander 153a and the expander 151a and a connection state between the expander 153a and the storage device 155a. Further, this control table includes a control table possessed by a lower expander connected to the expander 151a.

The memory 154a stores a control table in which a connection state between the expander 153a and the controller 114 or the controller 117 and a connection state between the expander 153a and the storage device 155a are stored. Further, this control table stores a connection state between the expander 151a and the expander 153a and a connection state between the expander 151a and the storage device 155a. Further, this control table includes a control table possessed by a lower expander connected to the expander 153a.

The expander 151a monitors the connection state between the storage device 155a and the lower expander connected to the data transmission path 121, and detects this when a change occurs in the connection state. The expander 151a notifies the controller 114 or the controller 117 of the change in the connection state via the data transmission path 121.

Further, the expander 151a updates the control table stored in the memory 152a in accordance with an instruction from the controller 114 or the controller 117. Further, the expander 151a notifies the controller 114 or the controller 117 of the updated control table.

Further, the expander 151a opens or closes the inter-expander data transmission path 158a in accordance with an instruction from the controller 114 or the controller 117. For example, when opening, the expander 151a instructs the inter-expander port 156a and instructs the expander 153a to enable the inter-expander port 157a via the communication path 159a. As a result, the inter-expander data transmission path 158a is opened.

Further, the expander 151a notifies the controller 114 or the controller 117 of the result of the opening process of the inter-expander data transmission path 158a via the data transmission path 121.

The expander 153a monitors a connection state between the storage device 155a and a lower expander connected to the data transmission path 122, and detects a change in the connection state when a change occurs in the connection state. The expander 153a notifies the controller 114 or the controller 117 of the detected connection state change via the data transmission path 122.

Further, the expander 153a updates the control table stored in the memory 154a in accordance with an instruction from the controller 114 or the controller 117. Furthermore, the expander 153a notifies the controller 114 or the controller 117 of the updated control table.

Further, the expander 153a opens or closes the inter-expander data transmission path 158a in accordance with an instruction from the controller 114 or the controller 117. For example, when opening, the expander 153a instructs the inter-expander port 157a and instructs the expander 151a via the communication path 159a to enable the inter-expander port 156a. As a result, the inter-expander data transmission path 158a is opened.

Further, the expander 153a notifies the controller 114 or the controller 117 of the result of the opening process of the inter-expander data transmission path 158a via the data transmission path 122.

Next, the drive enclosure 150b will be described.
The drive enclosure 150b includes an expander 151b including a memory 152b, an expander 153b including a memory 154b, and a storage device 155b that stores data. The expanders 151b and 153b are identified in the drive enclosure 150b by identifiers “0” and “1”, respectively. As the storage device 155b, for example, a magnetic disk or a solid state drive is used.

The expanders 151b and 153b are connected to the storage device 155b through a data transmission path such as SAS. The expander 151 b is connected in series to the data transmission path 121, and the expander 153 b is connected in series to the data transmission path 122. The expanders 151b and 153b relay data access from the controller 114 or the controller 117 to the storage device 155b, respectively.

A communication path 159b is provided between the expander 151b and the expander 153b. For example, I ² C (trademark), Ethernet (registered trademark), or the like is used for the communication path 159b. Further, the expander 151b is provided with an inter-expander port 156b, and the expander 153b is provided with an inter-expander port 157b.

Between the expander 151b and the expander 153b, an inter-expander data transmission path 158b that is connected to the inter-expander port 156b and the inter-expander port 157b is provided. For example, SAS is used for the inter-expander data transmission path 158b.

In the initial state, the inter-expander port 156b and the inter-expander port 157b are both disabled, and the inter-expander data transmission path 158b is closed. When the inter-expander port 156b and the inter-expander port 157b are both enabled, the inter-expander data transmission path 158b is opened.

The memory 152b stores a control table in which a connection state between the expander 151b and the controller 114 or the controller 117 and a connection state between the expander 151b and the storage device 155b are stored. Further, this control table stores a connection state between the expander 153b and the expander 151b and a connection state between the expander 153b and the storage device 155b. Further, this control table includes a control table possessed by a lower expander connected to the expander 151b.

The memory 154b stores a control table in which a connection state between the expander 153b and the controller 114 or the controller 117 and a connection state between the expander 153b and the storage device 155b are stored. Further, this control table stores a connection state between the expander 151b and the expander 153b and a connection state between the expander 151b and the storage device 155b. Further, this control table includes a control table possessed by a lower expander connected to the expander 153b.

The function of the expander 151b is the same as that of the expander 151a of the drive enclosure 150a. The function of the expander 153b is the same as that of the expander 153a of the drive enclosure 150a.

Next, the drive enclosure 150c will be described.
The drive enclosure 150c includes an expander 151c including a memory 152c, an expander 153c including a memory 154c, and a storage device 155c that stores data. The expanders 151c and 153c are identified in the drive enclosure 150c by identifiers “0” and “1”, respectively. As the storage device 155c, for example, a magnetic disk or a solid state drive is used.

The expanders 151c and 153c are connected to the storage device 155c through a data transmission path such as SAS. The expander 151 c is connected in series to the data transmission path 121, and the expander 153 c is connected in series to the data transmission path 122. The expanders 151c and 153c relay data access from the controller 114 or the controller 117 to the storage device 155c, respectively.

A communication path 159c is provided between the expander 151c and the expander 153c. For example, I ² C (trademark), Ethernet (registered trademark), or the like is used for the communication path 159c. Further, the expander 151c is provided with an inter-expander port 156c, and the expander 153c is provided with an inter-expander port 157c.

Between the expander 151c and the expander 153c, an inter-expander data transmission path 158c connected to the inter-expander port 156c and the inter-expander port 157c is provided. For example, SAS is used for the inter-expander data transmission path 158c.

In the initial state, the inter-expander port 156c and the inter-expander port 157c are both disabled, and the inter-expander data transmission path 158c is blocked. When the inter-expander port 156c and the inter-expander port 157c are both enabled, the inter-expander data transmission path 158c is opened.

The memory 152c stores a control table in which a connection state between the expander 151c and the controller 114 or the controller 117 and a connection state between the expander 151c and the storage device 155c are stored. Further, the control table stores a connection state between the expander 153c and the expander 151c and a connection state between the expander 153c and the storage device 155c.

The memory 154c stores a control table in which a connection state between the expander 153c and the controller 114 or the controller 117 and a connection state between the expander 153c and the storage device 155c are stored. Further, this control table stores a connection state between the expander 151c and the expander 153c and a connection state between the expander 151c and the storage device 155c.

The function of the expander 151c is the same as that of the expander 151a of the drive enclosure 150a. The function of the expander 153c is the same as that of the expander 153a of the drive enclosure 150a.

Next, the controller enclosure 110 will be described.
The controller enclosure 110 includes control modules (CMs) 111 and 112 and a storage device 113 that stores data. The control modules 111 and 112 are identified in the controller enclosure 110 by identifiers “0” and “1”, respectively. Further, for example, a magnetic disk or a solid state drive is used for the storage device 113.

The control module 111 includes a controller 114 and an expander 115 including a memory 116. The control module 112 includes a controller 117 and an expander 118 including a memory 119.

The controller 114 and the expander 115 are connected to the data transmission path 121, and the controller 117 and the expander 118 are connected to the data transmission path 122. Further, the controller 114 is connected to the expander 118 through a data transmission path such as SAS. The controller 117 is connected to the expander 115 via a data transmission path such as SAS.

The expander 115 and the expander 118 are connected to the storage device 113 through a data transmission path such as SAS. The expanders 115 and 118 relay data transmission between the storage device 113 and the controller 114 or the controller 117.

The memory 116 stores a control table in which a connection state between the expander 115 and the controller 114 or the controller 117 and a connection state between the expander 115 and the storage device 113 are stored. Further, this control table includes a control table included in the lower expander 151 a connected to the data transmission path 121.

The memory 119 stores a control table in which a connection state between the expander 118 and the controller 114 or the controller 117 and a connection state between the expander 118 and the storage device 113 are stored. Further, this control table includes a control table included in the lower expander 153 a connected to the data transmission path 122.

The expander 115 updates the control table stored in the memory 116 in accordance with an instruction from the controller 114 or the controller 117. Further, the expander 115 notifies the controller 114 or the controller 117 of the updated control table.

Further, the expander 115 monitors the connection state between the storage device 113 and the lower expander 151a connected to the data transmission path 121, and detects this when a change occurs in the connection state. Then, the expander 115 notifies the controller 114 or the controller 117 of the change in the connection state.

Further, the expander 118 updates the control table stored in the memory 119 in accordance with an instruction from the controller 114 or the controller 117. Further, the expander 118 notifies the controller 114 or the controller 117 of the updated control table.

Furthermore, the expander 118 monitors the connection state between the storage device 113 and the lower expander 153a connected to the data transmission path 122, and detects this when a change occurs in the connection state. Then, the expander 118 notifies the controller 114 or the controller 117 of the change in the connection state.

The controller 114 includes a control circuit 160, a memory 161, and a communication port 162. Processing by the controller 114 is executed by the control circuit 160. A data transmission path 121 is connected to the communication port 162.

The controller 117 includes a control circuit 163, a memory 164, and a communication port 165. Processing by the controller 117 is executed by the control circuit 163. A data transmission path 122 is connected to the communication port 165.

The controllers 114 and 117 perform the following processing through data transmission paths that pass through some of the expanders 115, 118, 151a to 151c, and 153a to 153c, respectively. That is, the controllers 114 and 117 respectively write data to the storage devices 113 and 155a to 155c, or read data from the storage devices 113 and 155a to 155c. This process is performed, for example, according to an instruction from a host device (not shown).

Further, when the controllers 114 and 117 receive notification of connection state change from any one of the expanders 115, 118, 151a to 151c, and 153a to 153c, they perform the following processing. That is, the controllers 114 and 117 instruct the expanders 115, 118, 151a to 151c, and 153a to 153c, respectively, to update the control table.

Furthermore, the controllers 114 and 117 collect all the control tables of the expanders 115, 118, 151a to 151c, and 153a to 153c, respectively. The collected control tables are stored in the memories 161 and 164. Furthermore, the controllers 114 and 117 refer to the collected control tables stored in the memories 161 and 164 and monitor the connection status to the storage devices 113 and 155a to 155c via the data transmission path 121 and the data transmission path 122. To do.

Further, the controllers 114 and 117 block or open communication via the inter-expander data transmission paths 158a to 158c of the drive enclosures 150a to 150c.

Next, the hardware of the control modules 111 and 112 will be described. Since the control modules 111 and 112 have the same configuration, the control module 111 will be described as a representative here.

FIG. 3 is a diagram illustrating an example of hardware of the control module according to the second embodiment.
The control module 111 includes a controller 301. Ports 302 and 303 are connected to the controller 301 via a bus. The ports 302 and 303 are connected to an external host device.

Furthermore, a RAM (Random Access Memory) 304 is connected to the controller 301 via a bus. A real time clock (RTC: Real Time Clock) 305 is connected between the controller 301 and the RAM 304.

Furthermore, a LAN (Local Area Network) controller 306 is connected to the controller 301 via a bus. LAN ports 307 and 308 are connected to the LAN controller 306 via a bus. Further, a ROM (Read Only Memory) 309 is connected to the LAN controller 306. A data management circuit 310 is connected between the controller 301 and the LAN controller 306. A flash memory 311 is connected to the data management circuit 310. Data stored in the flash memory 311 is managed by the data management circuit 310.

Furthermore, a driver (DRV: Driver) 312 is connected to the controller 301 via a bus. A debug port 313 is connected to the driver 312 via a bus. Furthermore, a driver 314 is connected to the controller 301 via a bus. A debug port 315 is connected to the driver 314 via a bus.

Further, a flash memory 316 is connected to the controller 301 via a bus. A system controller 317 is connected between the controller 301 and the flash memory 316. A RAM 318 is connected to the system controller 317 via a bus.

Furthermore, a bus 319 is connected to the controller 301. The bus 319 is connected to an external control module.
Further, a switch (SW) 320 is connected to the controller 301 via a bus. An electrical double layer capacitor unit (SCU) 321 that is a backup power source is connected to the switch 320 via a bus. Furthermore, buses 322 and 323 are connected to the switch 320. The bus 322 is connected to an external system ID (Identity) ROM. The bus 323 is connected to an external control module. For example, I ² C (trademark) is used for the bus 323.

Furthermore, a bus 324 is connected to the controller 301. The bus 324 is connected to an external control module. For example, I ² C (trademark) is used for the bus 324. Furthermore, SAS buses 325 and 326 are connected to the controller 301. The buses 325 and 326 are connected to an expander in an external control module.

Further, an expander 329 is connected to the controller 301 via buses 327 and 328 configured by SAS. A SAS port 330 is connected to the expander 329 via a bus. The SAS port 330 is composed of, for example, four ports.

The expander 329 is connected to a bus group 331 configured by SAS. The bus group 331 is composed of, for example, 12 buses. Each bus group 331 is connected to a hard disk drive (HDD). Further, buses 332 and 333 configured by SAS are connected to the expander 329. The buses 332 and 333 are connected to a controller in an external control module.

Further, buses 334 and 335 are connected to the expander 329. Buses 334 and 335 are connected to an external expander. For example, I ² C (trademark) is used for the buses 334 and 335. Furthermore, an EEPROM (Electrically Erasable Programmable ROM) 336 is connected to the expander 329 via a bus.

A switch 337 is connected between the expander 329 and the EEPROM 336. Buses 338 and 339 are connected to the switch 337. Buses 338 and 339 are connected to an external power supply unit. Further, a voltage / temperature detector 340 is connected to the switch 337 via a bus.

Furthermore, a driver 341 is connected to the expander 329 via a bus. A debug port 342 is connected to the driver 341 via a bus. Further, a flash memory 343 is connected to the expander 329 via a bus. A system controller 317 is connected between the expander 329 and the flash memory 343.

Next, the hardware of the drive enclosures 150a to 150c will be described. Since the drive enclosures 150a to 150c have the same configuration, the drive enclosure 150a will be described as a representative here.

FIG. 4 is a diagram illustrating an example of hardware of the drive enclosure according to the second embodiment.
The drive enclosure 150a includes an expander 400, an expander 410, and a back panel 420. The back panel 420 includes a hard disk drive group 421. The hard disk drive group 421 includes, for example, 24 hard disk drives.

The expander 400 has an expander circuit 401. The expander circuit 401 is connected to a multiplexer 402 through a bus. A SAS port 403 is connected to the multiplexer 402 via a bus. The SAS port 403 is composed of, for example, four ports. The SAS port 403 is connected to an external control module.

Further, a SAS port 404 is connected to the expander circuit 401. The SAS port 404 is composed of, for example, four ports. The SAS port 404 is connected to an external drive enclosure.

Furthermore, a ROM 405, a flash memory 406, and a controller 407 are connected to the expander circuit 401 via a bus. Further, the expander circuit 401 is connected to each hard disk drive of the hard disk drive group 421 via a bus.

The expander 410 has an expander circuit 411. A multiplexer 412 is connected to the expander circuit 411 via a bus. A SAS port 413 is connected to the multiplexer 412 via a bus. The SAS port 413 is composed of, for example, four ports. The SAS port 413 is connected to an external control module.

Further, a SAS port 414 is connected to the expander circuit 411. The SAS port 414 is configured by, for example, four ports. The SAS port 414 is connected to an external drive enclosure.

Furthermore, a ROM 415, a flash memory 416, and a controller 417 are connected to the expander circuit 411 via a bus. Further, the expander circuit 411 is connected to each hard disk drive of the hard disk drive group 421 via a bus.

The expander circuit 401 and the expander circuit 411 are connected by a bus 430 and a bus 431 configured by SAS. For example, I ² C (trademark), Ethernet (registered trademark), or the like is used for the bus 430. The drive enclosure 150a is provided with power supply units (PSU: 440, 441).

Next, an example of data transmission when no abnormality has occurred in the storage system 100 will be described. FIG. 5 is a diagram illustrating an example of data transmission of the storage system according to the second embodiment.

When no abnormality has occurred in the storage system 100, data transmission between the controller 114 and each of the storage devices 113 and 155a to 155c is performed via the following data transmission path. That is, data transmission is performed via the data transmission path 131 that passes through all or part of the expanders 115 and 151a to 151c.

In addition, data transmission between the controller 117 and each of the storage devices 113, 155a to 155c is performed via a data transmission path 132 that passes through all or part of the expanders 118, 153a to 153c.

Note that the inter-expander data transmission paths 158a to 158c are all blocked.
Here, each of the expanders 151a to 151c in FIG. 5 is realized by, for example, the expander 400 in FIG. 4, and each of the expanders 153a to 153c in FIG. 5 is realized by, for example, the expander 410 in FIG. Further, each of the storage devices 155a to 155c in FIG. 5 is realized by, for example, the hard disk drive group 421 in FIG.

FIG. 6 is a diagram illustrating a comparative example of storage systems.
FIG. 6 is a diagram illustrating an example of data transmission in a storage system in which all the inter-expander data transmission paths 158a to 158c are not blocked.

As shown in FIG. 6, in the storage system 100a, for example, when the inter-expander data transmission paths 158b and 158c are opened, a data transmission path 133 in a loop state is generated. In this case, for example, a SAS protocol violation occurs.

In the storage system 100, when no abnormality occurs, the inter-expander data transmission paths 158a to 158c are all blocked, so that it is possible to suppress the occurrence of a loop data transmission path.

Next, control tables stored in the memories 116, 119, 152a to 152c, and 154a to 154c will be described. FIG. 7 is a diagram illustrating an example of a control table according to the second embodiment.

FIG. 7A shows a control table 171 stored in the memory 116. FIG. 7B shows a control table 172 stored in the memory 119. FIG. 7C shows a control table 173 stored in the memory 152a. FIG. 7D is a control table 174 stored in the memory 154a. FIG. 7E shows a control table 175 stored in the memory 152b. FIG. 7F is a control table 176 stored in the memory 154b. FIG. 7G shows a control table 177 stored in the memory 152c. FIG. 7H is a control table 178 stored in the memory 154c. Here, a case where no abnormality has occurred in the storage system 100 as shown in FIG. 5 is taken as an example.

In each of the control tables 171 to 178, “state (1)” indicating a “port” that identifies the expander, a connection state between the expander and the controller 114 or the controller 117, or a connection state between the expanders. Is memorized. Furthermore, in each of the control tables 171 to 178, “Disk” for specifying the storage device and “State (2)” indicating the connection state between the expander and the storage device are stored.

Each of the control tables 173, 175, and 177 stores a connection state between the corresponding expander and the controller 114 or the controller 117, and a connection state between the corresponding expander and the storage device. Further, each of the control tables 173, 175, and 177 includes a connection state between the corresponding expander and the opposite expander in the same drive enclosure, and a connection state between the opposite expander and the storage device. Remembered.

The connection state between the corresponding expander and the opposite expander in the same drive enclosure is detected, for example, by the corresponding expander monitoring the open state of the data transmission path between the connected expanders. Is done. Alternatively, for example, the corresponding expander is detected by communicating with the opposite expander via the communication path.

Further, this control table includes a control table possessed by a lower expander connected to the corresponding expander.
For example, in the example shown in FIG. 5, regarding the drive enclosure 150a, the expander 151a and the controller 114 are connected, and the expander 151a and the storage device 155a are connected.

Therefore, in the control table 173 stored in the expander 151a, “state (1)” is “Enable” and “Disk” is “Enable” with respect to “DE (1) EXP (0)” indicating the expander 151 a. “B”, and “state (2)” becomes “Enable”.

In the example shown in FIG. 5, the inter-expander data transmission path 158a is closed, and the expander 153a and the expander 151a are not connected. Therefore, in the control table 173, “state (1)” becomes “Disable” with respect to “DE (1) EXP (1)” indicating the expander 153a.

Further, the control table 173 also stores a control table 175 included in the lower expander 151b connected to the expander 151a. That is, in the control table 173, the rows “DE (2) EXP (0)” to “DE (3) EXP (1)” correspond to the control table 175.

Each of the control tables 174, 176, and 178 stores a connection state between the corresponding expander and the controller 114 or the controller 117, and a connection state between the expander and the storage device. Further, each of the control tables 174, 176, and 178 includes a connection state between the corresponding expander and the opposite expander in the same drive enclosure, and a connection state between the opposite expander and the storage device. Remembered.

The connection state between the corresponding expander and the opposite expander in the same drive enclosure is detected, for example, by the corresponding expander monitoring the open state of the data transmission path between the connected expanders. Is done. Alternatively, for example, the corresponding expander is detected by communicating with the opposite expander via the communication path.

Further, this control table includes a control table possessed by a lower expander connected to the expander.
For example, in the example shown in FIG. 5, regarding the drive enclosure 150a, the expander 153a and the controller 117 are connected, and the expander 153a and the storage device 155a are connected.

Therefore, in the control table 174 stored in the expander 153a, “state (1)” is “Enable” and “Disk” is “Enable” with respect to “DE (1) EXP (1)” indicating the expander 153 a. “B”, and “state (2)” becomes “Enable”.

In the example shown in FIG. 5, the inter-expander data transmission path 158a is closed, and the expander 151a and the expander 153a are not connected. Therefore, “state (1)” becomes “Disable” with respect to “DE (1) EXP (0)” indicating the expander 151a.

Further, the control table 174 also stores a control table 176 included in the lower expander 153b connected to the expander 153a. That is, in the control table 174, the lines “DE (2) EXP (1)” to “DE (3) EXP (0)” correspond to the control table 176.

The control table 171 stores a connection state between the expander 115 and the controller 114 or the controller 117 and a connection state between the expander 115 and the storage device 113. Further, the control table 171 includes a control table 173 included in a lower expander 151 a connected to the expander 115.

For example, in the example illustrated in FIG. 5, the expander 115 and the controller 114 are connected, and the expander 115 and the storage device 113 are connected.
Therefore, in the control table 171 stored in the expander 115, “status (1)” becomes “Enable” and “Disk” becomes “A” for “CM (0) EXP” indicating the expander 115. “Status (2)” becomes “Enable”. Further, the control table 171 also stores a control table 173 included in the lower expander 151 a connected to the expander 115. That is, in the control table 171, the lines “DE (1) EXP (0)” to “DE (3) EXP (1)” correspond to the control table 173.

The control table 172 stores a connection state between the expander 118 and the controller 114 or the controller 117 and a connection state between the expander 118 and the storage device 113. Further, the control table 172 includes a control table 174 included in the lower expander 153 a connected to the expander 118.

For example, in the example shown in FIG. 5, the expander 118 and the controller 117 are connected, and the expander 118 and the storage device 113 are connected.
Therefore, in the control table 172 stored in the expander 118, “status (1)” becomes “Enable” and “Disk” becomes “A” for “CM (1) EXP” indicating the expander 118. “Status (2)” becomes “Enable”. Further, the control table 172 also stores a control table 174 included in the lower expander 153 a connected to the expander 118. That is, in the control table 172, the lines “DE (1) EXP (1)” to “DE (3) EXP (0)” correspond to the control table 174.

Next, the processing procedure of the controllers 114 and 117 will be described.
FIG. 8 is a flowchart illustrating an example of a processing procedure of the controller according to the second embodiment. 9, 11, 13, and 15 are diagrams illustrating an example of the state of the storage system according to the second embodiment. 10, FIG. 12, and FIG. 14 are diagrams illustrating an example of a control table according to the second embodiment. The processing procedure will be described below with reference to FIGS. 9 to 15 along the flowchart of FIG.

The processing shown below is started when the controller 114 or the controller 117 receives a notification of a connection state change from any one of the expanders 115, 118, 151a to 151c, and 153a to 153c.

Since the controllers 114 and 117 perform the same processing, the case where the controller 114 mainly performs processing will be described here as a representative.
Further, as an initial state, it is assumed that the inter-expander data transmission paths 158a to 158c are all closed and the controller 114 has collected all the control tables 171 to 178 in advance.

[Step S100] The controller 114 updates the control table for all the expanders connected to the data transmission path to which the expander that has been notified of the connection state change is connected among the data transmission paths 121 and 122. Instruct.

For example, as shown in FIG. 9, when an abnormality occurs in the expander 151c and the expander 153a, the controller 114 receives the notification of the connection state change from the expander 151b and the expander 118, and next Perform the process. At this time, the controller 117 also receives a notification of a connection state change from the expander 151b and the expander 118.

That is, the controller 114 instructs the expanders 115, 151a to 151c connected to the data transmission path 121 to update the control tables 171, 173, 175, 177. Further, the controller 114 instructs the expanders 118, 153a to 153c connected to the data transmission path 122 to update the control tables 172, 174, 176, and 178.

FIG. 10 is an example of the updated control tables 171 to 178. FIG. 10A shows the control table 171. FIG. 10B shows the control table 172. FIG. 10C shows the control table 173. FIG. 10D shows the control table 174. FIG. 10E shows the control table 175. FIG. 10F is a control table 176. FIG. 10G shows the control table 177. FIG. 10H shows the control table 178.

When each of the expanders 115, 118, 151a to 151c, 153a to 153c receives an update instruction for the control tables 171 to 178 from the controller 114, for example, the data stored in the control tables 171 to 178 are deleted. Thereafter, each of the expanders 115, 118, 151a to 151c, and 153a to 153c detects a new connection state, and stores the detected connection state in each of the control tables 171 to 178. The update instruction of each control table 171 to 178 from the controller 114 to each expander 115, 118, 151a to 151c, 153a to 153c is, for example, an instruction path provided separately from the data transmission paths 121 and 122. Done with.

Since the data transmission path between the expander 151c and the expanders 153a to 153c is disconnected from the controller 114, the data stored in the control tables 174, 176, 177, and 178 is deleted. No data relating to the new connection state is stored. Alternatively, at this time, each control table 174, 176, 177, 178 may store data indicating that the connection state is invalid after the stored data is deleted.

[Step S101] The controller 114 collects all the control tables updated in step S100. The collected control table is stored in the memory 161.
[Step S102] The controller 114 refers to the collected control tables 171 to 178, and among the storage devices 113 and 155a to 155c, there is a storage device that cannot access data from either the data transmission path 121 or the data transmission path 122. It is determined whether or not to do. If it exists, the controller 114 proceeds with the process to step S103. If not, the controller 114 ends the process.

Here, when there is a storage device inaccessible to data, that is, communication between at least one of the expanders 115 and 151a to 151c and the controller 114 is interrupted, and among the expanders 118 and 153a to 153c. This is a case where communication between at least one of the above and the controller 114 is interrupted.

In the case of the example shown in FIG. 9, the controller 114 refers to the control tables 171 to 178 shown in FIG. 10, and determines that the data access from the controller 114 is impossible for the storage device 155c.

That is, since the controller 114 has lost communication with the expander 151c for the data transmission path 121 and has lost communication with the expander 153b for the data transmission path 122, data cannot be accessed for the storage device 155c. Is determined.

[Step S103] If it is determined that there is a storage device inaccessible to data, the controller 114 refers to the collected control tables 171 to 178 and refers to the collected data in the data transmission path 121 or the data transmission path 122. Detect pandas.

In the example shown in FIG. 9, the controller 114 refers to the control tables 171 to 178 shown in FIG. 10, and among the “ports” whose “state (1)” is “Enable”, “ DE (2) EXP (0) "is detected. Thus, the controller 114 detects the expander 151b as the farthest expander that can communicate.

[Step S104] The controller 114 instructs the expander at the farthest end detected in Step S103 to open the connected inter-expander data transmission path. As a result, the inter-expander data transmission path connected to the expander at the farthest end is opened.

In the example shown in FIG. 9, since the farthest end expander is the expander 151b, the controller 114 instructs the expander 151b to open the inter-expander data transmission path 158b. The opening instruction is performed, for example, through the data transmission path 121 or an instruction path provided elsewhere. As a result, the inter-expander data transmission path 158b is opened as shown in FIG. By opening the inter-expander data transmission path 158b, the expanders 153b and 153c and the controller 114 can communicate with each other via the inter-expander data transmission path 158b.

[Step S105] The controller 114 instructs the expanders 115, 118, 151a to 151c, and 153a to 153c to update the control tables 171 to 178.

FIG. 12 is an example of the control tables 171 to 178 updated based on the instruction in step S105 in the state shown in FIG. FIG. 12A shows the control table 171. FIG. 12B shows the control table 172. FIG. 12C shows the control table 173. FIG. 12D shows the control table 174. FIG. 12E shows the control table 175. FIG. 12F shows the control table 176. FIG. 12G shows the control table 177. FIG. 12H shows the control table 178.

Since the inter-expander data transmission path 158b is opened, the expander 153b and the expander 151b are connected. For example, the control table 175 is updated as follows. That is, in the control table 175, the “state (1)” of “DE (2) EXP (1)” that is “Disable” in FIG. 10 becomes “Enable”. Further, since communication between the expanders 153b and 153c and the controller 114 becomes possible, the new connection state is stored in the control tables 176 and 178, in which data is not stored in FIG.

[Step S106] The controller 114 collects all the control tables 171 to 178. The collected control tables 171 to 178 are stored in the memory 161.
[Step S107] The controller 114 refers to the collected control tables 171 to 178, and determines whether or not there is one data transmission path between the storage device and the controller 114 for each of the storage devices 113 and 155a to 155c. Determine. If there is one data transmission path, the controller 114 ends the process. When the number of data transmission paths is not one, that is, when the data transmission paths for the same storage device overlap, the controller 114 proceeds with the process to step S108.

In the case of the example shown in FIG. 11, the controller 114 refers to the control tables 171 to 178 shown in FIG. For example, in the control table 175, the controller 114 sets “Disk” to “C” for “DE (2) EXP (0)” and “DE (2) EXP (1)”, and “Status (2 ) ”Are both“ Enable ”.

Thereby, the controller 114 determines that the data transmission path via the expander 151b and the data transmission path via the expander 153b overlap with the storage device 155b.

[Step S108] The controller 114 blocks the data transmission path that passes through the distant expander among the overlapping data transmission paths to the storage device. Here, the far expander refers to an expander having a larger number of stages from the controller 114.

In the case of the example shown in FIG. 11, the expander 153b (fourth stage) is farther than the expander 151b (third stage), that is, the number of stages from the controller 114 is larger. For this reason, the controller 114 blocks the data transmission path to the storage device 155b via the expander 153b as shown in FIG. That is, the controller 114 closes the data transmission path between the expander 153b and the storage device 155b.

[Step S109] The controller 114 instructs the expanders 115, 118, 151a to 151c, and 153a to 153c to update the control tables 171 to 178, and ends the processing.

FIG. 14 is an example of the control tables 171 to 178 updated in the state shown in FIG. FIG. 14A shows the control table 171. FIG. 14B shows the control table 172. FIG. 14C shows the control table 173. FIG. 14D shows the control table 174. FIG. 14E shows the control table 175. FIG. 14F is a control table 176. FIG. 14G shows the control table 177. FIG. 14H is the control table 178.

Since the data transmission path between the expander 153b and the storage device 155b is blocked, for example, in the control table 175, the “state (2)” of “DE (2) EXP (1)” becomes “Disable”.

As a result of the above processing, even when an abnormality occurs in the expander 151c and the expander 153a as shown in FIG. 9, the controller 114 accesses the storage devices 113 and 155a to 155c as shown in FIG. be able to.

That is, as shown in FIG. 15, data transmission between the controller 114 and each of the storage devices 113 and 155a to 155c is performed via the data transmission path 141. That is, the storage device 155c is accessed through the expander 152b, the inter-expander data transmission path 158b, the expander 153b, and the expander 153c. Similarly, data transmission between the controller 117 and each of the storage devices 113 and 155a to 155c is performed via the data transmission path 142. In other words, the controller 117 performs the storage device 155a to 155c via the expander 115 without passing through the expander 153a in which an abnormality has occurred.

Next, the processing procedure of each of the expanders 151a to 151c and 153a to 153c will be described.
FIG. 16 is a flowchart illustrating an example of an expander processing procedure according to the second embodiment.

The processing shown below is started when each of the expanders 151a to 151c and 153a to 153c detects an instruction to open an inter-expander data transmission path from the controller 114 or the controller 117.

Since the expanders 151a to 151c and 153a to 153c perform the same processing, the expander 151a will be described as a representative here.
[Step S200] The expander 151a performs processing for enabling the inter-expander port 156a of its own apparatus.

[Step S201] The expander 151a determines whether or not the processing for enabling the inter-expander port 156a of its own device has succeeded. If the process is successful, the expander 151a advances the process to step S202. If the process is not successful, the expander 151a advances the process to step S206.

[Step S202] The expander 151a prepares for communication with the expander 153a, which is the opposite device, via the communication path 159a.
[Step S203] The expander 151a performs processing for enabling the inter-expander port 157a of the expander 153a, which is the opposite device, via the communication path 159a. When the expander 153a enables the inter-expander port 157a, the inter-expander data transmission path 158a is opened.

[Step S204] The expander 151a determines whether the opening process of the inter-expander data transmission path 158a is successful. When the opening process of the inter-expander data transmission path 158a is successful, the expander 151a advances the process to step S205. When the opening process of the inter-expander data transmission path 158a is not successful, the expander 151a advances the process to step S206.

[Step S205] The expander 151a notifies the controller 114 or the controller 117 of the success of the opening process of the inter-expander data transmission path 158a and ends the process.

[Step S206] The expander 151a notifies the controller 114 or the controller 117 of the failure of the opening process of the inter-expander data transmission path 158a and ends the processing.

Next, the procedure for opening the inter-expander data transmission paths 158a to 158c will be described with reference to a sequence diagram.
FIG. 17 is a sequence diagram illustrating an example of a procedure for opening an inter-expander data transmission path according to the second embodiment.

The following processing is started when each of the expanders 151a to 151c and 153a to 153c detects a change in the connection state. Here, as an example, it is assumed that any of the expanders 151a to 151c connected to the data transmission path 121 has detected a change in the connection state. It is assumed that the expanders 153a to 153c connected to the data transmission path 122 have not detected a change in connection state.

As an initial state, the inter-expander data transmission paths 158a to 158c are all blocked. Further, since the controllers 114 and 117 perform the same processing, here, a case where the controller 114 mainly performs processing will be described as a representative.

[Step S300] Among the expanders 151a to 151c, the expander that has detected the change in the connection state notifies the controller 114 of the change in the connection state.
[Step S301] The controller 114 instructs the expanders 151a to 151c to update the control table.

[Step S302] Each of the expanders 151a to 151c updates the control tables 173, 175, and 177.
[Step S303] The expanders 151a to 151c notify the controller 114 of the data in the control tables 173, 175, and 177.

[Step S304] The controller 114 requests the expander at the farthest end capable of communication (hereinafter referred to as the farthest end expander) to open the connected inter-expander data transmission path. Here, any of the expanders 151a to 151c is assumed to be the farthest end expander.

[Step S305] Upon receiving a request from the controller 114, the farthest end expander validates the inter-expander port of its own device.
[Step S306] The farthest end expander requests a corresponding expander (hereinafter referred to as a counter device) among the expanders 153a to 153c to validate the inter-expander port.

[Step S307] The opposing device validates the inter-expander port of its own device.
[Step S308] The opposite device notifies the farthest end expander of the completion of the process of enabling the inter-expander port of the own device.

[Step S309] The farthest end expander notifies the controller 114 that the inter-expander data transmission path has been successfully opened.
[Step S310] In accordance with the opening process of the inter-expander data transmission path, the expander that detects the change in the connection state among the expanders 151a to 151c and 153a to 153c notifies the controller 114 of the change in the connection state.

[Step S311] In response to the notification of the connection state change, the controller 114 instructs the expanders 151a to 151c and 153a to 153c to update the control tables 173 to 178.

[Step S312] The expanders 151a to 151c and 153a to 153c update the control tables 173 to 178.
[Step S313] The expanders 151a to 151c and 153a to 153c notify the controller 114 of the control tables 173 to 178.

As described above, in the storage system 100, when the controller 114 or the controller 117 detects a storage device that cannot access data from either the data transmission path 121 or the data transmission path 122, the following processing is executed. In the storage system 100, even if the storage device itself has failed, the storage device itself has failed or the data transmission path between the expander and the storage device has been interrupted. Since it is not possible to determine whether or not, the following processing is executed.

That is, the controller 114 or the controller 117 transmits a communication opening instruction via at least one data transmission path among the inter-expander data transmission paths 158a to 158c to any one of the expanders 151a to 151c and 153a to 153c. To do.

Thus, even when a failure or the like occurs in the data transmission path 121 and the data transmission path 122 due to an abnormality in a plurality of expanders, data access from the controller 114 or the controller 117 to each of the storage devices 113 and 155a to 155c is possible. It becomes possible.

In the storage system 100, the inter-expander data transmission paths 158a to 158c are controlled to open and close in accordance with an instruction from the controller 114 or the controller 117. Thereby, it can suppress that the data transmission path | route of a loop state arises.

Further, in the storage system 100, the controller 114 or the controller 117 selects the inter-expander data transmission path connected to the farthest expander that can communicate among the plurality of inter-expander data transmission paths 158a to 158c. Open.

The expander at the farthest end is detected by the controller 114 or the controller 117 referring to the control tables 171 to 178 and specifying the expander having the largest number of stages from the controller 114 or the controller 117.

According to this, the data transmission path to the communicable part of the data transmission path 121 or the data transmission path 122 can be used to the maximum extent. Thereby, it is possible to efficiently generate a data access path from the controller 114 or the controller 117 to each of the storage devices 113 and 155a to 155c.

In this way, in the storage system 100, it is possible to improve the reliability of data access to the storage devices 113 and 155a to 155c.
Here, another example in which an abnormality has occurred in the storage system 100 will be described.

18 and 19 are diagrams illustrating an example of the state of the storage system according to the second embodiment.
In this example, as shown in FIG. 18, an abnormality occurs in the data transmission path 121, the expander 151c, and the expander 153c between the controller 114 and the expander 115. Of these, maintenance of the expander 151c is performed. Is going.

In this case, in this state, the controller 114 can access the storage devices 113, 155a, and 155b via the data transmission path 134, but cannot access the storage device 155c.

In this case, in the storage system 100, the inter-expander data transmission path 158b connected to the farthest expander 153b that can communicate in the data transmission path 122 is opened. For this reason, as shown in FIG. 19, the controller 114 can access the storage device 155 c via the data transmission path 135.

[Third Embodiment]
Next, another embodiment of the drive enclosure applied to each drive enclosure 150a to 150c will be described as a third embodiment.

FIG. 20 is a diagram illustrating an example of a drive enclosure according to the third embodiment.
The drive enclosure 200 includes an expander 201 connected to the data transmission path 121, an expander 202 connected to the data transmission path 122, and inter-expander data transmission paths 203 and 204.

The drive enclosure 200 differs from the drive enclosures 150a to 150c of the second embodiment shown in FIG. 2 in the following three points. The first point is that there are two inter-expander data transmission paths 203 and 204. The second point is that no communication path is provided. The third point is that it has four inter-expander ports 205, 206, 207, 208. Other configurations of the drive enclosure 200 are the same as those of the drive enclosures 150a to 150c. That is, each expander 201, 202 has memories 209, 210, and a storage device 211 is connected to each expander 201, 202.

The expander 201 has inter-expander ports 205 and 206, and the expander 202 has inter-expander ports 207 and 208. The expander 201 can independently control the process of disabling and enabling the inter-expander ports 205 and 206 between the inter-expander ports 205 and 206. The expander 202 can independently control the inter-expander ports 207 and 208 to disable and enable the inter-expander ports 207 and 208.

For example, SAS is used for the inter-expander data transmission paths 203 and 204. The inter-expander data transmission path 203 is connected to the inter-expander port 205 and the inter-expander port 207, and the inter-expander data transmission path 204 is connected to the inter-expander port 206 and the inter-expander port 208.

In the initial state, the inter-expander ports 205 and 208 are valid, and the inter-expander ports 206 and 207 are invalid. That is, in the initial state, the inter-expander data transmission path 203 is valid between the expander 201 and invalid between the expander 202. The inter-expander data transmission path 204 is in an invalid state between the expander 201 and the expander 202 in an initial state. That is, the inter-expander data transmission paths 203 and 204 are both closed in the initial state.

Next, the processing procedure of each expander 201, 202 will be described. Since the expanders 201 and 202 perform the same processing, the processing procedure of the expander 201 will be described here as a representative.

FIG. 21 is a flowchart illustrating an example of an expander processing procedure according to the third embodiment.
The processing shown below is started when the expander 201 detects an instruction to open an inter-expander data transmission path from the controller 114 or the controller 117. The case where the opening instruction is detected from the controller 117 means that the controller 117 is connected to the data transmission path 121 via the expander 115.

As described above, in the initial state, the inter-expander ports 205 and 208 are valid, and the inter-expander ports 206 and 207 are invalid.
[Step S400] In response to the opening instruction from the controller 114 or the controller 117, the expander 201 enables the inter-expander port 206 that is in an invalid state of its own apparatus. As a result, the inter-expander data transmission path 204 is opened.

[Step S401] The expander 201 determines whether or not the process of enabling the inter-expander port 206 of its own device has succeeded. If the process is successful, the expander 201 advances the process to step S402. In this case, the inter-expander data transmission path 204 is open. If the process is not successful, the expander 201 advances the process to step S405.

[Step S402] The expander 201 performs a process of enabling the inter-expander port 207 of the expander 202 which is the opposite device via the opened inter-expander data transmission path 204. As a result, the inter-expander data transmission path 203 is opened.

[Step S403] The expander 201 determines whether the opening process of the inter-expander data transmission path 203 is successful. If successful, the expander 201 advances the process to step S404. If unsuccessful, the expander 201 advances the process to step S405.

[Step S404] The expander 201 notifies the controller 114 or 117 of the success of the opening process and ends the process.
[Step S405] The expander 201 notifies the controller 114 or 117 of the failure of the opening process and ends the process.

Note that when only the inter-expander data transmission path 204 is opened, the process of enabling the inter-expander port 207 need not be performed.
As described above, in the drive enclosure 200, the inter-expander data transmission path 203 is in the valid state between the expander 201 and the inter-expander data transmission path 203 in the invalid state with the expander 202 in the initial state. including. Further, the inter-expander data transmission path includes an inter-expander data transmission path 204 that is in a valid state with the expander 202 and in an invalid state with the expander 201 in the initial state.

Then, each of the expanders 201 and 202 opens the inter-expander data transmission path by enabling the inter-expander data transmission paths 203 and 204 with the expander in the invalid state.

According to this, a data transmission path between the expanders can be opened without providing a communication path for transmitting a signal for opening processing between the expander 201 and the expander 202.

The above merely shows the principle of the present invention. In addition, many modifications and changes can be made by those skilled in the art, and the present invention is not limited to the precise configuration and application shown and described above, and all corresponding modifications and equivalents may be And the equivalents thereof are considered to be within the scope of the invention.

10, 100 Storage system 11a, 11b, 11c, 113, 155a, 155b, 155c, 211 Storage device 12, 114, 117 Controller 13, 14, 17a, 17b, 17c, 19, 121, 122, 131, 132, 133 134, 135, 141, 142 Data transmission path 15a, 15b, 15c, 16a, 16b, 16c Relay device 18a, 18b, 18c Storage device 110 Controller enclosure 111, 112 Control module 115, 118, 151a, 151b, 151c, 153a, 153b, 153c, 201, 202 Expander 116, 119, 152a, 152b, 152c, 154a, 154b, 154c, 161, 164, 209, 210 Memory 150a, 150 , 150c, 200 Drive enclosure 156a, 156b, 156c, 157a, 157b, 157c, 205, 206, 207, 208 Inter-expander port 158a, 158b, 158c, 203, 204 Inter-expander data transmission path 159a, 159b, 159c Path 160, 163 Control circuit 162, 165 Communication port 171, 172, 173, 174, 175, 176, 177, 178 Control table

Claims (11)

1. A storage device, a first relay device connected to the first data transmission path and relaying data access to the storage device, and a second data transmission path connected to the storage device, and open / close control according to an instruction from the outside A plurality of storage devices each having a second relay device that is connected to the first relay device via a third data transmission path and relays data access to the storage device;
   Connected to a first data transmission path in which the first relay devices of each of the plurality of storage devices are connected in series, and to a second data transmission path in which the second relay devices of the plurality of storage devices are connected in series The communication status of each of the plurality of storage devices via the third data transmission path is blocked, and the connection status to the storage device via the first data transmission path and the second data transmission path is determined. When monitoring and detecting a storage device that is inaccessible to data from either the first data transmission path or the second data transmission path, an instruction to open communication via the third data transmission path of at least one storage apparatus And a controller to send
   Storage system.
2. When the controller detects a storage device that is inaccessible to data from either the first data transmission path or the second data transmission path, the controller determines either the first data transmission path or the second data transmission path. 2. The storage system according to claim 1, wherein the farthest end storage device that can communicate with the storage device is detected, and communication via the third data transmission path of the detected storage device is opened.
3. Each of the plurality of storage devices has a control table for managing data of devices communicable via the first data transmission path and the second data transmission path,
   The controller collects data of the control table from the plurality of storage devices, and detects a farthest end storage device that can communicate based on the collected data of the control table. The storage system according to item 2.
4. In the storage device, the first relay device and the second relay device are connected by a communication path different from the third data transmission path, and the first relay device and the second relay device are When receiving the opening instruction from the controller, the opening instruction from the controller is transmitted to the other relay device via the communication path, and the connection with each of the third data transmission paths is made effective. The storage system according to any one of claims 1 to 3, wherein the storage system is characterized in that:
5. The third data transmission path of the storage device is
   In an initial state, the connection between the connected first relay apparatus and the third data transmission path is in an effective state, and the connection between the connected second relay apparatus and the third data transmission path is invalid. A fourth data transmission path that is open when both ends are in a valid state after transition from the initial state,
   In an initial state, the connection between the connected second relay apparatus and the third data transmission path is in an effective state, and the connection between the connected first relay apparatus and the third data transmission path is invalid A fifth data transmission path that is open when both ends are in the valid state after transitioning from the initial state.
   When the first relay device and the second relay device of the storage apparatus receive the opening instruction from the controller, the data transmission path and the own apparatus among the fourth data transmission path and the fifth data transmission path Opening the third data transmission path by changing the effective state between the data transmission path in the invalid state and the own device;
   The storage system according to any one of claims 1 to 3, wherein:
6. The controller, with respect to the storage apparatus that has issued a third data transmission path opening instruction, out of the first relay apparatus and the second relay apparatus of the storage apparatus from the relay apparatus far from the controller. Instructing to close the connection to the storage device in the storage device,
   The first relay device and the second relay device of the storage device block the connection with the storage device in the storage device in response to an instruction to block the connection to the storage device from the controller. The storage system according to any one of claims 1 to 5, wherein the storage system is characterized in that:
7. A storage device;
   A first relay device connected to the first data transmission path and relaying data access to the storage device;
   A second data relay is connected to the first relay device via a third data transmission route that is connected to the second data transmission route and controlled to be opened and closed according to an instruction from the outside, and relays data access to the storage device. Relay device,
   A storage device.
8. The first relay device and the second relay device are connected by a communication path different from the third data transmission path, and the first relay device and the second relay device are When receiving an instruction to open the third data transmission path, the opening instruction is transmitted to the other relay device via the communication path, and the connection with the third data transmission path is enabled. The storage device according to claim 7.
9. The third data transmission path is:
   The fourth relay is opened when the connection with the first relay device connected is in the valid state, the connection with the second relay device connected is in the invalid state, and both ends are in the valid state. A data transmission path;
   The fifth relay is opened when the connection with the connected second relay device is in the valid state, the connection with the connected first relay device is in the invalid state, and both ends are in the valid state. Including a data transmission path,
   When the first and second relay apparatuses receive an instruction to open the third data transmission path, the first and second relay apparatuses change the invalid state of the fourth or fifth data transmission path to the valid state. Opening the third data transmission path;
   The storage apparatus according to claim 7, wherein:
10. A plurality of storage devices and a set of first relay devices and second relay devices that are redundantly connected to the storage devices and each relay data access to the storage devices are provided,
    A plurality of first relay devices are connected in series via a first data transmission path,
    A plurality of second relay devices are connected in series via the second data transmission path,
    The first relay device and the second relay device that constitute each set are connected by a third data transmission path that can be controlled to open and close,
    In the controller that controls the storage system,
    A first communication port connected to the first data transmission path;
    A second communication port connected to the second data transmission path;
    Blocking communication through the third data transmission path of each of the plurality of relay apparatuses, monitoring the connection status to the storage device through the first data transmission path and the second data transmission path, Control for opening communication via a third data transmission path of at least one set of relay devices when a storage device that is inaccessible to data is detected from either the first data transmission path or the second data transmission path Circuit,
    The controller characterized by having.
11. When communication with at least one first relay device is interrupted and communication with at least one second relay device is interrupted, the control circuit causes the first data transmission path or the second data transmission path to And detecting a third data transmission path between the first relay device and the second relay device constituting the detected pair. The controller according to claim 10, wherein the controller is opened.

Images