JP2021170261A - Storage control device and control program - Google Patents

Abstract

To enable a storage system having a redundant path configuration to execute efficient preventive maintenance.SOLUTION: A storage control device includes: a disconnection processing part for disconnecting a connection with a first module 22 in a pseudo manner if a recoverable failure occurs in the first module 22 in a primary path; a connection processing part for validating a connection with a second module 22 in a secondary path to execute an access test; and a specification part for specifying a cause of the failure according to a result of the access test by the connection processing part.SELECTED DRAWING: Figure 7

Description

The present invention relates to a storage control device and a control program.

In recent enterprise Redundant Arrays of Independent Disks (RAID) controllers, the data transfer communication path between the RAID controller and the disk may be redundant. As a result, even if a failure occurs in one of the data transfer communication paths, the operation can be continued in the remaining data transfer communication path.

Japanese Unexamined Patent Publication No. 2009-20513 Japanese Unexamined Patent Publication No. 2006-164304

However, even in a device in which the data transfer communication path is made redundant, the operation may be stopped in the event of a double failure of the data transfer communication path.

FIG. 1 is a diagram illustrating preventive maintenance in the RAID system 600.

Preventive maintenance is the preventive replacement of parts that can be maintained and replaced in the device and that can be operated but have fallen into a minor failure state. Preventive maintenance processes include, for example, disconnection of defective parts and replacement of defective parts.

The RAID system 600 includes a plurality of Controller Modules (CM) 6 (may be referred to as CM # 0 and # 1) and a plurality of (three in the illustrated example) Disk Enclosure (two in the illustrated example). DE) 7 (may be referred to as DE # 01 to # 03) is provided.

The DE7 includes a plurality of discs 71 and two Input Output Modules (IOMs) 72 (may be referred to as IOM # 0, # 1). In FIG. 1, for the sake of simplicity, only one disk 71 is shown in each DE7.

The CM6 includes a Central Processing Unit (CPU) 61 and an Expander (EXP) 62. The CPU 61 functions as an Input Output Controller (IOC) 601.

Here, in a redundant back-end path configuration, if a normal component is mistakenly maintained and disconnected after the preventive maintenance of one IOM72 of the DE7 is started, both paths of the DE7 are blocked and all the disks 71 in the DE7 can be accessed. It disappears.

When such a phenomenon occurs, it is assumed that there is a one-point failure and a two-point failure. It is assumed that a single point failure is caused by a simple work mistake of a maintenance person. In addition, a two-point failure includes an abnormality in both paths, one point is a failure that cannot be operated and the suspected part cannot be identified, and the other point is a minor failure that can be continuously operated and the suspected part can be identified. Is assumed to be.

FIG. 1 shows the case of a two-point failure. IOM # 0 of DE # 02 is a failed component-1, and is a recoverable failure mode (in other words, a minor abnormality) subject to preventive maintenance. Further, IOM # 1 of DE # 02 is a failure component-2, which is a failure mode in which a silent failure cannot be recovered (in other words, a serious abnormality).

Hereinafter, with reference to FIG. 1, a state in which all routes are blocked as a result of accidentally maintaining and disconnecting a normal component will be described.

First, as shown by reference numeral A1, a silent failure, which is an abnormality that cannot be detected, occurs in IOM # 1 of DE # 02. Next, as shown by reference numeral A2, a recoverable failure occurs in IOM # 0 of DE # 02, and preventive maintenance is carried out. As a result, as shown by reference numeral A3, the route on the IOM # 0 side of each DE7 cannot be used. On the other hand, as shown by reference numeral A4, access to the disk 71 is started by the route on the IOM # 1 side of each DE7. Then, as shown by reference numeral A5, an abnormality due to a silent failure in IOM # 1 of DE # 02 becomes apparent, and both paths are blocked.

When the IOM72, which is an unrecoverable failure mode, is replaced, it is possible to continue the operation by using the route on the IOM72 side, which is a recoverable failure mode. However, as shown in FIG. 1, when preventive maintenance of IOM72, which is a recoverable failure mode, is performed, both paths become unusable.

In addition, patrols with data transfer may not be able to perform patrols with data transfer constantly because the host Input Output (IO) performance deteriorates.

One aspect is aimed at enabling efficient preventive maintenance in a storage system with a redundant path configuration.

On one side, the storage controller is a storage controller in which a plurality of storage devices are cascaded by a primary path and a secondary path, and a recoverable failure has occurred in the first module in the primary path. In this case, a disconnection processing unit that pseudo-disconnects the connection with the first module, a connection processing unit that enables an access test by enabling the connection with the second module in the secondary path, and the connection processing. A specific unit for identifying the cause of the failure is provided according to the result of the access test by the unit.

On one side, efficient preventive maintenance can be performed in a storage system with a redundant path configuration.

It is a figure which illustrates the preventive maintenance in a RAID system. It is a block diagram which shows typically the hardware configuration example of the RAID system in one example of Embodiment. It is a block diagram which shows typically the software structure example of CM shown in FIG. It is a figure explaining the detection process of the part subject to preventive maintenance in the RAID system shown in FIG. It is a table which shows the suspected part management information according to the detection result of the part subject to preventive maintenance shown in FIG. It is a figure explaining the separation process of the component subject to preventive maintenance in the RAID system shown in FIG. It is a figure explaining the 1st example of the access test process for the redundant path in the RAID system shown in FIG. It is a figure explaining the 2nd example of the access test process for the redundant path in the RAID system shown in FIG. It is a table which shows the suspected part management information according to the access test result for the redundant path shown in FIG. It is a flowchart explaining the preventive maintenance process in the RAID system shown in FIG.

Hereinafter, one embodiment will be described with reference to the drawings. However, the embodiments shown below are merely examples, and there is no intention of excluding the application of various modifications and techniques not specified in the embodiments. That is, the present embodiment can be variously modified and implemented within a range that does not deviate from the purpose.

Further, each figure does not mean that it includes only the components shown in the figure, but may include other functions and the like.

Hereinafter, since the same reference numerals indicate the same parts in the drawings, the description thereof will be omitted.

[A] Example of Embodiment [A-1] System Configuration Example FIG. 2 is a block diagram schematically showing a hardware configuration example of the RAID system 100 in the example of the embodiment.

The RAID system 100 is an example of a storage system, and has a plurality of (two in the illustrated example) CM1 (may be referred to as CM # 0 and # 1) and a plurality of (three in the illustrated example) DE2. (It may be referred to as DE # 01 to # 03.).

Each DE2 is cascade-connected to CM # 0 and # 1 by a primary path and a secondary path. The primary path connects CM # 0 in the order of DE # 01, # 02, # 03. The secondary path connects CM # 1 in the order of DE # 03, # 02, # 01.

DE2 is an example of a storage device group, and includes a plurality of disks 21 and two IOMs 22 (may be referred to as IOMs # 0 and # 1). In FIG. 2, for the sake of simplicity, only one disk 21 is shown in each DE2.

The disk 21 is an example of a storage device, and stores various information in response to a command from CM1. The IOM 22 is an example of a module and relays communication with CM1 or another DE2.

CM1 is an example of a storage control device, and includes a CPU 11 and an EXP 12.

EXP12 relays communication with each DE2 or CM1 of another system.

FIG. 3 is a block diagram schematically showing a software configuration example of CM1 shown in FIG.

The CPU 11 functions as an IOC 101 as shown in FIG. 2, and also functions as a cutting processing unit 111, a connection processing unit 112, and a specific unit 113 as shown in FIG.

The IOC101 controls communication with each DE2 or another CM1 system.

The disconnection processing unit 111 pseudo-disconnects the connection with the IOM 22 in the primary path when a recoverable failure occurs in the IOM 22 in the primary path. In the cascade connection of the primary path, the disconnection processing unit 111 may pseudo-disconnect the connection from the IOM 22 of the DE2 connected to the end of the plurality of DE2s in a pseudo manner.

The connection processing unit 112 performs an access test by enabling the connection with the IOM 22 in the secondary path. The connection processing unit 112 may perform an access test on the IOM 22 on the secondary path side of the DE2 in which the connection is pseudo-disconnected by the disconnection processing unit 111, with the sequential connection enabled. Further, the connection processing unit 112 may re-enable the connection with the IOM 22 of the primary path when the IOM 22 of the secondary path is identified as the cause of the failure by the identification unit 113.

The identification unit 113 identifies the cause of the failure according to the result of the access test by the connection processing unit 112. For example, the identification unit 113 identifies the IOM 22 of the primary path as the cause of the failure when the result of the access test for the IOM 22 of the secondary path is normal. On the other hand, the identification unit 113 identifies the IOM 22 of the secondary path as the cause of the failure when the result of the access test for the IOM 22 of the secondary path is abnormal.

The device for controlling the operation of the entire CM1 is not limited to the CPU 11, and may be, for example, any one of MPU, DSP, ASIC, PLD, and FPGA. Further, the device for controlling the operation of the entire CM1 may be a combination of two or more of the CPU, MPU, DSP, ASIC, PLD and FPGA. MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

FIG. 4 is a diagram for explaining the detection process of the preventive maintenance target component in the RAID system 100 shown in FIG.

As shown by reference numeral B1, IOM # 0 of DE # 02 is detected as a failed component in which a recoverable error has occurred. As a result, although access to the disk 21 via IOM # 0 of DE # 01 is possible as shown by reference numeral B2, access to the disk 21 via IOM # 0 of DE # 02 as shown by reference numerals B3 and B4 is possible. Will fail. Further, as shown by reference numeral B5, the access to the disk 21 via IOM # 0 of DE # 03 also fails.

FIG. 5 is a table showing suspected location management information according to the detection result of the preventive maintenance target component shown in FIG.

The suspected location management information is associated with the suspected location, a point addition value, and a severe failure flag.

In the example shown in FIG. 5, since no error has occurred in IOM # 0, # 1 of DE # 01, IOM # 1 of DE # 02, and IOM # 1 of DE # 03, additional points are added as initial values. Is set to "0" and the severe failure flag is set to "Off". On the other hand, in IOM # 0 of DE # 02 and IOM # 0 of DE # 03, since an access failure has occurred due to a recoverable error, the point addition value is set to "10" and the severe failure flag is set. Is set to "off" as the initial value.

FIG. 6 is a diagram illustrating a process of separating parts subject to preventive maintenance in the RAID system 100 shown in FIG.

As shown by reference numeral C1, IOM # 0 of DE # 02 is set as a target for preventive maintenance. Further, as shown by reference numeral C2, IOM # 0 of DE # 03 is set to a state in which it cannot be used in a pseudo manner. Then, as shown by reference numeral C3, access to the disk 21 via IOM # 1 of DE # 03 is generated.

Here, as a setting to the state where it cannot be used in a pseudo manner, for example, the power of the IOM22 is not turned off or the Serial Attached Small computer system interface (SAS) connection is not disconnected, and the path is temporarily passed in the driver layer in the firmware of CM1. It is to make it unusable. For example, the target EXP access response in the SAS driver layer is set to SASSTS = 28 (Port Unavailable).

FIG. 7 is a diagram illustrating a first example of access test processing for a redundant path in the RAID system 100 shown in FIG.

As shown by reference numeral D1, IOM # 0 of DE # 02 is set to a state in which it cannot be used in a pseudo manner, similarly to IOM # 0 of DE # 03 shown by reference numeral C2 of FIG. Then, access is performed via IOM # 1 of DE # 03 and IOM # 1 of DE # 02. Since no abnormality is detected in IOM # 1 of DE # 03 and IOM # 1 of DE # 02 as shown by reference numerals D2 and D3, the disks of DE # 03 and the disks of DE # 02 as shown by reference numerals D4 and D5. Access to 21 occurs. As a result, as shown by reference numeral D6, IOM # 0 of DE # 02 is set as the first suspected part as the component most likely to be the cause of the failure.

As a result, the worker can perform preventive maintenance of the IOM22 identified as the suspected part on the primary path side while maintaining the communication on the secondary path side.

FIG. 8 is a diagram illustrating a second example of access test processing for a redundant path in the RAID system 100 shown in FIG.

As shown by reference numeral E1, IOM # 0 of DE # 02 is set to a state in which it cannot be used in a pseudo manner, similarly to IOM # 0 of DE # 03 shown by reference numeral C2 of FIG. Then, access is performed via IOM # 1 of DE # 03 and IOM # 1 of DE # 02. Since no abnormality is detected in IOM # 1 of DE # 03 as shown by reference numeral E2, access to the disk 21 of DE # 03 occurs as shown by reference numeral E4. On the other hand, as shown by reference numeral E3, an abnormality is detected in IOM # 1 of DE # 02, so that an access failure to the disk 21 of DE # 02 occurs as shown by reference numeral E5. As a result, as shown by reference numeral E6, IOM # 1 of DE # 02 is set as the first suspected part as the component most likely to be the cause of the failure. Further, as shown by reference numeral E7, IOM # 0 of DE # 02 is set as the second suspected part as the component having the second highest possibility of causing the failure.

Here, if preventive maintenance of IOM2 specified as the second suspected place on the primary path side is carried out, both the primary path side and the secondary path side will be blocked. Therefore, the IOM 22 with the recoverable error identified as the second suspect may be reconnected to the CM1.

FIG. 9 is a table showing suspected location management information according to the access test results for the redundant path shown in FIG.

In the suspected location management information shown in FIG. 9, the added point value in IOM # 1 of DE # 02 set as the first suspected location with reference numeral E6 in FIG. 8 is "" as compared with the suspected location management information shown in FIG. It is set to "100". Then, when the added point value exceeds the threshold value, the severe failure flag is set to "On".

[A-2] Operation Example The preventive maintenance process in the RAID system 100 shown in FIG. 2 will be described with reference to the flowcharts (steps S1 to S9) shown in FIG.

When the preventive maintenance is started, the cutting processing unit 111 puts the IOM 22 at the end of the SAS cascade in a pseudo-unusable state (step S1).

The connection processing unit 112 connects the access to the reverse path (in other words, the secondary path) and determines whether or not an abnormality has occurred in the reverse path (step S2).

If no abnormality has occurred in the reverse path (see the NO route in step S2), the connection processing unit 112 checks whether the reverse path connection test has been performed up to DE2 to which the suspected IOM22 subject to preventive maintenance belongs. Determine (step S3).

If the reverse path connection test has not been performed up to DE2 to which the suspected IOM22 belongs (see the NO route in step S3), the disconnection processing unit 111 cannot use the IOM22 that is one minute closer to the IOC101 in the SAS cascade. The state is set (step S4). Then, the process returns to step S2.

On the other hand, when the reverse path connection test is performed up to DE2 to which the suspected IOM22 belongs (see the YES route in step S3), the specific unit 113 has an abnormality in the original path (in other words, the primary path). Is instructed as the first suspected place (step S5). Then, the preventive maintenance process is completed.

If an error occurs in the reverse path in step S2 (see the YES route in step S2), the connection processing unit 112 maps the error location in the suspected location management information (step S6).

The connection processing unit 112 restores the IOM 22 that has been put into a pseudo-unusable state (step S7).

The identification unit 113 indicates the IOM 22 mapped as the error location as the first suspected location (step S8).

The identification unit 113 instructs the IOM 22 in which the abnormality has occurred in the original path as the second suspected portion (step S9). Then, the preventive maintenance process is completed.

[A-3] Effect According to the storage control device and the control program in the above-described example of the embodiment, for example, the following effects can be achieved.

The disconnection processing unit 111 pseudo-disconnects the connection with the IOM 22 in the primary path when a recoverable failure occurs in the IOM 22 in the primary path. The connection processing unit 112 performs an access test by enabling the connection with the IOM 22 in the secondary path. The identification unit 113 identifies the cause of the failure according to the result of the access test by the connection processing unit 112.

As a result, efficient preventive maintenance can be performed in the RAID system 100 having a redundant path configuration. Then, it is possible to prevent the entire path of DE2 from being blocked due to the maintenance disconnection of the suspected part by the worker, and to perform the maintenance while continuing the RAID access by the upper host.

The identification unit 113 identifies the IOM 22 of the primary path as the cause of the failure when the result of the access test for the IOM 22 of the secondary path is normal. On the other hand, the identification unit 113 identifies the IOM 22 of the secondary path as the cause of the failure when the result of the access test for the IOM 22 of the secondary path is abnormal. As a result, the location of the cause of the failure can be easily identified.

The connection processing unit 112 re-enables the connection with the IOM 22 of the primary path when the IOM 22 of the secondary path is identified as the cause of the failure by the identification unit 113. As a result, it is possible to prevent all paths of DE2 from being blocked.

The disconnection processing unit 111 pseudo-disconnects the connection from the IOM 22 of the DE2 connected to the end of the plurality of DE2s in the cascade connection of the primary path. The connection processing unit 112 performs an access test on the IOM 22 on the secondary path side of the DE2 in which the connection is pseudo-disconnected by the disconnection processing unit 111, with the sequential connection enabled. As a result, the cause of the failure can be efficiently identified.

[B] Other disclosed techniques are not limited to the above-described embodiments, and can be variously modified and implemented without departing from the spirit of the present embodiment. Each configuration and each process of the present embodiment can be selected as necessary, or may be combined as appropriate.

[C] Additional Notes The following additional notes will be further disclosed with respect to the above embodiments.

(Appendix 1)
A storage control device in which multiple storage devices are cascaded by a primary path and a secondary path.
A disconnection processing unit that pseudo-disconnects the connection with the first module when a recoverable failure occurs in the first module in the primary path.
A connection processing unit that enables an access test by enabling the connection with the second module in the secondary path, and
A specific unit that identifies the cause of the failure according to the result of the access test by the connection processing unit, and
A storage control device.

(Appendix 2)
The identification unit identifies the first module as the cause of the failure when the result of the access test for the second module is normal.
The storage control device according to Appendix 1.

(Appendix 3)
The identification unit identifies the second module as the cause of the failure when the result of the access test for the second module is abnormal.
The storage control device according to Appendix 1 or 2.

(Appendix 4)
The connection processing unit re-enables the connection with the first module when the second module is identified as the cause of the failure by the identification unit.
The storage control device according to Appendix 3.

(Appendix 5)
In the cascade connection of the primary path, the disconnection processing unit pseudo-disconnects the connection from the module of the storage device group connected to the end of the plurality of storage device groups in a pseudo manner.
The connection processing unit performs the access test by enabling sequential connection of the modules on the secondary path side of the storage device group in which the connection is pseudo-disconnected by the disconnection processing unit.
The storage control device according to any one of Items 1 to 4.

(Appendix 6)
A computer in which multiple storage devices are cascaded by a primary path and a secondary path,
When a recoverable failure occurs in the first module in the primary path, the connection with the first module is pseudo-disconnected.
The access test was performed by enabling the connection with the second module in the secondary path.
Identify the cause of the failure according to the results of the access test.
A control program that executes processing.

(Appendix 7)
When the result of the access test for the second module is normal, the first module is identified as the cause of the failure.
The control program according to Appendix 6, which causes the computer to execute the process.

(Appendix 8)
When the result of the access test for the second module is abnormal, the second module is identified as the cause of the failure.
The control program according to Appendix 6 or 7, which causes the computer to execute the process.

(Appendix 9)
If the second module is identified as the cause of the failure, re-enable the connection with the first module.
The control program according to Appendix 8, which causes the computer to execute the process.

(Appendix 10)
In the cascade connection of the primary path, the connection is pseudo-disconnected from the modules of the storage device group connected to the end of the plurality of storage device groups.
The access test is performed on the modules on the secondary path side of the storage device group in which the connection is pseudo-disconnected, with the sequential connection enabled.
The control program according to any one of Supplementary note 6 to 9, which causes the computer to execute the process.

100,600: RAID system 1,6: CM
11,61: CPU
101,601: IOC
111: Cutting processing unit 112: Connection processing unit 113: Specific unit 12, 62: EXP
2, 7: DE
21,71: Disk 22,72: IOM

Claims (6)

A storage control device in which multiple storage devices are cascaded by a primary path and a secondary path.
A disconnection processing unit that pseudo-disconnects the connection with the first module when a recoverable failure occurs in the first module in the primary path.
A connection processing unit that enables an access test by enabling the connection with the second module in the secondary path, and
A specific unit that identifies the cause of the failure according to the result of the access test by the connection processing unit, and
A storage control device. The identification unit identifies the first module as the cause of the failure when the result of the access test for the second module is normal.
The storage control device according to claim 1. The identification unit identifies the second module as the cause of the failure when the result of the access test for the second module is abnormal.
The storage control device according to claim 1 or 2. The connection processing unit re-enables the connection with the first module when the second module is identified as the cause of the failure by the identification unit.
The storage control device according to claim 3. In the cascade connection of the primary path, the disconnection processing unit pseudo-disconnects the connection from the module of the storage device group connected to the end of the plurality of storage device groups in a pseudo manner.
The connection processing unit performs the access test by enabling sequential connection of the modules on the secondary path side of the storage device group in which the connection is pseudo-disconnected by the disconnection processing unit.
The storage control device according to any one of claims 1 to 4. A computer in which multiple storage devices are cascaded by a primary path and a secondary path,
When a recoverable failure occurs in the first module in the primary path, the connection with the first module is pseudo-disconnected.
The access test was performed by enabling the connection with the second module in the secondary path.
Identify the cause of the failure according to the results of the access test.
A control program that executes processing.

Images