JP4411197B2 - Loop fault detection apparatus and method - Google Patents

Description

The present invention relates to a loop failure detection device or the like that is provided in a loop in which a plurality of storage devices are connected in series from upstream to downstream and detects a failure that has occurred in the loop. Hereinafter, the “magnetic disk device” is simply referred to as “disk”, and the “fiber channel” is abbreviated as “FC”.

  In order to meet the recent demands for high speed and flexible connection, a large number of FC interfaces are employed as internal and external interfaces such as disks. In order to meet the demand for larger capacity, a large number of disks are connected in a loop on one FC interface.

  Such a failure of the FC loop is an intermittent failure that is extremely infrequent in the initial stage, and thus is relieved by a retry. Therefore, as a result of being left unattended because the failure did not surface, there was a case that caused a serious failure. In addition, the failure disk does not detect its own failure, and a plurality of other normal disks sporadically report an abnormality. Therefore, it has been extremely difficult to identify the fault location.

  Patent Document 1 describes a method of identifying a suspected disk by examining whether or not a failure has occurred by disconnecting / connecting the disk itself from the loop while allocating a margin during the inspection process. However, with this method, it is difficult to cope with failures during operation.

  Further, in Patent Document 2, when a CRC (cyclic redundancy check) error is detected in a specific electronic device in the FC loop and no CRC error is detected in an adjacent upstream electronic device, a failure suspected portion is between them. We conclude that there is. However, if the failure occurrence frequency is low, for example, as shown in FIG. 4, it is erroneously determined that there is a failure between the disks 47 and 48.

  CRC is one of the methods for detecting bit errors that occur during data transmission. The data transmission block to be inspected is regarded as binary data, and this binary data is processed by a calculation formula called a generator polynomial to create inspection data having a fixed number of bits. The generated inspection data is sent along with the actual data. On the receiving side, the actual data and the inspection data sent are processed by the same generator polynomial as that on the transmitting side to detect the presence or absence of an error.

Japanese Patent Laid-Open No. 2003-167796 JP 2002-368768 A

  In a system in which a plurality of disks are loop-connected via FC, a loop blockage failure may occur in which all disks cannot be used due to an interface failure of one disk. In addition, even if such a serious failure does not occur, there are many cases in which a minor loop failure occurs as a precursor. In either situation, the failed disk causing the FC failure cannot be identified, or the suspected range cannot be narrowed down. For this reason, there has been a problem that preventive maintenance replacement work and recovery work are prolonged and scaled up, resulting in a serious obstacle to maintenance and operation.

Accordingly, an object of the present invention is to provide a loop failure detection device and the like that can reliably detect a failure that has occurred in a loop formed by connecting a plurality of storage devices in series.

The loop fault detection apparatus according to the present invention is provided in a loop formed by connecting a plurality of storage devices in series from upstream to downstream, and detects a fault occurring in the loop. The loop fault detection apparatus according to the present invention includes an upstream fault detection unit, a downstream fault detection unit, and a fault identification unit. The upstream failure detection means sends a read command from the upstream to each storage device, receives the data frame read by each storage device downstream, detects an error in those data frames, and results in the error. The upstream side including the storage device that transmitted the data frame is determined to be an upstream range that is affected by the failure. Each storage device has a function of reporting an error in the received data frame. The downstream failure detection means includes a storage device that transmits a write command and a data frame from upstream to each storage device, receives a data frame error reported by each storage device downstream, and reports a data frame error. The downstream side is determined to be the downstream range that is affected by the failure. The failure identification unit determines that there is a failure at a boundary portion determined by both the upstream range determined by the upstream failure detection unit and the downstream range determined by the downstream failure detection unit.

Suppose a plurality of storage devices are scattered in a loop, and there is a failure somewhere in the loop. At this time, data transmitted further downstream from each storage device downstream from the failure location is received without causing an error because it does not pass through the failure location. On the other hand, data transmitted from each storage device upstream from the failure location is received with an error by passing through the failure location. Therefore, it can be determined that there is a failure on the upstream side including the storage device in which the data error has occurred. Further, when the data transmitted from the upstream is received by each storage device on the upstream side of the failure location, it is received without causing an error because it does not pass through the failure location. On the other hand, if it is received by each storage device downstream from the fault location, it is received with an error by passing through the fault location. Therefore, it is possible to determine that the downstream side including the storage device in which the data error has occurred is the downstream range that is affected by the failure. Therefore, it can be determined that there is a failure at the boundary between the downstream range affected by the failure and the upstream range affected by the failure. Even if this failure is an intermittent failure with a low occurrence frequency, the failure can be detected more reliably by superimposing the two ranges.

  The loop fault detection method according to the present invention captures the operation of each means of the loop fault detection apparatus according to the present invention as a procedure. Further, the loop fault detection method according to the present invention is described in the claims only corresponding to the highest concept of the loop fault detection apparatus according to the present invention, but each subordinate concept of the loop fault detection apparatus according to the present invention is described in the claims. Corresponding items may also be described in the claims. Furthermore, the present invention can be understood as a loop fault detection program for causing a computer to function each means of the loop fault detection apparatus according to the present invention.

  In other words, the present invention may have the following functions 1 to 3. 1. A function that detects a CRC error in a sent frame and identifies the failure range. 2. A function that analyzes error reports sent from devices and identifies the failure range. 3. Further, a function for further narrowing down the failure range using the results of 1 and 2 above. As a result, according to the present invention, in the method for identifying the failed disk causing the FC loop failure, the resolution for identifying the failed disk can be dramatically increased.

According to the present invention, an error in data that occurs until the data transmitted from each storage device is received downstream is detected, and the upstream side including the storage device in which the data error has occurred is affected by the failure. It is determined that it is in the range of the upstream side to receive, the data error that occurred until the data transmitted from the upstream is received by each storage device , and the downstream side including the storage device that caused the data error is detected By determining that there is a failure at the boundary between the upstream range affected by the failure and the downstream range affected by the failure, Since the two ranges are overlapped, it is possible to reliably detect a failure even with an intermittent failure with a low occurrence frequency.

  FIG. 1 is a block diagram showing an embodiment of a loop fault detection apparatus according to the present invention. FIG. 2 is a block diagram showing an example of the operation of the upstream failure detection means in the present embodiment. FIG. 3 is a block diagram showing an example of the operation of the downstream failure detection means in this embodiment. FIG. 4 is a chart showing an example of the operation of the fault identification unit in the present embodiment. Hereinafter, description will be given based on these drawings.

  The loop failure detection device 11 of this embodiment is realized in the disk control device 20. The disk control device 20 has a function of transferring data between the host computer 10 and the disks 41 to 4n (n is an integer of 2 or more). Since the operations of the disk control device 20 other than the loop failure detection device 11 are well known, description thereof will be omitted.

  The loop failure detection device 11 is provided in the FC loop 30 in which the disks 41 to 4n are connected in series from the upstream 50 to the downstream 51, and detects a failure occurring in the FC loop 30. The loop fault detection device 11 includes an upstream fault detection unit, a downstream fault detection unit, and a fault identification unit, which will be described in detail later. The upstream failure detection means detects a frame error that occurs until the frame transmitted from the disks 41 to 4n is received by the downstream 51, and detects the upstream 50 side including the disk in which the frame error has occurred as a failure X. Is determined to be in the range A on the upstream 50 side affected by the above. The downstream failure detection means detects a frame error that occurs until a frame transmitted from the upstream 50 is received by the disks 41 to 4n, and detects the downstream 51 side including the disk in which the frame error has occurred as a failure X. Is determined to be in the range B on the downstream 51 side affected by the above. The failure identification unit determines that there is a failure X at the boundary between the upstream A range A determined by the upstream failure detection unit and the downstream B range B determined by the downstream failure detection unit.

  It is assumed that the disks 41 to 4n are scattered in the FC loop 30 and there is a failure X somewhere in the FC loop 30. At this time, the frame transmitted further downstream 51 from the disk on the downstream side 51 from the fault location is received without causing an error because it does not pass through the fault location. On the other hand, a frame transmitted from the disk 50 on the upstream side of the failure location is received with an error by passing through the failure location. Therefore, it can be determined that there is a failure X on the upstream 50 side including the disk in which the frame error has occurred. However, in the case of an intermittent failure with a low occurrence frequency, not all frames transmitted from the disk 50 on the upstream side of the failure location result in an error.

  In addition, when a frame transmitted from the upstream 50 is received by a disk located on the upstream side 50 from the failure location, it is received without causing an error because it does not pass through the failure location. On the other hand, if it is received by the disk 51 on the downstream side of the failure location, it is received with an error by passing through the failure location. Therefore, it can be determined that there is a failure X on the downstream 51 side including the disk in which the frame error has occurred. However, in the case of intermittent failures with a low frequency of occurrence, not all frames received by the disk 51 on the downstream side of the failure location result in an error.

  Therefore, it can be determined that there is a failure X at the boundary between the downstream range B affected by the failure X and the upstream range A affected by the failure X. Even if the failure X is an intermittent failure with a low occurrence frequency, the failure X can be detected more reliably by superimposing the two ranges A and B.

  Specifically, the upstream failure detection means transmits a read command from the upstream 50 to the disks 41 to 4n, receives the frames read by the disks 41 to 4n at the downstream 51, and detects errors in those frames. The upstream 50 side including the disk in which the frame error has occurred is determined as the range A on the upstream 50 side affected by the failure X. The downstream side failure detection means transmits the write command and the frame from the upstream 50 to the disks 41 to 4n, receives the frame error detected by the disks 41 to 4n at the downstream 51, and includes the disk in which the frame error has occurred. The downstream 51 side is determined to be the range B on the downstream 51 side affected by the failure X.

  In this case, the upstream failure detection means includes a command issue unit 23, a frame transmission unit 21, a frame reception unit 22, and a CRC check unit 25, and the downstream failure detection means includes a command issue unit 23, a frame transmission unit 21, and a frame reception unit. 22 and a command response analyzing unit 24, and the failure specifying means is built in the command response analyzing unit 24. The command issuing unit 23, the frame transmitting unit 21, and the frame receiving unit 22 are shared by the upstream failure detection unit and the downstream failure detection unit. The command issuing unit 23 issues a read command, a write command, and a frame. The frame transmitting unit 21 transmits the read command, the write command, and the frame issued from the command issuing unit 23 to the disks 41 to 4n. The frame receiving unit 22 receives errors of the frames read by the disks 41 to 4n and the frames detected by the disks 41 to 4n. The CRC check unit 25 detects an error in the frame read by the disks 41 to 4n and received by the frame receiving unit 22, and the upstream 50 side including the disk in which the frame error has occurred is upstream affected by the failure X. It is determined that it is in the range A on the side. The command response analyzing unit 24 extracts the error of the frame detected by the disks 41 to 4n and received by the frame receiving unit 22 from the responses of the disks 41 to 4n to the write command, and determines the disk in which the frame error has occurred. The downstream 51 side that is included is determined as the range B on the downstream 51 side that is affected by the failure X.

  Next, the loop failure detection apparatus 11 according to the present embodiment will be described in more detail with reference to FIGS.

  A plurality of disks 41 to 4n are connected to the disk controller 20 connected to the host computer 10, and these form one FC loop 30. The disks 41 to 4n have a function of reporting that an error frame has been received. In the disk controller 20, to the upstream 50 side of the FC loop 30, a command issuing unit 23 that issues a read / write command, and the commands and data are transmitted to the disks 41 to 4n hanging in the FC loop 30. A frame transmission unit 21 is provided. In the disk controller 20, on the downstream 51 side of the FC loop 30, a frame reception unit 22 that receives command responses and data transmitted from the disks 41 to 4n, and a CRC check unit 25 that checks the CRC of the received frame, A command response analysis unit 24 is provided for extracting command responses of the disks 41 to 4n from the received frames and analyzing whether or not there is an error with respect to the command.

  2 and 3 show an example of the operation of the loop fault detection device 11, and specifically, n = 8. Hereinafter, the operation of the loop failure detection apparatus 11 will be described with reference to FIGS. 2 and 3.

  FIG. 2 is an explanatory diagram showing a failure detection operation when a read command is issued from the command issuing unit 23. The read command issued from the command issuing unit 23 is transmitted from the upstream 50 side to the disks 41 to 48 through the frame transmission unit 21. The disks 41 to 48 send the read data designated by the read command to the downstream 51 side. The frame receiving unit 22 receives data frames sent from the disks 41 to 48. The CRC check unit 25 performs a CRC check on all the data frames, and checks “which data frame sent from which disk 41 to 4n is broken”.

  FIG. 3 is an explanatory diagram showing a failure detection operation when a write command is issued from the command issuing unit 23. A data frame is transmitted from the upstream 50 side to the disks 40 to 48 through the frame transmitting unit 21 in accordance with the write command issued from the command issuing unit 23. The disks 41 to 48 perform the data frame writing process, and send a completion report of the write command to the downstream 51 side as a data frame. The frame receiving unit 22 receives data frames sent from the disks 41 to 48. The command response analysis unit 24 analyzes the command completion report, and determines which of the disks 41 to 48 has a response that the data frame sent from the frame transmission unit 21 has already been broken when it arrives at the disks 41 to 48. Check if it was from.

  As shown in FIG. 2 and FIG. 3, taking as an example the case where there is a failure X that disturbs the frame on the output side of the disk 44, the CRC check unit 25 checks the disks 41 to 50 upstream of the failure location. An error is detected in the CRC check of the data frame from 44, and no error is detected in the CRC check of the data frame from the disks 45 to 48 on the downstream side 51 side from the failure point.

  On the contrary, in the check of the command response unit 24, no error is detected in the command response from the disks 41 to 44 on the upstream side 50 from the failed part, and the command responses from the disks 45 to 48 on the downstream side 51 from the failed part. An error is detected.

  FIG. 4 shows the failure detection status in the above description. In the case of an intermittent failure with a low occurrence frequency, the resolution for identifying the suspicion can be increased at an early stage by finding a change point between the two detection methods as shown in the figure.

  As described above, based on the transfer direction of the frame of the FC loop 30, that is, upstream 50 / downstream 51, the CRC check unit 25 that checks the CRC of the frame at the most downstream of the frame and the command responses of the disks 41 to 48 are sent. And a command response analysis unit 24 for checking. These two check mechanisms narrow down the suspicion from both the upstream 50 side and the downstream 51 side, and specify the failed disk.

  Needless to say, the present invention is not limited to the above embodiment. For example, the present invention can be applied to any electronic device as long as it has a function of reporting that an error frame has been received, not limited to a disk. The above means can also be realized by a computer program. Furthermore, it is possible to completely automate the failure recovery state by automatically disconnecting the identified failed disk from the FC loop.

It is a block diagram which shows one Embodiment of the loop failure detection apparatus which concerns on this invention. It is a block diagram which shows an example of operation | movement of the upstream fault detection means in this embodiment. It is a block diagram which shows an example of operation | movement of the downstream fault detection means in this embodiment. It is a graph which shows an example of operation | movement of the fault specific | specification means in this embodiment.

DESCRIPTION OF SYMBOLS 10 Host computer 11 Loop fault detection apparatus 20 Disk control apparatus 21 Frame transmission part 22 Frame reception part 23 Command issue part 24 Command response analysis part 25 CRC check part 30 FC loop 41-4n Disk 50 Upstream of FC loop 51 Downstream of FC loop

Claims (2)

In a loop failure detection device, which is provided in a loop formed by connecting a plurality of storage devices in series from upstream to downstream and detects a failure occurring in the loop,
An upstream failure detection means, a downstream failure detection means and a failure identification means;
The upstream failure detection means transmits a read command from the upstream to each storage device, receives a data frame read by each storage device downstream, detects an error in those data frames, The upstream side including the storage device that has transmitted the data frame in error is determined to be an upstream range that is affected by the failure ,
Each of the storage devices has a function of reporting an error in the received data frame,
The downstream failure detection means transmits a write command and a data frame from the upstream to each storage device, receives an error of the data frame reported by the storage device, and receives an error of the data frame. Determining that the downstream side including the storage device that has reported
The failure identification means has the failure at a boundary portion determined by both the upstream range determined by the upstream failure detection means and the downstream range determined by the downstream failure detection means. To judge ,
A loop failure detection device characterized by the above. In a loop failure detection method, which is provided in a loop formed by connecting a plurality of storage devices in series from upstream to downstream, and detects a failure occurring in the loop.
Each of the storage devices has a function of reporting an error in the received data frame,
Sending a read command from the upstream to each storage device, receiving data frames read by each storage device downstream, detecting errors in those data frames, The upstream side including the transmitted storage device is determined to be an upstream range affected by the failure,
Including a storage device that transmits a write command and a data frame from the upstream to each of the storage devices, receives an error of the data frame reported by the storage device downstream, and reports an error of the data frame The downstream side is determined to be the downstream range affected by the obstacle,
Determining that there is the failure at a boundary portion determined by both the upstream range and the downstream range;
A loop failure detection method characterized by the above.

Images