JP2013149209A - Storage device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately predict a rebuild time under the consideration of an operation load.SOLUTION: A storage device (10) includes: a plurality of drives (14); a statistical information acquisition part (15) for acquiring load information in operation as for a whole part or a part of the plurality of drives (14); and a rebuild time prediction part (16) for predicting a rebuild time when a part of the plurality of drives (14) are replaced on the basis of the load information in operation.

Description

  The present invention relates to a storage apparatus that predicts a rebuild time of a drive.

  The storage device is equipped with a plurality of hard disk drives, solid state drives, and other drives, and is used for storing data. Generally, a storage apparatus stores data in a plurality of drives with redundancy so that data is not lost even if one or more drives fail. For example, RAID 1 and RAID 5 are typical techniques for storing data in a plurality of drives with redundancy.

  When a storage device drive fails, maintenance personnel replace the failed drive with a normal drive. When the drive is replaced, the storage apparatus restores the data to the replaced drive while referring to the data of the drive that has not been replaced. This restoration process is called rebuilding. Some storage apparatuses have spare drives in advance so that maintenance personnel can start rebuilding before replacing the drives.

  Rebuilding involves performance degradation and the risk of multiple failures. During rebuilding, access to the drive for restoring data occurs, so the transfer speed and response time of the storage apparatus deteriorate. Also, if one or more drives that have not been replaced fail during rebuilding, data cannot be restored.

  The rebuild time has become longer due to the increased capacity of drives due to technological advances. For example, it takes several hours to several days to restore the entire data of a 3 terabyte HDD at a speed of several minutes to several tens of minutes per gigabyte.

  The manufacturer of the storage device may publish a guide for rebuilding time. The user designs the system with reference to this guideline. Specifically, for example, the software is designed so that there is no problem in business even if the performance of the storage device is temporarily lowered by rebuilding. In addition, for example, a highly available drive that has a low possibility of further failure of the drive during rebuilding is employed.

  For example, JP 2009-252114 A is known as a document that refers to the estimation of rebuild time.

JP 2009-252114 A

  However, the conventional rebuilding method still has a problem that it is difficult to predict the rebuilding time during operation. The reason why it is difficult to predict the rebuild time is that the rebuild time standard published by the manufacturer is measured in a low load state, whereas the actual rebuild time varies depending on the operational load. Also, the configuration of the logical drive, the rebuild priority setting, the disk drive speed, etc. greatly affect the rebuild time.

  Therefore, in view of such a problem, an object of the present invention is to accurately predict the rebuild time.

  In order to solve the above problems, a storage apparatus according to the present invention includes a plurality of drives, a statistical information acquisition unit that acquires load information during operation for all or part of the plurality of drives, and load information during operation. And a rebuild time predicting unit that predicts a rebuild time when a part of the plurality of drives is replaced.

  According to the present invention, the rebuild time can be accurately predicted by considering the operation load of the drive.

2 is a block diagram illustrating a configuration of a storage apparatus according to the present embodiment. FIG. It is a flowchart which shows the flow of the process which calculates prediction rebuild time in consideration of the operational load concerning this embodiment. It is a flowchart which shows the flow of the process which calculates prediction rebuild time using the pseudo-rebuild concerning this embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a storage apparatus 10 according to this embodiment.

  The storage device 10 includes an input / output unit 11, a setting information management unit 12, a drive control unit 13, two or more drives 14, a statistical information acquisition unit 15, and a rebuild time prediction unit 16. The storage device 10 is, for example, a SAN (Storage Area Network) storage, a NAS (Network Attached Storage), or a DAS (Direct Attached Storage).

  The storage utilization device 20 is a device that uses the storage resources of the storage device 10, and is, for example, a WWW server, a database server, or a recording device. The storage device 10 and the storage using device 20 are connected directly via the network 30 or indirectly via a switch. The network 30 is, for example, Fiber Channel, Ethernet, SCSI, or USB.

  The storage management terminal 40 is a terminal that is operated by an administrator of the storage apparatus 10 to display and change setting information of the storage apparatus 10, and is, for example, a personal computer. The storage apparatus 10 and the storage management terminal 40 are directly connected via the network 50 or indirectly via a switch. The network 50 is, for example, Fiber Channel, Ethernet, USB, or RS232-C. The storage management terminal 40 displays the predicted rebuild time when the drive 14 is replaced. For example, the storage management terminal 40 provides the administrator of the storage apparatus 10 with a user interface for RAID configuration setting, logical drive configuration setting, and failure information reference of the storage apparatus 10.

  The networks 30 and 50 are not necessarily different networks, and may be the same network. In addition, either one or both of the storage use device 20 and the storage management terminal 40 may be built in the storage device 10. For example, the storage apparatus 10 may have a function of a WWW server or a database server. For example, the storage apparatus 10 may include a management interface for displaying and changing the setting information of the storage apparatus 10.

  The input / output unit 11 is an interface that receives read and write requests from the storage use device 20. The input / output unit 11 is, for example, an FC or iSCSI host bus adapter (HBA), or an Ethernet or USB controller.

  The setting information management unit 12 manages setting information including how to store data in each drive 14 and configuration information indicating the state of each drive 14 and predicted rebuild time information. To do. The configuration information includes, for example, RAID configuration information, logical drive configuration information, and drive failure information. The setting information may include drive performance information measured in advance. Statically including drive performance information in the setting information is particularly effective when there are limited types of drives that can be exchanged. Further, the setting information may include the priority of rebuild processing. The setting information may be stored by physically or logically securing a storage area in the drive 14 or may be stored in a storage device other than the drive 14.

  The drive control unit 13 receives a request from the storage utilization device 20 from the input / output unit 11, requests read / write from the drive 14 based on the setting information of the setting information management unit 12, and sends the result to the input / output unit 11. respond. Further, when the drive control unit 13 detects a failure of the drive 14 due to a failure of reading or writing, the drive control unit 13 stores the failure information in the setting information management unit 12. The drive control unit 13 interrupts reading and writing for the failed drive 14 and waits for the administrator of the storage apparatus 10 to replace the drive 14. When the drive 14 is replaced, the drive control unit 13 performs rebuilding. If the rebuild is completed normally, the drive control unit 13 returns the drive failure information stored in the setting information management unit 12 to normal.

  The drive 14 is a storage device capable of reading (reading) and writing (writing) as typified by a hard disk drive (HDD) and a solid state drive (SSD). The drive 14 may be an internal drive built in the storage apparatus 10 or an external drive connected to the storage apparatus 10 via a cable or a network. The drive 14 may be a logical storage device that uses a part or all of one or more physical storage devices as a storage area.

  The statistical information acquisition unit 15 acquires load information during operation for all or some of the plurality of drives 14. Specifically, the statistical information acquisition unit 15 monitors the read and write of the drive control unit 13 and acquires access statistical information including drive load information indicating how much read and write are requested to each drive 14. . Further, the statistical information acquisition unit 15 may acquire drive performance information indicating how much each drive 14 can process read and write, and include the drive performance information in the access statistical information. The dynamic inclusion of drive performance information in the access statistical information is particularly effective when the types of drives 14 that can be replaced are not limited.

  The rebuild time prediction unit 16 predicts the rebuild time when each drive 14 or a combination of drives 14 is exchanged based on the statistical information of the statistical information acquisition unit 15. The rebuild time can be calculated from drive load information and drive performance information. The rebuild time can also be calculated from the rebuild performance information acquired during the pseudo rebuild by the rebuild time predicting unit 16 instructing the drive control unit 13 to perform the pseudo rebuild. As the drive performance information, information acquired by the statistical information acquisition unit 15 from the drive control unit 13 may be used, or information stored in the setting information management unit 12 may be used. Statically storing drive performance information in the setting information management unit 12 in advance is particularly effective when the types of drives 14 that can be replaced are limited.

  FIG. 2 is a flowchart showing a flow of processing for calculating the predicted rebuild time in consideration of the operation load according to the present embodiment.

  The rebuild time prediction unit 16 calculates rebuild times for some or all of the drives 14 (step 201). In a configuration in which rebuilding can be performed even if two or more drives 14 such as a triple mirror or RAID 6 are replaced, the rebuilding time may be calculated for a combination of drives 14.

  The rebuild time prediction unit 16 determines whether or not there is a predicted rebuild time that exceeds a preset threshold (step 202). The threshold can be set by the administrator via the storage management terminal 40, and is, for example, 3 hours. If another drive 14 fails in a time within the threshold, data may be lost. Therefore, the threshold is determined in consideration of the reliability of the drive 14 and the importance of the data.

  If there is a predicted rebuild time exceeding the threshold (step 202; YES), the rebuild time predicting unit 16 displays a warning on the storage management terminal 40 to prompt the administrator (step 203). The warning message is, for example, “Warning: If either HDD1 or HDD2 disk fails, rebuilding is expected to take more than 3 hours. If another disk fails during rebuilding, data may be lost. Is preferred. On the other hand, when there is no predicted rebuild time exceeding the threshold (step 202; NO), the process ends.

  The processing of steps 201 to 203 may be repeated automatically and periodically, or may be repeated periodically in response to an instruction from the administrator. The repeated return interval is, for example, 10 minutes. During maintenance work such as creation or duplication of a logical drive, the rebuilding time calculation process may be interrupted because accurate rebuilding time cannot be predicted or the maintenance work may be delayed. The calculation of the rebuild time may be performed for all of the plurality of drives 14 or may be performed for a part of the plurality of drives 14. Further, the rebuild time may be calculated only for some of the drives 14 selected by the administrator.

  The rebuild time prediction unit 16 predicts the rebuild time from the drive load information, drive performance information, and setting information as necessary. The setting information is required when the setting information affects the rebuild time, such as when rebuilding with an algorithm that restores only the storage area where the valid data is actually saved, or when the rebuild priority can be set. is there.

  The configuration information included in the setting information may include, for example, RAID configuration information and logical drive configuration information. In the following description, for the sake of convenience, a case where the storage apparatus 10 includes two drives 14 and each drive 14 is an SSD of the same model is illustrated. The two SSDs have a total storage capacity of 1 terabyte, are configured with RAID1, and the entire storage area is configured as one logical drive. In other words, the storage capacity of the logical drive is 1 terabyte, and even if one of the SSDs fails, the data of the logical drive is not lost. RAID1 writes the same data to both drives when writing, and reads data from either drive when reading.

  The drive load information includes, for example, a recently measured ratio (drive read ratio) of reading or writing (drive write ratio) per unit time for each SSD. For example, if the total read time in 10 minutes is 3 minutes and the total write time is 2 minutes, the drive read rate is 30% and the drive write rate is 20%. Here, a load temporarily generated due to maintenance such as replication or initialization of a logical drive according to an instruction from the administrator may be excluded from the drive read rate and the drive write rate.

  The drive performance information includes, for example, the maximum number of read times per second of the SSD, the maximum number of write times per second, and the maximum data size (maximum IO size) that can be read and written by one I / O.

  The rebuild time prediction unit 16 calculates the number of reads and the number of writes necessary for rebuild from the configuration information and drive performance information. When the entire data of a non-replaceable drive is simply restored from the beginning to the end by rebuilding, the read count (rebuild read count) and write count (rebuild write count) required for rebuilding are both SSD capacity (1 terabyte) ) Divided by the maximum IO size. The reason for using the maximum IO size is that rebuilding can be performed by simple read and write in sequential access. For example, if the maximum IO size is 128 kilobytes, the number of rebuild reads and the number of rebuild writes are both 8,000,000.

  The rebuild time prediction unit 16 calculates rebuild I / O performance from the drive load information and drive performance information. Assuming that the I / O from the storage use device 20 always has priority over the I / O for rebuilding, the number of seconds that can be read from the non-replaceable drive for rebuilding (the number of rebuild reads per second) is the maximum number of reads per second. X (1- (drive read rate + drive write rate)). The number of seconds that can be written to the replacement drive for rebuilding (number of rebuild writes per second) is determined by the maximum number of writes per second × (1−drive write rate) since the replacement drive is not subject to a read load. For example, if the maximum number of reads is 10,000, the maximum number of writes is 5,000, the drive read rate is 30%, and the drive write rate is 20%, the rebuild read count is 5,000 and the rebuild write count is 4,000.

  The rebuild time prediction unit 16 calculates the rebuild time from the calculated number of rebuild reads, number of rebuild writes, number of rebuild reads per second, and number of rebuild writes per second. The slower of the read from the non-replacement drive and the write to the replacement drive determines the predicted rebuild time. In the above example, since the write is slower, the predicted rebuild time is 8,000,000 ÷ 4,000 = 2,000 seconds.

  In the above description, in order to simplify the description, the case where the drive 14 is an SSD is illustrated, but the drive 14 may be an HDD or another storage device. When the drive 14 is an HDD, drive performance information may be designed in consideration of disk rotation and head seek. In particular, it is considered that the HDD has lower random access performance than the SSD. The method of calculating the number of rebuild reads per second and the number of rebuild writes per second largely depends on the detailed design and implementation of the software of the drive control unit 13.

  Further, in order to simplify the description, the configuration in which the storage apparatus 10 includes two drives 14 is illustrated, but three or more drives 14 may be included. When there are three or more drives 14, the predicted rebuild time may be calculated for the drive that becomes the bottleneck. When the performance of the CPU and HBA is not sufficient and they may become a bottleneck, the CPU usage rate and the HBA usage rate may be considered in addition to the drive read rate and the drive write rate.

  Further, in order to simplify the description, the storage apparatus 10 has been illustrated with a configuration including the same type of drive 14, but different types of drives 14 may be mounted together. For example, the present embodiment can also be applied to a case where the storage apparatus 10 includes HDDs, SSDs, and other storage devices. Even when different types of drives 14 are mixedly mounted, the predicted rebuild time may be calculated for the drive 14 that becomes a bottleneck.

  Further, in order to simplify the description, the case where the two drives 14 are configured with RAID 1 is illustrated, but the present embodiment is applicable to RAIDs other than RAID 0. The present embodiment can also be applied to a drive configuration having a triple mirror or other redundancy. Even in cases other than RAID1, the predicted rebuild time may be calculated for a drive that becomes a bottleneck, or for a CPU or HBA. In a configuration that can withstand a failure of two or more drives, a predicted rebuild time when a combination of two or more drives is replaced may be calculated. Specifically, for example, when a triple mirror is configured with HDD1, HDD2, and HDD3, only HDD1, only HDD2, only HDD3, a combination of HDD1 and HDD2, a combination of HDD2 and HDD3, and a combination of HDD3 and HDD1 The predicted rebuild time may be calculated for six patterns of combinations. For example, when the HDD 1 and the HDD 2 are replaced at the same time, the most time-consuming process among the read from the HDD 3, the write to the HDD 1, and the write to the HDD 2 determines the predicted rebuild time.

  Further, in order to simplify the description, the configuration in which one logical drive is set in the two drives 14 is illustrated, but a plurality of logical drives may be set in the plurality of drives 14.

  Further, in order to simplify the description, the case where the entire data of the drive 14 is restored has been illustrated, but the present embodiment is also applicable to the case where only the storage area being used is restored. In this case, the number of rebuild reads and the number of rebuild writes may be calculated with reference to the configuration information. The calculation method largely depends on the detailed design and implementation of the software of the drive control unit 13.

  In the above description, data written by rebuilding is not checked, but this embodiment can also be applied to the case where data written by rebuilding is read and verified. In this case, it is sufficient to consider the number of rebuild reads for the replacement drive.

  In the above description, the case where the drive 14 is replaced with the same type of drive 14 is illustrated, but the present embodiment is also applicable to the case where the drive 14 is replaced with a different model. If it is known in advance that the drive 14 will be replaced with a different model drive 14, the replaced model is set in the setting information management unit 12, and the rebuild time prediction unit 16 uses the drive performance information of the model. To calculate the estimated rebuild time. When the drive performance information of the model does not exist, the prediction accuracy decreases, but other drive performance information may be substituted.

  The storage management terminal 40 acquires and displays the predicted rebuild time from the setting information management unit 12 automatically or based on an instruction from the administrator. The storage management terminal 40 may display the predicted rebuild times of all the drives 14 or combinations of drives 14 or may display the predicted rebuild times of some drives 14 or combinations of drives 14. Further, only the predicted rebuild time that satisfies the conditions set in advance or set by the administrator may be displayed. The storage management terminal 40 or the storage device 10 may notify the predicted rebuild time that satisfies the conditions by means such as e-mail or SNMP Trap.

  Next, a method for calculating the predicted rebuild time using pseudo rebuild will be described.

  When the rebuild time prediction unit 16 instructs the drive control unit 13 to perform a pseudo rebuild (pseudo rebuild), the statistical information acquisition unit 15 acquires rebuild performance information from the drive control unit 13, and the rebuild time prediction unit 16 performs rebuild performance information. Is used to calculate the predicted rebuild time.

  The pseudo rebuild is performed without exchanging the drive 14, and the drive control unit 13 writes the data to be written to the replacement drive to the unused drive or discards the data by making it appear to have been written in the normal rebuild. The unused drive is, for example, a spare drive or a RAID unconfigured drive, and if it is the same model as the replacement drive, a more reliable result than a different model can be obtained. Since data to be written to an unused drive is not read, it may not be the same as a normal rebuild. For example, data in which all bits are 0, 1 or pseudo random numbers may be written.

  The statistical information acquisition unit 15 acquires rebuild performance information from the drive control unit 13. The rebuild performance information includes performance information related to pseudo rebuild. The rebuild performance information is, for example, the pseudo rebuild start time and end time.

  The rebuild time prediction unit 16 predicts the rebuild time from the rebuild performance information. For example, the rebuild time prediction unit 16 calculates the time required for the pseudo rebuild from the difference between the end time of the pseudo rebuild and the start time of the pseudo rebuild, and sets this as the predicted rebuild time.

  FIG. 3 is a flowchart showing the flow of processing for calculating the predicted rebuild time using the pseudo rebuild according to this embodiment.

  The rebuild time prediction unit 16 instructs the drive control unit 13 to start a pseudo rebuild (step 301).

  When the pseudo rebuild is instructed, the drive control unit 13 starts the pseudo rebuild (step 302).

  When the pseudo rebuild is completed, the statistical information acquisition unit 15 acquires the start time and the end time of the pseudo rebuild as rebuild performance information from the drive control unit 13 (step 303).

  The rebuild time prediction unit 16 obtains the start time and end time of the pseudo rebuild from the statistical information acquisition unit 15, calculates the predicted rebuild time, and stores the calculated predicted rebuild time in the setting information management unit 12 (step 304). .

  The storage management terminal 40 acquires the predicted rebuild time from the setting information management unit 12 and displays it (step 305).

  In order to simplify the explanation, the case of completing the pseudo rebuild was illustrated, but the purpose of the pseudo rebuild is to acquire rebuild performance information, so the pseudo rebuild is interrupted when sufficient information is obtained. May be. For example, the rebuild performance information may include a pseudo rebuild start time, an interrupt time, and a progress rate at the time of the interrupt. In this case, the predicted rebuild time = (interruption time−start time) ÷ progress rate. For example, the rebuild performance information may include the read speed and the write speed of the pseudo rebuild. In this case, the rebuild time prediction unit 16 calculates the read and write data amounts necessary for the rebuild from the configuration information, calculates the read time by dividing the read data amount by the read speed of the pseudo rebuild, and simulates the write data amount. The write time is calculated by dividing by the rebuild write speed, and the longer of the read time and the write time may be used as the predicted rebuild time. For a drive whose performance changes depending on the address to be accessed, the predicted rebuild time may be corrected in consideration of the change in performance. For example, in an HDD in which the transfer speed decreases as the inner circumference is reached, the inner read and write may be corrected later than the outer circumference.

Part or all of this embodiment can be described as in the following supplementary notes, but is not limited to the following.
(Appendix 1)
A plurality of drives 14;
A statistical information acquisition unit 15 that acquires load information during operation for all or some of the plurality of drives 14;
A rebuild time predicting unit 16 that predicts a rebuild time when some of the drives 14 are replaced based on load information during operation;
A storage apparatus 10 comprising:
(Appendix 2)
The storage device 10 according to attachment 1, wherein
The statistical information acquisition unit 15 acquires performance information related to reading and writing for all or some of the plurality of drives 14.
The rebuild time prediction unit 16 is a storage apparatus 10 that predicts rebuild times when some of the drives 14 are replaced based on load information and performance information during operation.
(Appendix 3)
A plurality of drives 14;
A drive control unit 13 that performs a pseudo rebuild on all or part of the plurality of drives 14 to obtain rebuild performance information related to the pseudo rebuild;
A rebuild time predicting unit 16 that predicts a rebuild time when some of the drives 14 are replaced based on the rebuild performance information;
A storage apparatus 10 comprising:
(Appendix 4)
The storage device 10 according to any one of supplementary notes 1 to 3,
The storage apparatus 10 further includes a management terminal 40 for displaying the predicted rebuild time.
(Appendix 5)
The storage device 10 according to any one of supplementary notes 1 to 3,
The storage apparatus 10 further includes a management terminal 40 that displays a warning to the administrator when the predicted rebuild time exceeds a predetermined threshold.

DESCRIPTION OF SYMBOLS 10 ... Storage apparatus 11 ... Input / output part 12 ... Setting information management part 13 ... Drive control part 14 ... Drive 15 ... Statistical information acquisition part 16 ... Rebuild time prediction part 20 ... Storage utilization apparatus 30 ... Network 40 ... Storage management terminal 50 ... network

Claims (5)

Multiple drives,
A statistical information acquisition unit that acquires load information during operation for all or a part of the plurality of drives;
A rebuild time prediction unit for predicting a rebuild time when a part of the plurality of drives is replaced based on the load information during operation;
A storage device comprising: The storage device according to claim 1,
The statistical information acquisition unit acquires performance information related to reading and writing for all or part of the plurality of drives,
The rebuilding time predicting unit predicts a rebuilding time when a part of the plurality of drives is replaced based on the load information and the performance information during the operation. Multiple drives,
A drive control unit that performs a pseudo rebuild on all or a part of the plurality of drives and obtains rebuild performance information related to the pseudo rebuild; and
A rebuild time prediction unit that predicts a rebuild time when a part of the plurality of drives is replaced based on the rebuild performance information;
A storage device comprising: The storage apparatus according to any one of claims 1 to 3,
A storage apparatus further comprising a management terminal for displaying the predicted rebuild time. The storage apparatus according to any one of claims 1 to 3,
A storage apparatus further comprising a management terminal for displaying a warning to an administrator when the predicted rebuild time exceeds a predetermined threshold.