CN111966569A

CN111966569A - Hard disk health degree evaluation method and device and computer readable storage medium

Info

Publication number: CN111966569A
Application number: CN201910416165.XA
Authority: CN
Inventors: 邱红飞; 李先绪; 陈泳; 李志云; 郑文武; 黄植勤; 陈辉
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2019-05-20
Filing date: 2019-05-20
Publication date: 2020-11-20

Abstract

The disclosure relates to a hard disk health degree evaluation method and device and a computer readable storage medium. The hard disk health degree evaluation method comprises the following steps: collecting self-monitoring data of a hard disk; processing self-monitoring data of the hard disk; training by adopting the processed self-monitoring data of the hard disk to obtain a hard disk health assessment model; inputting the processed current hard disk self-monitoring data into a hard disk health evaluation model to obtain the current health degree of the hard disk. According to the method, the health degree and the evaluation result of the current hard disk can be obtained by analyzing mass SMART data monitoring indexes of the hard disk, constructing a neural network, training data and generating an evaluation model.

Description

Hard disk health degree evaluation method and device and computer readable storage medium

Technical Field

The disclosure relates to the field of hard disk monitoring and protection, and in particular, to a hard disk health degree assessment method and device, and a computer-readable storage medium.

Background

With the continuous development of big data and cloud computing technology, large enterprises and organizations have a large number of servers, the number and scale of hard disks are huge, and the hard disks of the servers are the components with the highest failure rate, and the performance and the service life of the hard disks, whether mechanical hard disks or Solid State Drive (SSD) hard disks, are reduced along with the lasting operation of applications, so that the regular monitoring of the health degree of the hard disks is very important for the operation and maintenance of a data center.

SMART (Self-Monitoring Analysis and Reporting Technology) is one of standard conditions that each hard disk manufacturer must follow, and determines the health condition of a hard disk by Monitoring state information of a motor, a magnetic head, temperature and the like of the hard disk during operation and comparing the state information with a safety threshold set by the hard disk manufacturer.

Disclosure of Invention

The inventor finds that: in operation and maintenance, the following disadvantages also exist in the management of the health degree of the hard disk in the related art:

1. the system is lack of centralized monitoring, high in management difficulty, most data centers lack of centralized hard disk health monitoring and early warning systems at present, operation and maintenance personnel need to check and analyze the data by adopting instructions or a single machine, and management of large quantities of hard disk abnormity is difficult to deal with.

2. The data characteristics are complex. The SMART data has a plurality of characteristic parameters which reach more than 250 items, common operation and maintenance personnel cannot evaluate the health degree of the hard disk according to the parameter information at all, and the actual early warning effect is not large.

3. The expert model is generally adopted for analyzing the defects of the expert model and the health degree of the existing hard disk, the expert model is the knowledge and experience accumulation of operation and maintenance personnel for many years, for whether some indexes really represent the health degree of the hard disk, each expert has some difference in index understanding, and if some small-probability abnormal conditions occur, the indexes of the expert model are not involved, and the health degree of the hard disk cannot be evaluated.

4. There is a lack of hard disk staging. The SMART data provides the current evaluation state of the hard disk, the state generally has three states of normal, warning, fault or error, and the hard disk is not graded carefully.

5. There is a lack of future state prediction. Due to the lack of uniform data analysis, the massive SMART data does not play a great role in future health and life cycle prediction of the hard disk.

In view of at least one of the above technical problems, the present disclosure provides a hard disk health assessment method and apparatus, and a computer-readable storage medium, which can analyze a large amount of SMART data monitoring indicators of a hard disk.

According to one aspect of the present disclosure, there is provided a hard disk health assessment method, including:

collecting self-monitoring data of a hard disk;

processing self-monitoring data of the hard disk;

training by adopting the processed self-monitoring data of the hard disk to obtain a hard disk health assessment model;

inputting the processed current hard disk self-monitoring data into a hard disk health evaluation model to obtain the current health degree of the hard disk.

In some embodiments of the present disclosure, the method for evaluating health of a hard disk further includes:

training by adopting the processed time-series hard disk self-monitoring data to obtain a hard disk health prediction model;

and inputting the processed self-monitoring data of the hard disk in the preset time period into a hard disk health prediction model, and predicting the health degree and life cycle of the preset prediction cycle of the hard disk.

In some embodiments of the present disclosure, the processing the hard disk self-monitoring data includes:

and carrying out data annotation on the self-monitoring data of the hard disk by adopting health degree grading.

In some embodiments of the present disclosure, the data tagging of the hard disk self-monitoring data by using health classification includes:

calculating the mean value of main parameters of the hard disk self-monitoring data;

determining the health degree grade of the hard disk according to the average value;

and carrying out data annotation on the hard disk self-monitoring data by adopting the health degree grade.

and at least one of denoising and data normalization processing is carried out on the hard disk self-monitoring data.

and selecting characteristic data from the hard disk self-monitoring data according to the correlation so as to reduce the dimensionality of the high-dimensional data.

In some embodiments of the disclosure, the training by using the processed hard disk self-monitoring data, and acquiring the hard disk health assessment model includes:

taking the processed hard disk self-monitoring data as a sample set, and dividing the data in the sample set into a training set, a verification set and a test set;

constructing a neural network model for deep learning;

training model parameters by adopting a training set, adjusting model hyper-parameters by adopting a verification set, and verifying the accuracy of hard disk classification by adopting a test set.

and training by taking the sampling data and the health degree grade as model input to obtain a hard disk health assessment model.

According to another aspect of the present disclosure, there is provided a hard disk health assessment method, including:

collecting self-monitoring data of a hard disk;

processing self-monitoring data of the hard disk;

According to another aspect of the present disclosure, there is provided a hard disk health assessment apparatus including:

the data acquisition module is used for acquiring self-monitoring data of the hard disk;

the data processing module is used for processing the self-monitoring data of the hard disk;

the data training module is used for training by adopting the processed hard disk self-monitoring data to obtain a hard disk health assessment model;

and the health evaluation module is used for inputting the processed self-monitoring data of the current hard disk into the hard disk health evaluation model to acquire the current health degree of the hard disk.

In some embodiments of the present disclosure, the hard disk health assessment apparatus is configured to perform an operation for implementing the hard disk health assessment method according to any of the above embodiments.

a memory to store instructions;

and the processor is used for executing the instructions to enable the hard disk health degree evaluation device to execute the operation of implementing the hard disk health degree evaluation method according to any one of the above embodiments.

the data training module is used for training by adopting the processed time series hard disk self-monitoring data to obtain a hard disk health prediction model;

and the health prediction module is used for inputting the processed hard disk self-monitoring data of the preset time period into the hard disk health prediction model and predicting the health degree and the life cycle of the preset prediction cycle of the hard disk.

According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the instructions, when executed by a processor, implement the hard disk health assessment method according to any of the above embodiments.

According to the method, the health degree and the evaluation result of the current hard disk can be obtained by analyzing mass SMART data monitoring indexes of the hard disk, constructing a neural network, training data and generating an evaluation model.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic diagram of some embodiments of a hard disk health assessment method according to the present disclosure.

FIG. 2 is a schematic diagram of another embodiment of a hard disk health assessment method according to the disclosure.

Fig. 3 is a schematic diagram of some further embodiments of the hard disk health assessment method according to the disclosure.

Fig. 4 is a schematic diagram of some embodiments of the hard disk health assessment apparatus according to the present disclosure.

FIG. 5 is a schematic diagram of another embodiment of a hard disk health assessment apparatus according to the present disclosure.

FIG. 6 is a schematic diagram of a hard disk health assessment apparatus according to still other embodiments of the present disclosure.

FIG. 7 is a schematic diagram of another embodiment of a hard disk health assessment apparatus according to the present disclosure.

FIG. 8 is a schematic diagram of a hard disk health assessment apparatus according to further embodiments of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

Fig. 1 is a schematic diagram of some embodiments of a hard disk health assessment method according to the present disclosure. Preferably, the present embodiment may be executed by the hard disk health degree evaluation apparatus of the present disclosure. The method comprises the following steps:

and step 11, collecting the self-monitoring data of the hard disk.

In some embodiments of the present disclosure, the hard disk self-monitoring data may be SMART data.

SMART data may vary from hard disk manufacturer to hard disk manufacturer, and in particular SSD hard disks, the SMART data definition may vary greatly.

In some embodiments of the present disclosure, where xie and west numbers are selected, the SSD hard disk selects magnesium light to acquire data.

In some embodiments of the present disclosure, step 11 may comprise: using the volume data set provided at Backblaze, recorded information of free hard disk data sets, including hard disk information, is published each day for a predetermined period of time (e.g., from 2013 to 2018) at the Backblaze website. Such as serial number, date, hard disk manufacturer, model number, hard disk SMART data.

In some embodiments of the present disclosure, SMART column attribute primary parameters include:

ID: the attribute ID is typically a decimal or hexadecimal number between 1 and 255.

ATTRIBUTE _ NAME: hard disk manufacturer defined attribute names. I.e., the name of a certain detection item, is a literal interpretation of the ID code.

Current value (value): the current value is the result of formula calculation of each ID item from the measured Raw data (Raw value) while the hard disk is running, and is between 1 and 253. 253 means best case and 1 means worst case.

Critical value (Threshold): the threshold value is a threshold value designated by the hard disk manufacturer and indicating the reliability of a certain item, and if the current value of a certain parameter is close to the threshold value, the hard disk will become unreliable, which may result in data loss or hard disk failure.

In some embodiments of the present disclosure, SMART provides the current evaluation status of the hard disk after analyzing the current values, the comparison results of the worst values and the critical values, and the data values of the items. The states generally include normal, warning, fault, or error states. And if the current value is far larger than the critical value, the current value is a normal mark. When the current value is larger than the critical value but close to the critical value, the alarm mark is obtained; when the current value is less than the critical value, it is a fault or error sign.

In some embodiments of the present disclosure, step 11 may comprise: and (5) extracting key data. The mechanical structure has service life, performance reduction over service life and noise, and the service life of the mechanical hard disk is generally more than 3 years or 5 years. The service life of the solid state disk is actually shorter than that of a mechanical hard disk in theory, but the service life of the solid state disk can reach 5-10 years in practical use.

In some embodiments of the present disclosure, the step of extracting the key data may include: and extracting all SMART data of the hard disk, wherein the SMART data are a warning mark (when the Pre-failure/advisory BIT is greater than but close to the critical value) and a fault or error mark (when the Pre-failure/advisory BIT is greater than 1 and the current value and the worst value are less than the critical value).

In some embodiments of the present disclosure, SMART data may contain hard disk basic information (model, capacity, temperature, sector, etc.), seek information (seek time, seek performance, etc.), count information, and error information, such as:

01(001) Read _ Error _ Rate bottom layer data Read Error Rate

04(004) Start _ Stop _ Count Start/Stop Count

05(005) Reallocated _ Sector _ Ct remap Sector number

09(009) Power _ On _ Hours Power-On time accumulation, total Power-On time after factory shipment, and general disk lifetime of thirty thousand Hours

0A (010) Spin _ Retry _ Count spindle Spin-up Retry number (i.e. hard disk spindle motor start Retry number)

0B (011) Calibration _ Retry _ Count disk Calibration Retry number

0C (012) Power _ Cycle _ Count disk Power-on times

C2(194) Temperature _ Celsius

C7(199) UDMA _ CRC _ Error _ Count parity Error Rate

C8(200) Write _ Error _ Rate Write Error Rate

F1(241) Total _ LBAs _ write: the unit of data indicating the total write-in of the disk from the factory is LBAS 512Byte

F2(242) Total _ LBAs _ Read: the unit of data representing the total read data of the disk from the factory is LBAS 512 Byte.

In some embodiments of the present disclosure, the magnesium optical SSD hard disk data may include:

1: read Error Rate of Raw bottom layer data

5: the number of new bad blocks in the use of Re-allocated inductors Count

9: accumulated Power-up time of Power On homes Count

12: power Cycle Count device Power-on period

170: grow Failing Block Count

171: program Fail Count

172: erase Fail Count Erase error Count

173: wear Leveling Count average number of erasures

174: abnormal Power failure times of Unnexpected Power Loss Count

181: non-4k Aligned Access Non-4 KB Aligned Access number

183: SATA Interface Downshift Interface destage count

187: reported Un-recoverable Errors count

188: command Timeout instruction Timeout count

189: factory Bad Block Count

196: re-allocation Event Count bad block remap Event Count

197: the Current Pending Sector Count value is always 0

198: irreparable Error found during Smart Off-line Scan Uncolorable Error Count self-test

199: transfer CRC Error Rate between Ultra DMA CRC Error Rate host and interface

202: percent Percentage Of The Rated Life Used (MLC 5000/SLC 100000 calculation) falls from 100

206: write Error Rate of underlying data.

And step 12, processing the self-monitoring data of the hard disk.

In some embodiments of the present disclosure, step 12 may comprise:

and 121, carrying out data annotation, and establishing an expert model to carry out state classification on the hard disk.

In some embodiments of the present disclosure, step 121 may comprise: and carrying out data annotation on the self-monitoring data of the hard disk by adopting health degree grading.

In some embodiments of the present disclosure, step 121 may comprise:

step 1211, calculating an average value of the main parameters of the hard disk self-monitoring data.

In some embodiments of the present disclosure, for a mechanical hard disk, extracting SMART primary parameters may include information of a reassigned sector parameter, a current to-be-mapped sector parameter, a misplaced unrecoverable parameter, and an instruction timeout parameter.

In some embodiments of the present disclosure, for the SSD hard disk, step 1211 may include: extracting the remapped sector count/retired block count, SSD remaining life calculating the mean value of each parameter,

step 1212, determining the health level of the hard disk according to the mean value; and carrying out data annotation on the hard disk self-monitoring data by adopting the health degree grade.

In some embodiments of the present disclosure, step 1212 may comprise: determining a grade determination value Y by adopting a formula (1), wherein the current value in the formula (1) is the current value of the mean value, and the critical value is a preset mean critical value; the corresponding health level is determined according to the level decision value, as shown in table 1.

Rank determination value Y ═ (current value-critical value)/(255-critical value) × 100% (1)

TABLE 1

Rank of	Level decision value Y	Remarks for note
			P1	80％～99％	Excellent performance
P2	60％～80％	Good performance
			P3	30％～60％	In general
P4	10％～30％	Warning
			P5	Less than 10 percent	Fault of

In some embodiments of the present disclosure, step 121 may comprise: and marking the data by adopting an expert model according to key parameter values of the SMART data and combining an empirical formula of an expert, and classifying the hard disk into grades of P1-P5.

Step 122, data preprocessing.

In some embodiments of the present disclosure, step 122 may comprise: and at least one of denoising and data normalization processing is carried out on the hard disk self-monitoring data.

In some embodiments of the present disclosure, the denoising process may include: single characteristic deleting processing, namely deleting all the same characteristics of a certain row of characteristic values; deleting missing features, namely deleting the feature columns when the missing proportion of the feature columns reaches a specific threshold value; and deleting high-correlation features, namely calculating the correlation among feature variables by a feature selection method, and deleting features lower than a certain threshold.

In some embodiments of the present disclosure, the denoising process may further include: and filling the missing value, including filling the missing value with the data of the previous row of the missing value, the average value of the columns and the average value according to each hard disk grade.

In some embodiments of the present disclosure, the data normalization process may include: training data needs to be standardized, values of general variables in the method are between 0 and 1, the effect on the model is weakened due to the fact that the values of certain variables are large, and the original feature data are subjected to normalization scaling and are achieved through dispersion standardization and standard deviation standardization.

And step 123, selecting characteristics.

In some embodiments of the present disclosure, step 123 may comprise: and selecting characteristic data from the hard disk self-monitoring data according to the correlation, thereby achieving the purpose of reducing the dimensionality of high-dimensional data, reducing the training complexity and improving the precision and accuracy of data training.

In some embodiments of the present disclosure, step 123 may comprise: the method adopts an MIC (maximum Information Coefficient) method, a last Least absolute contraction and selection operator, a minimum absolute contraction and selection operator and a lasso algorithm, and is a compression estimation \ PCA (Principal Component Analysis) method for Analysis and characteristic selection, thereby achieving the purpose of reducing the dimensionality of high-dimensional data.

In some embodiments of the present disclosure, step 123 may further comprise: taking the data after the characteristic selection as a sample set; and the data in the sample set is divided into a training set, a validation set and a test set.

In some embodiments of the present disclosure, step 123 may comprise: taking SMART data after feature selection as a sample set, and selecting the first 50 features with large correlation as data columns by a Principal Component Analysis (PCA) method; and dividing the data in the sample set into a training set, a verification set and a test set. Wherein 70% of the training set, 15% of the validation set and 15% of the test set are subjected to cross validation.

And step 13, training by adopting the processed self-monitoring data of the hard disk to obtain a hard disk health assessment model.

In some embodiments of the present disclosure, step 13 may comprise: constructing a Neural network model (such as RNN (Recurrent Neural Networks) and a multilayer perceptron) for deep learning; training model parameters by adopting a training set, adjusting model hyper-parameters by adopting a verification set, and verifying the accuracy of hard disk classification by adopting a test set; and finally generating a neural network model meeting the system requirements.

In some embodiments of the present disclosure, as shown in table 2, step 13 comprises: and training by taking the sampling data f 1-fn and the health degree grade y as model input to obtain a hard disk health assessment model.

TABLE 2

L1-Ln in Table 2 represents n sets of data, where each set of data includes sample data f 1-fn and a health level y, which is an actual value (input value) of the health level, the health level y being an actual value of the health level

Is an evaluation value (output value) of the health degree level.

In some embodiments of the present disclosure, step 13 may comprise: training a constructed data set by constructing a neural network model, wherein the neural network model comprises a Support Vector Machine (SVM) algorithm and a multilayer neural network (RNN) algorithm; in the training process, if the verification error is not reduced in a period of time (such as 500 times of iteration) training, stopping the training process and storing the training parameters of the model; in model tuning, the hyper-parameters mainly include iteration times and learning rate. The parameters of the SVM support vector machine are mainly the selection of kernel functions, and the test mainly adopts linear kernel functions and Gaussian kernels.

In some embodiments of the present disclosure, step 13 may further include: and verifying the classification accuracy of the model through the test set, and if the system requirements are met, putting the model into use.

And step 14, inputting the processed self-monitoring data of the current hard disk into a hard disk health assessment model to obtain the current health degree of the hard disk.

According to the hard disk health degree evaluation method provided by the embodiment of the disclosure, the health degree and the evaluation result of the current hard disk can be obtained by analyzing the monitoring indexes of the SMART data of the mass hard disks, constructing a neural network, training data and generating an evaluation model.

According to the embodiment of the invention, massive SMART data monitoring indexes of the hard disk can be analyzed, and a neural network model is established through training data to obtain the evaluation result of the health degree of the current hard disk, so that the abnormal hard disk can be replaced in time, and the safety and the reliability of a data center are ensured. The embodiment of the disclosure solves the technical problem of inaccurate prediction results caused by the fact that some factors which easily cause hard disk faults cannot be collected or quantified in a hard disk fault prediction system in the related art.

The embodiment of the disclosure can realize the monitoring of all hard disk states and the evaluation of health degree of the data center through centralized monitoring, and can handle the management of the abnormality of a large number of hard disks.

According to the embodiment of the disclosure, the characteristic data is selected from the hard disk self-monitoring data through correlation, so that the purpose of reducing the dimensionality of high-dimensional data is achieved, the training complexity is reduced, and the precision and the accuracy of data training are improved.

The embodiment of the disclosure adopts an expert model to label data according to key parameter values of SMART data and combines an empirical formula of an expert, and hard disks are classified into grades from P1 to P5. Hard disks are more finely graded.

The embodiment of the disclosure establishes a centralized and unified hard disk management and control system based on mass hard disk SMART data. The embodiment of the disclosure adopts hard disk health classification to label data; the above embodiments of the present disclosure adopt a machine learning algorithm to select data features; according to the embodiment of the disclosure, the neural network is constructed, the data is trained and the model is generated, and the health degree of the hard disk can be evaluated, so that the abnormal hard disk can be replaced in time, and the safety and reliability of the data center are ensured.

FIG. 2 is a schematic diagram of another embodiment of a hard disk health assessment method according to the disclosure. Preferably, the present embodiment may be executed by the hard disk health degree evaluation apparatus of the present disclosure. Steps 21-24 of the embodiment of fig. 2 of the present disclosure are the same as or similar to steps 11-14, respectively, of the embodiment of fig. 1. The method comprises the following steps:

and step 21, collecting the self-monitoring data of the hard disk.

And step 22, processing the self-monitoring data of the hard disk.

And step 23, training by adopting the processed self-monitoring data of the hard disk to obtain a hard disk health assessment model.

And 24, inputting the processed self-monitoring data of the current hard disk into a hard disk health evaluation model to obtain the current health degree of the hard disk.

And 25, training by adopting the processed time series hard disk self-monitoring data to obtain a hard disk health prediction model.

In some embodiments of the present disclosure, step 25 may comprise: SMART data of a period of time (3-5 years per day) is input, the SMART data of each hard disk can reach 1800 pieces, a regression prediction model is established by adopting algorithms such as ridge regression, elastic network, support vector regression algorithm, RNN and the like, the health degree of the hard disk in a future predetermined prediction illumination period (for example, 1 month or 3 months) is predicted, and after hundreds of rounds of training, higher prediction accuracy is achieved.

In some embodiments of the present disclosure, step 25 may comprise: training is performed by using an LSTM (Long short-Term Memory networks), and the training process can include the following steps:

the number of layers of the LSTM is set to 2, 5 classes, the number of input neurons is set to 30, i.e., the number of features, and the second hidden layer is set, i.e., the cell has 100 cells.

m _ inputs equals 30, number of input neurons

max _ time is 1800, for a total of 1800 time series data

lstm _ size 100, hidden layer neuron number

m _ classes 5, 5 health grades P1-P5

back _ szie 128, 128 samples per batch

m _ back, total number of batches

During training, the parameter learning rate is set to be 0.0001, the initial values of the weight and bias value matrix are defined, the training size of each batch is 128, the iteration number is 100, and the parameters are continuously adjusted in the training process to generate the best model.

And 26, inputting the processed hard disk self-monitoring data of the preset time period into a hard disk health prediction model, and predicting the health degree and life cycle of the preset prediction cycle of the hard disk.

According to the method, the SMART data of the time sequence is input, the prediction model is established by using the ridge regression, the elastic network, the support vector regression algorithm, the RNN and other algorithms, in the five-stage prediction of hard disk grading from P1 to P5, the health degree of the hard disk in the future 3 months is predicted by reviewing samples of 3-5 years, and after hundreds of rounds of training, the accuracy of model prediction can reach higher accuracy.

The embodiment of the disclosure adopts a centralized and unified monitoring system, and solves the technical problem of inaccurate prediction results caused by the fact that some factors which easily cause hard disk faults cannot be collected or quantified in a hard disk fault prediction system in the related technology. The embodiment of the disclosure can analyze massive SMART data monitoring indexes of the hard disk, train data and generate an evaluation model and a prediction model by constructing a neural network and a regression prediction method, so that evaluation prediction results of the current health degree and the future health degree of the hard disk can be obtained. Therefore, the abnormal hard disk can be replaced in time, and the safety and the reliability of the data center are guaranteed.

According to the embodiment of the disclosure, through unified data analysis, the current health degree and the future health degree of the hard disk can be determined through massive SMART data, and life cycle prediction is carried out.

Fig. 3 is a schematic diagram of some further embodiments of the hard disk health assessment method according to the disclosure. Preferably, the present embodiment may be executed by the hard disk health degree evaluation apparatus of the present disclosure. The disclosure provides that steps 31-32 of the embodiment of fig. 3 are the same as or similar to steps 11-12 of the embodiment of fig. 1, and steps 21-22 of the embodiment of fig. 2, respectively; steps 33-34 of the fig. 3 embodiment of the present disclosure are the same as or similar to steps 25-26, respectively, of the fig. 2 embodiment. The method comprises the following steps:

and step 31, collecting the self-monitoring data of the hard disk.

And step 32, processing the self-monitoring data of the hard disk.

And step 33, training by adopting the processed time series hard disk self-monitoring data to obtain a hard disk health prediction model.

In one embodiment of the present disclosure, the hard disk health prediction model may include:

forget gate output: f. of_t＝σ(W_f[h_t-1，x_t])+b_f

And (3) inputting and outputting: i.e. i_t＝σ(W_i[h_t-1，x_t])+b_i

Neuronal status:

and (3) outputting by an output gate: o_t＝σ(W_o[h_t-1，x_t])+b_o

h_t＝o_t*tanh(C_t)

Wherein x is_tIs input; h is_tIs an output; i.e. i_tIs the output of the input gate; f. of_tForgetting gate output; c_tThe cell unit state at the current time t; o_tIs the output of the output gate; w and b are parameter matrixes; σ is the ReLu function; tan h is the hyperbolic tangent activation function.

And step 34, inputting the processed hard disk self-monitoring data of the preset time period into a hard disk health prediction model, and predicting the health degree and life cycle of the preset prediction cycle of the hard disk.

Based on the method for evaluating the health degree of the hard disk provided by the embodiment of the disclosure, a centralized and unified management and control system of the hard disk can be established based on mass SMART data of the hard disk; performing data annotation by adopting hard disk health classification; selecting data characteristics by adopting a machine learning algorithm; a neural network is constructed, data are trained, a hard disk health prediction model is generated, and the future health degree and the life cycle of the hard disk can be predicted. Therefore, abnormal hard disks can be replaced in time, and the safety and reliability of the data center are guaranteed.

According to the embodiment of the disclosure, through unified data analysis, the future health degree of the hard disk can be determined through massive SMART data, and the life cycle prediction is carried out.

Fig. 4 is a schematic diagram of some embodiments of the hard disk health assessment apparatus according to the present disclosure. As shown in fig. 4, the hard disk health degree evaluation apparatus of the present disclosure may include a data acquisition module 41, a data processing module 42, a data training module 43, and a health evaluation module 44, wherein:

and the data acquisition module 41 is used for acquiring the self-monitoring data of the hard disk.

And the data processing module 42 is used for processing the hard disk self-monitoring data.

And the data training module 43 is configured to train with the processed hard disk self-monitoring data to obtain a hard disk health assessment model.

And the health evaluation module 44 is configured to input the processed current hard disk self-monitoring data into a hard disk health evaluation model to obtain the current health degree of the hard disk.

In some embodiments of the present disclosure, the hard disk health degree evaluation apparatus is configured to perform operations for implementing the hard disk health degree evaluation method according to any of the above embodiments (for example, the embodiments of fig. 1 or fig. 2).

Based on the hard disk health degree evaluation device provided by the embodiment of the disclosure, the health degree and the evaluation result of the current hard disk can be obtained by analyzing the monitoring indexes of the SMART data of the mass hard disks, constructing a neural network, training data and generating an evaluation model.

FIG. 5 is a schematic diagram of another embodiment of a hard disk health assessment apparatus according to the present disclosure. As shown in fig. 5, the hard disk health degree evaluation apparatus of the present disclosure may include a data acquisition module 41, a data processing module 42, a data training module 43, and a health prediction module 45, wherein:

And the data training module 43 is configured to train the hard disk self-monitoring data of the processed time sequence to obtain a hard disk health prediction model.

And the health prediction module 45 is used for inputting the processed hard disk self-monitoring data of the preset time period into the hard disk health prediction model and predicting the health degree and the life cycle of the preset prediction cycle of the hard disk.

In some embodiments of the present disclosure, the hard disk health degree evaluation apparatus is configured to perform operations for implementing the hard disk health degree evaluation method according to any of the above embodiments (for example, the embodiment of fig. 3 or fig. 2).

Based on the hard disk health degree evaluation device provided by the embodiment of the disclosure, a hard disk centralized unified management and control system can be established based on mass hard disk SMART data; performing data annotation by adopting hard disk health classification; selecting data characteristics by adopting a machine learning algorithm; a neural network is constructed, data are trained, a hard disk health prediction model is generated, and the future health degree and the life cycle of the hard disk can be predicted. Therefore, abnormal hard disks can be replaced in time, and the safety and reliability of the data center are guaranteed.

FIG. 6 is a schematic diagram of a hard disk health assessment apparatus according to still other embodiments of the present disclosure. As shown in fig. 6, the hard disk health degree evaluation apparatus of the present disclosure may include a data acquisition module 41, a data processing module 42, a data training module 43, a health evaluation module 44, and a health prediction module 45, wherein:

The data training module 43 is used for training by adopting the processed hard disk self-monitoring data to obtain a hard disk health assessment model; and training by adopting the processed time series hard disk self-monitoring data to obtain a hard disk health prediction model.

In some embodiments of the present disclosure, the hard disk health degree evaluation apparatus is configured to perform operations for implementing the hard disk health degree evaluation method according to any of the above embodiments (for example, any of fig. 1 to 3).

The hard disk health degree evaluation device provided by the embodiment of the disclosure adopts a centralized and unified monitoring system, and solves the technical problem that some factors which easily cause hard disk faults in a hard disk fault prediction system in the related technology cannot be collected or quantized to cause inaccurate prediction results. The embodiment of the disclosure can analyze massive SMART data monitoring indexes of the hard disk, train data and generate an evaluation model and a prediction model by constructing a neural network and a regression prediction method, so that evaluation prediction results of the current health degree and the future health degree of the hard disk can be obtained. Therefore, the abnormal hard disk can be replaced in time, and the safety and the reliability of the data center are guaranteed.

FIG. 7 is a schematic diagram of another embodiment of a hard disk health assessment apparatus according to the present disclosure. Compared with the embodiment of fig. 6, the specific internal components of the data processing module 42 are shown in the embodiment of fig. 7, as shown in fig. 7, the hard disk health degree evaluation apparatus of the present disclosure may include a data acquisition module 41, a data processing module 42, a data training module 43, a health evaluation module 44, and a health prediction module 45, and the data processing module 42 in any one of the embodiments of fig. 4 to 7 may include a data labeling unit 421, a data preprocessing unit 422, and a feature selecting unit 423, where:

and the data acquisition module 41 is used for acquiring SMART data of the hard disk.

In some embodiments of the present disclosure, the data acquisition module 41 may be configured to acquire hard disk SMART feature data, which refers to taking SMART data in different time periods as original sample data.

In some embodiments of the present disclosure, the data collection module 41 may download, at the backsblaze website, the daily record information of the free hard disk data set, which is obtained from 2013 to 2018, including hard disk information, such as serial number, date, hard disk manufacturer, model, and hard disk SMART data, using the volume data set provided in the backsblaze.

And the data labeling unit 421 is configured to label data according to key parameter values of SMART data by using an expert model and combining an expert empirical formula, and classify the hard disk into grades P1-P5.

In some embodiments of the present disclosure, the data tagging unit 421 may be configured to tag the hard disk as a grade P1-P5 after the system analyzes the comparison result of the current value, the worst value and the critical value of the key parameter and the data value, wherein SMART provides the current evaluation status of the hard disk, and the status generally includes three statuses of normal, warning, failure or error.

The data preprocessing unit 422 is configured to label data by using the data labeling unit, select relevant features by using the feature selecting unit, and perform data normalization processing such as denoising and normalization on the data.

The feature selection unit 423 is configured to select a feature according to the correlation, so as to achieve the purpose of reducing the dimensionality of the high-dimensional data, reduce the complexity of training, and improve the precision and accuracy of data training.

In some embodiments of the present disclosure, the feature selection unit 423 may be configured to perform analysis by using an MIC method \ lasso \ PCA, and select features, thereby achieving the purpose of reducing high-dimensional data dimensions; finally, the data after the feature selection is used as a sample set; and the data in the sample set is divided into a training set, a validation set and a test set.

In some embodiments of the present disclosure, the data training module 43 may be configured to construct a neural network model (e.g., RNN, multi-layer perceptron) for deep learning, perform model training with a training set, optimize hyper-parameters with a verification set, verify the accuracy of hard disk classification with a test set, and finally generate a hard disk health assessment model meeting system requirements.

In some embodiments of the present disclosure, the data training module 43 may be configured to input SMART data of a time sequence, establish a prediction model by using a ridge regression \ elastic net \ support vector regression algorithm, RNN, and other algorithms, and in five-level predictions for hard disk classification from P1 to P5, review samples from 3 to 5 years predicts the health of a hard disk in the future 3 months, and after hundreds of rounds of training, the accuracy of model prediction reaches a higher accuracy.

In some embodiments of the present disclosure, health assessment module 44 may be configured to input current hard disk SMART data to derive a current health of the hard disk based on a health assessment model.

In some embodiments of the present disclosure, the health prediction module 45 may be configured to input SMART data of the hard disk for a period of time based on a neural network or a regression prediction model, and may obtain the future health and life cycle of the hard disk.

The hard disk health degree evaluation device provided by the embodiment of the disclosure establishes a hard disk centralized unified management and control system based on mass hard disk SMART data; performing data annotation by adopting hard disk health classification; selecting data characteristics by adopting a machine learning algorithm; the neural network is constructed, data are trained and a model is generated, and the health degree and the life cycle of the hard disk can be evaluated and predicted, so that the abnormal hard disk can be replaced in time, and the safety and the reliability of the data center are guaranteed.

FIG. 8 is a schematic diagram of a hard disk health assessment apparatus according to further embodiments of the present disclosure. As shown in fig. 6, the hard disk health assessment apparatus of the present disclosure may include a memory 81 and a processor 82, wherein:

a memory 81 for storing instructions.

A processor 82, configured to execute the instructions, so that the hard disk health assessment apparatus performs operations for implementing the hard disk health assessment method according to any of the embodiments described above (for example, any of fig. 1 to fig. 3).

The hard disk health degree evaluation device provided by the embodiment of the disclosure adopts a centralized and unified monitoring system, and solves the technical problem that some factors which easily cause hard disk faults in a hard disk fault prediction system in the related art cannot be collected or quantized to cause inaccurate prediction results. The embodiment of the disclosure can analyze massive SMART data monitoring indexes of the hard disk, train data and generate an evaluation model and a prediction model by constructing a neural network and a regression prediction method, and obtain the evaluation prediction result of the current health degree and the future health degree of the hard disk.

According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the instructions, when executed by a processor, implement the hard disk health assessment method according to any of the embodiments (for example, any of fig. 1 to 3) above.

The computer readable storage medium provided by the above embodiment of the present disclosure establishes a centralized and unified management and control system for hard disks based on a large amount of SMART data of hard disks; performing data annotation by adopting hard disk health classification; selecting data characteristics by adopting a machine learning algorithm; the neural network is constructed, data are trained and a model is generated, and the health degree and the life cycle of the hard disk can be evaluated and predicted, so that the abnormal hard disk can be replaced in time, and the safety and the reliability of the data center are guaranteed.

The functional units described above may be implemented as a general purpose processor, a Programmable Logic Controller (PLC), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof, for performing the functions described herein.

Thus far, the present disclosure has been described in detail. Some details that are well known in the art have not been described in order to avoid obscuring the concepts of the present disclosure. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware to implement the above embodiments, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method for evaluating health of a hard disk is characterized by comprising the following steps:

collecting self-monitoring data of a hard disk;

processing self-monitoring data of the hard disk;

2. The hard disk health assessment method according to claim 1, further comprising:

3. The method for evaluating health of a hard disk according to claim 1 or 2, wherein the processing of the hard disk self-monitoring data comprises:

4. The method for evaluating health of a hard disk according to claim 3, wherein the data labeling of the self-monitored data of the hard disk by using the health classification comprises:

5. The method for evaluating health of a hard disk according to claim 1 or 2, wherein the processing of the hard disk self-monitoring data comprises:

6. The method for evaluating health of a hard disk according to claim 1 or 2, wherein the processing of the hard disk self-monitoring data comprises:

7. The hard disk health assessment method according to claim 1 or 2, wherein the training by using the processed hard disk self-monitoring data to obtain the hard disk health assessment model comprises:

constructing a neural network model for deep learning;

8. The hard disk health assessment method according to claim 1 or 2, wherein the training by using the processed hard disk self-monitoring data to obtain the hard disk health assessment model comprises:

9. A method for evaluating health of a hard disk is characterized by comprising the following steps:

collecting self-monitoring data of a hard disk;

processing self-monitoring data of the hard disk;

10. A hard disk health assessment apparatus, comprising:

11. The hard disk health assessment apparatus according to claim 10, wherein the hard disk health assessment apparatus is configured to perform operations for implementing the hard disk health assessment method according to any one of claims 1 to 8.

12. A hard disk health assessment apparatus, comprising:

a memory to store instructions;

a processor configured to execute the instructions to cause the hard disk health assessment apparatus to perform operations for implementing the hard disk health assessment method according to any one of claims 1 to 8.

13. A hard disk health assessment apparatus, comprising:

14. A computer-readable storage medium storing computer instructions which, when executed by a processor, implement the hard disk health assessment method according to any one of claims 1 to 9.