CN115269322A

CN115269322A - Universal hard disk state monitoring method based on multi-dimensional data fusion

Info

Publication number: CN115269322A
Application number: CN202210881328.3A
Authority: CN
Inventors: 邓玲; 刘彬彬; 王振帅
Original assignee: Beijing Institute of Computer Technology and Applications
Current assignee: Beijing Institute of Computer Technology and Applications
Priority date: 2022-07-26
Filing date: 2022-07-26
Publication date: 2022-11-01

Abstract

The invention relates to a general hard disk state monitoring method based on multi-dimensional data fusion, and belongs to the technical field of data storage. In the scheme, a set of statistical method of a disk space state lightweight mapping table DMAP and a disk fine-grained IO is designed; transversely comparing multidimensional characteristic data obtained by SMART data, kernel logs, DMAP (data access point) and IO (input/output) fine-grained information processing and screening, and quickly positioning an abnormal hard disk in real time; by carrying out fusion decision on abnormal hard disk multidimensional data, state misjudgment is eliminated, and a slow disk and a fault disk are accurately found. The invention is a lightweight application technology, can provide the service of quickly positioning the slow disk and the bad disk without influencing the storage service, and effectively supports the high performance and the high reliability of the storage system.

Description

Universal hard disk state monitoring method based on multi-dimensional data fusion

Technical Field

The invention belongs to the technical field of data storage, and particularly relates to a general hard disk state monitoring method based on multi-dimensional data fusion.

Background

In the production environment of storage systems, hard disk problems occur at high frequency, threatening data security and storage system performance. Under the performance requirements of high concurrency and high bandwidth, the hard disk failure is not a main factor for restricting the performance of the storage system, and the slow disk becomes a main factor for restricting the performance of medium and small-sized storage systems. The existing hard disk state monitoring mostly uses hard disk SMART information and a machine learning method based on the SMART information. But SMART messages cannot report errors before failures and can only report a few failures; hard disks of different manufacturers and models report different information amounts, for example, SAS disks and SSD report only a few features, so the machine learning method is only suitable for a hard disk of a certain model with a large amount of data, and cannot find a slow disk in the system, which is not a general hard disk state monitoring technology. Aiming at medium and small storage systems and hard disks reporting a small amount of SMART information, a feasible and general hard disk state monitoring technology is urgently needed.

Disclosure of Invention

Technical problem to be solved

The technical problem to be solved by the invention is as follows: the method for monitoring the abnormal state is practical and usable in the production environment and is suitable for various mechanical disks and solid-state disks of small and medium-sized storage systems.

(II) technical scheme

In order to solve the technical problem, the invention provides a general hard disk state monitoring method based on multi-dimensional data fusion, which comprises the following steps:

collecting multidimensional data of a hard disk in a storage system, wherein the multidimensional data comprises hard disk SMART data, kernel log data, lightweight mapping table DMAP data, IO fine-grained statistical data and storage system operation log data;

obtaining multidimensional characteristic data by performing model matching, key information extraction or statistical analysis on the multidimensional data, and performing data screening on the multidimensional characteristic data by using operation log data;

and comparing the screened multidimensional characteristic data, performing fusion processing and decision-making on the obtained abnormal data, and positioning the hard disk in two states of slow speed and failure.

Preferably, after the multidimensional data is collected, the DMAP data therein is also managed as follows:

firstly, the initialization management of DMAP data is carried out, the address space of a physical hard disk is sliced according to the block size of 1024M, each block address space corresponds to one byte of the DMAP data, the number of times of data inconsistency of the corresponding address space is recorded, and the state of one hard disk is represented by a plurality of bytes.

After initialization of DMAP data is completed, after the data is verified by the storage system, calling an API (application programming interface) to submit a verification result, and analyzing a mapping relation between a data address and a physical address based on the verification result;

and finally, maintaining dynamic mapping of the DMAP data and the physical address of the hard disk, and identifying the increased and decreased physical disk.

Preferably, the first three of the collected hard disk SMART data, kernel log data, storage system operation log data, DMAP data, and IO fine-grained statistical data are data that can be directly provided by the storage system, and the second two are two kinds of data generated for characterizing the state of the hard disk, wherein the storage system operation log data do not participate in the lateral comparison.

Preferably, the obtaining of the multidimensional feature data by performing model matching, key information extraction or statistical analysis on the multidimensional data specifically comprises:

firstly, carrying out model matching on SMART data of a hard disk to obtain a series of SMART characteristics; secondly, by analyzing the Kernel log data, warning and error information related to the hard disk are extracted, and Kernel features are obtained; then, DMAP (Dimethylacetamide) characteristics are obtained by analyzing and counting DMAP data obtained by verifying data of a storage system; and finally, obtaining fine-grained IO characteristics according to IO fine-grained statistical data, namely the information of time delay, bandwidth, occupation ratio and retry times of IO with different sizes and grades in a period of time, wherein the SMART characteristics, the Kernel characteristics, the DMAP characteristics and the fine-grained IO characteristics form multidimensional characteristic data.

Preferably, in the collected multidimensional data, the obtaining manner of the IO fine-grained statistical information is as follows:

according to the size of IO, the IO is divided into four levels: 0-4K,4-16K,16-512K, >512K, running a statistical service in a kernel state, collecting information of IO requests each time, performing statistical analysis once per unit time, storing information of IO different-grade occupation ratios, time delay, bandwidth and retry times into a circular queue, and sending data to an application program through a socket.

Preferably, the specific process of performing data screening on the multidimensional feature data by using the operation log data is as follows:

analyzing the operation log data of the storage system, and extracting illegal operation information;

listing data with time correlation between the multidimensional characteristic data and illegal operation;

and analyzing the correlation between the operation and the feature data, and filtering out strongly correlated data in the multi-dimensional feature data.

Preferably, when comparing the screened multi-dimensional feature data, the comparison object is a same type of hard disk, that is, a hard disk with the same model and the same role and status in the storage system; in the transverse comparison process of the multi-dimensional data, comparing the characteristics one by one to find out the characteristics with abnormal deviation and the hard disk, wherein the judgment standard for judging whether the abnormal deviation exists is preset; and the contrast of each dimension is established on the basis of element consistency.

Preferably, when multi-dimensional feature data of the same type of hard disk is transversely compared, the transverse comparison process comprises three parts, namely system analysis, hard disk classification and data comparison:

firstly, analyzing a storage system, and marking a label for a hard disk according to the characteristics and data storage rules of the storage system and the role, the position and the model;

secondly, dividing the hard disks into different groups according to different labels of the hard disks, wherein the hard disks in the same group have the same role and model, and the storage system uniformly distributes data on the hard disks;

and finally, comparing the multidimensional characteristic data of the hard disks one by one according to groups, finding out the hard disks with abnormality, and preparing data for abnormal data fusion processing and decision making.

Preferably, the obtained abnormal data is subjected to fusion processing and decision making, when the hard disk in two states of low speed and fault is positioned, relevant features in the multi-dimensional feature data are fused to obtain a series of irrelevant features, and finally, the states of the hard disk including normal, low speed and fault are further subjected to weighted summation for decision making to obtain the states of the hard disk; during weighted summation, different decision purposes correspond to different weighting coefficients, and for slow disk decision, the weight proportion of the fine-grained IO characteristic and the SMART characteristic is larger than that of the other two characteristics; for bad disk decision, the weight ratio of the DMAP feature, the Kernel feature and the SMART feature is larger than that of the fine-grained IO feature. The decision result is two binary (0, 1) values, which respectively represent the detection results of the slow speed and the fault of the hard disk.

The invention also provides a monitoring system for realizing the method.

(III) advantageous effects

In order to accurately position a slow disk and a bad disk in a storage system in time, the invention designs a general hard disk state monitoring method based on multi-dimensional data fusion. In the scheme, a set of statistical method of a disk space state lightweight mapping table DMAP and a disk fine-grained IO is designed; transversely comparing multidimensional characteristic data obtained by SMART data, kernel logs, DMAP (data access point) and IO (input/output) fine-grained information processing and screening, and quickly positioning an abnormal hard disk in real time; by carrying out fusion decision on abnormal hard disk multidimensional data, state misjudgment is eliminated, and a slow disk and a fault disk are accurately found. The invention is a lightweight application technology, can provide the service of quickly positioning the slow disk and the bad disk without influencing the storage service, and effectively supports the high performance and the high reliability of the storage system.

Drawings

FIG. 1 is a multi-dimensional data fusion decision architecture diagram of the present invention;

FIG. 2 is a DMAP management architecture diagram of the present invention;

FIG. 3 is a schematic diagram of the cross-comparison service design of the present invention;

FIG. 4 is a multi-dimensional data fusion diagram of the present invention.

Detailed Description

In order to make the objects, contents and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.

The invention provides a general hard disk state monitoring method based on multi-dimensional data fusion. As shown in fig. 1, the method of the present invention is implemented by using five modules, where the data acquisition module is used to acquire multidimensional data of a hard disk in a storage system, including hard disk SMART data, kernel log data, DMAP data, IO fine-grained statistical data, and storage system operation log data, the data preprocessing module is used to perform model matching, key information extraction, or statistical analysis on the multidimensional data to obtain multidimensional feature data, and perform data screening on the multidimensional feature data by using the operation log data, and the transverse comparison module is used to compare the multidimensional feature data obtained by screening, and submit abnormal data to the data fusion decision module for fusion processing and decision.

In the multidimensional data collected by the data collection module, the management method of the lightweight mapping table DMAP is as follows:

the invention provides a set of general hard Disk space state mapping table (DMAP, disk status MAP) operation API by utilizing the data checking function of a storage system, and FIG. 2 describes a management architecture of the DMAP, wherein a DMAP Manager module is the core of the whole architecture, and the management of the DMAP is mainly divided into three processes of initialization, updating and remapping.

Firstly, the DMAP is initialized and managed, the DMAP Manager slices the address space of a physical hard disk according to the block size of 1024M, each block address space corresponds to one byte of the DMAP, the number of times of data inconsistency of the corresponding address space is recorded, and the state of one hard disk is represented by a plurality of bytes. In general, the probability of data bit flipping and write error of a healthy hard disk is close to 0.

After the initialization of the DMAP is finished, after the data verification module of the storage system verifies the data, the API is called to submit the verification result to the DMAP manager, and the DMAP manager analyzes the mapping relation between the data address and the physical address and updates the result into the DMAP.

And finally, the DMAP manager maintains the dynamic mapping between the DMAP and the physical address of the hard disk, and identifies the increased and decreased physical disks.

The data acquisition module is the basis of the invention, and in the acquired SMART data, kernel log data, storage system operation log data, DMAP data and IO fine-grained statistical data, the first three are data which can be directly provided by a storage system, and the last two are two kinds of data which are generated by the invention for accurately representing the state of the hard disk, wherein the storage system operation log data do not participate in transverse comparison. In order to obtain multi-dimensional characteristic data, a data preprocessing module firstly carries out model matching on the SMART data of the hard disk to obtain a series of SMART characteristics; secondly, extracting warning and error information related to the hard disk through analysis of the Kernel log to obtain Kernel characteristics; then, DMAP (data access control) characteristics are obtained by analyzing and counting DMAP data obtained by verifying data of a storage system; and finally, obtaining fine-grained IO characteristics according to the information such as the time delay, the bandwidth, the occupation ratio, the retry times and the like of IO with different levels within a period of time. The SMART characteristics, the Kernel characteristics, the DMAP characteristics and the fine-grained IO characteristics form multi-dimensional characteristic data.

In the multidimensional data collected by the data collection module, the acquisition mode of IO fine-grained statistical information is as follows:

according to the invention, the hard disk IO statistics are divided into more fine-grained types, so that the IO statistics of each time can be conveniently subjected to component analysis. According to the size of IO, the IO is divided into four levels: 0-4K,4-16K,16-512K, >512K. The statistical service runs in a kernel mode and is responsible for collecting information of IO requests every time, performing statistical analysis once per unit time, storing IO different-grade proportion, time delay, bandwidth and re-request information (retry times) into a circular queue, and sending data to an application program through a socket.

The multi-dimensional feature data can be used for transverse comparison only through the most critical data screening process, data screening is an important link for reducing misjudgment, and the feature data which is easy to misjudge and formed due to non-compliant operation and maintenance operation is eliminated. The specific implementation process comprises the following steps:

The core of the invention is the transverse comparison of multi-dimensional characteristic data, and the basis (comparison object) is the same type of hard disk, namely, the hard disks with the same model and the same role and status in the storage system. The design goal of modern storage systems is uniform data distribution and symmetric architecture, and most storage systems in the same batch use the same type of hard disk in a production environment, which provides a material basis for lateral contrast. In the transverse comparison of the multidimensional data, one-to-one comparison is carried out according to the characteristics, and the characteristics with abnormal deviation and the hard disk are found out.

The essence of the algorithm of the transverse comparison is that the comparison of each dimension is established on the basis of consistent elements, for example, the IO components are required to be the same when the fine-grained IO feature comparison is carried out, and the information required by the Kernel feature comparison is of the same type.

The transverse comparison module carries out transverse comparison on the multi-dimensional characteristic data of the same hard disk in the following way:

the invention finds out the hard disk with abnormal state by transversely comparing the similar hard disks, does not depend on big data analysis, and can effectively operate in medium and small file systems. As shown in fig. 3, the horizontal comparison process mainly includes three parts, i.e., system analysis, hard disk classification, and data comparison.

Firstly, analyzing a storage system, wherein the currently supported main storage systems comprise RAID, CEPH and Gluster, and labeling labels for hard disks according to roles, positions, models and the like according to the characteristics of the storage system and data storage rules;

secondly, according to different labels of the hard disks, dividing the hard disks into different groups, wherein the hard disks in the same group have the same role and model, and the storage system uniformly distributes data on the hard disks, which should have similar external characteristics;

and finally, comparing the multidimensional characteristic data of the hard disks one by one according to groups, finding out the hard disks with abnormity, and preparing data for multidimensional data fusion decision.

Designing a multidimensional data fusion decision model of the data fusion decision module:

the invention realizes the processing of multi-dimensional data fusion and avoids the performance and cost loss of a storage system caused by the misjudgment of single-dimensional data decision. As shown in fig. 4, the multidimensional data mainly includes hard disk SMART data, DMAP data, kernel log data, and IO fine-grained statistical data, and these data form a plurality of feature data after data preprocessing. And performing multi-dimensional data fusion to fuse related features in the feature data to obtain a series of irrelevant features, and finally performing further weighting to make a decision to obtain the state (normal, slow and fault) of the hard disk. The multidimensional data fusion decision is the final purpose to be realized by the invention, the data fusion process is a process of relevant characteristic fusion, characteristic numeralization and weighted summation, and different decision purposes correspond to different weighting coefficients. For slow disk decision, the weight ratio of IO characteristics and SMART characteristics is large; the DMAP feature, kernel feature, and SMART feature are focused on bad disk decisions. The decision result is two binary values (0 and 1) which respectively represent the detection results of the slow speed and the fault of the hard disk and are used by a storage system and operation and maintenance personnel.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims

1. A general hard disk state monitoring method based on multi-dimensional data fusion is characterized by comprising the following steps:

2. The method of claim 1, wherein after collecting the multidimensional data, the DMAP data therein is further managed as follows:

3. The method of claim 1, wherein the collected SMART data, kernel log data, operation log data of the storage system, DMAP data and IO fine-grained statistical data of the hard disk are data which can be directly provided by the storage system, and the collected SMART data, kernel log data, operation log data of the storage system, DMAP data and IO fine-grained statistical data of the storage system are two kinds of data which are generated for representing the state of the hard disk, wherein the operation log data of the storage system does not participate in transverse comparison.

4. The method according to claim 3, wherein the obtaining of the multidimensional feature data by performing model matching, key information extraction, or statistical analysis on the multidimensional data specifically comprises:

firstly, carrying out model matching on SMART data of a hard disk to obtain a series of SMART characteristics; secondly, by analyzing the Kernel log data, warning and error information related to the hard disk are extracted, and Kernel features are obtained; then, DMAP (data access control) characteristics are obtained by analyzing and counting DMAP data obtained by verifying data of a storage system; and finally, obtaining fine-grained IO characteristics according to IO fine-grained statistical data, namely information of time delay, bandwidth, occupation ratio and retry times of IO with different sizes and grades in a period of time, wherein the SMART characteristics, the Kernel characteristics, the DMAP characteristics and the fine-grained IO characteristics form multidimensional characteristic data.

5. The method according to claim 4, wherein in the collected multidimensional data, the IO fine-grained statistical information is obtained as follows:

6. The method of claim 5, wherein the specific process of performing data filtering on the multidimensional feature data by using the operation log data is as follows:

analyzing the data of the operation log of the storage system, and extracting illegal operation information;

7. The method of claim 5, wherein when comparing the filtered multi-dimensional feature data, the comparison objects are the same type of hard disks, that is, hard disks with the same model and the same role and status in the storage system; in the transverse comparison process of the multi-dimensional data, comparing the characteristics one by one to find out the characteristics with abnormal deviation and the hard disk, wherein the judgment standard for judging whether the abnormal deviation exists is preset; and the comparison of each dimension is established on the basis of element consistency.

8. The method of claim 7, wherein when performing the transverse comparison on the multi-dimensional feature data of the same type of hard disk, the transverse comparison process comprises three parts, namely system analysis, hard disk classification and data comparison:

9. The method of claim 8, wherein the obtained abnormal data is subjected to fusion processing and decision making, when the hard disk with two states of low speed and failure is positioned, the related features in the multidimensional feature data are fused to obtain a series of irrelevant features, and finally the decision making is further performed by weighted summation to obtain the states of the hard disk, including three states of normal, low speed and failure; during weighted summation, different decision purposes correspond to different weighting coefficients, and for slow disk decision, the weight proportion of the fine-grained IO characteristic and the SMART characteristic is larger than that of the other two characteristics; for bad disk decision, the weight ratio of the DMAP feature, the Kernel feature and the SMART feature is larger than that of the fine-grained IO feature. The decision result is two binary (0, 1) values, which respectively represent the detection results of the slow speed and the fault of the hard disk.

10. A monitoring system for implementing the method of any one of claims 1 to 9.