CN117520086A - Storage performance monitoring method, system and storage medium - Google Patents
Storage performance monitoring method, system and storage medium Download PDFInfo
- Publication number
- CN117520086A CN117520086A CN202311351924.1A CN202311351924A CN117520086A CN 117520086 A CN117520086 A CN 117520086A CN 202311351924 A CN202311351924 A CN 202311351924A CN 117520086 A CN117520086 A CN 117520086A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- information
- configuration information
- performance monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012544 monitoring process Methods 0.000 title claims abstract description 42
- 238000003062 neural network model Methods 0.000 claims abstract description 33
- 230000002159 abnormal effect Effects 0.000 claims abstract description 29
- 238000010606 normalization Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000004458 analytical method Methods 0.000 claims abstract description 16
- 230000000007 visual effect Effects 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 7
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 239000013307 optical fiber Substances 0.000 description 6
- 238000012795 verification Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3041—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is an input/output interface
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
Abstract
The invention discloses a storage performance monitoring method, a system and a storage medium, which comprise the following steps: collecting configuration information of a storage system; checking the integrity and normalization of the configuration information to obtain first data passing the checking, and inputting the first data into a database; acquiring sample data from the first data, and inputting the sample data into a four-element long-short-term memory neural network model obtained by pre-training to obtain a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation; analyzing the reason and the range of abnormal fluctuation based on a preset rule to obtain an analysis result; and visually displaying and storing the performance monitoring result based on the configuration information and the analysis result. The invention can accurately reflect the read-write performance of the service system, and can intuitively reflect the read-write performance of the service system by combining visual display.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a storage performance monitoring method, a storage performance monitoring system, and a storage medium.
Background
Storage performance refers to the speed and efficiency exhibited by a storage system in processing data. Storage performance is typically measured in terms of the IOPS (input/output operands per second) and throughput (data transfer rate) of the storage system. IOPS refers to input/output operands that can be processed by a storage system per second, and is typically used to measure random read-write performance of the storage system; throughput, in turn, refers to the amount of data that a storage system can transfer per second, and is typically used to measure the sequential read-write performance of the storage system.
With the maturity of the 5G age, scenes such as mobile payment, mobile phone banking and the like gradually replace a counter, and become a main channel for users to communicate with the bank. Improving the user experience of a mobile client is an important measure for improving the viscosity of a user, and ensuring quick response is an important measure for improving the user experience. Relative to CPU, memory, and program software processing, stored access is the longest-time portion of the overall underlying hardware and software, often the primary factor affecting user experience.
The storage brands used in the data center at present are numerous, each brand storage interface is independent, the existing storage performance monitoring method mainly depends on a storage monitoring tool provided by a storage manufacturer, and the read-write performance of a service system cannot be intuitively and accurately reflected due to the lack of a unified performance monitoring platform.
Disclosure of Invention
The invention mainly aims to provide a storage performance monitoring method, a storage performance monitoring system and a storage medium, and aims to solve the technical problem that the existing storage performance monitoring method cannot intuitively and accurately reflect the read-write performance of a service system.
In order to achieve the above object, the present invention provides a storage performance monitoring method, which includes the following steps: collecting configuration information of a storage system; checking the integrity and normalization of the configuration information to obtain first data passing the checking, and inputting the first data into a database; acquiring sample data from the first data, and inputting the sample data into a four-element long-short-term memory neural network model obtained by pre-training to obtain a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation; analyzing the reason and the range of abnormal fluctuation based on a preset rule to obtain an analysis result; and visually displaying and storing the performance monitoring result based on the configuration information and the analysis result.
Optionally, collecting configuration information of the storage system at least includes the following steps: collecting one or more than two of stored volume names, volume IDs, port WWNs and space sizes of a storage system through REST API and SSH protocols; collecting optical switch operation information through SSH protocol, wherein the optical switch operation information at least comprises one or two of port login WWN and interface error packet information; and acquiring one or more than two of storage access information of a host and system information, deployment service information and responsible person information registered by the ITSM service desk through SFTP.
Optionally, checking the integrity of the configuration information, specifically: acquiring host information, the number of storage systems and historical data of configuration information recorded by an IT service desk, comparing the historical data with the configuration information, and checking the integrity of the configuration information; the normalization of the configuration information is checked, specifically: and checking the format and the range of the configuration information, and generating corresponding prompt information for the configuration information of the missing data and/or the configuration information of the format error.
Optionally, the sample data is specifically a quadruple < anchor, active 1, active 2, negative >, where anchor represents the target sample, active 1 represents the sample collected from the same host but stored in a different SAN than the target sample, active 2 represents the sample collected from a different host but stored in a different SAN than the target sample, and negative represents the sample collected from a different host but stored in a different SAN than the target sample; the target samples include time, read rate, and write rate.
Optionally, the quaternary long-short-term memory neural network model is specifically obtained through the following training steps: sample data are obtained from the first data, and data normalization processing and shift processing are carried out on the sample data to obtain training samples; taking the storage time as an X axis, and taking each metadata in the sample data as a characteristic to form a Y axis, so as to generate a quaternary data comparison graph; dividing a training sample into a training set and a testing set according to a preset data set dividing proportion, and carrying out dimension lifting processing to obtain a plurality of three-dimensional feature vectors; and (3) inputting the three-dimensional feature vector into a loss function for calculation, updating the super-parameters through back propagation, and generating a quaternary long-short-term memory neural network model.
Alternatively, the loss function takes the form of MSE loss, namely:
wherein f (x) represents a predicted value of the quaternary long-short-term memory neural network model, y represents an actual value, and n represents the number of samples.
Optionally, in the process of analyzing the cause and the range of the abnormal fluctuation, acquiring the stored high-delay volume information and the read rate information of each system, and analyzing the cause and the range of the abnormal fluctuation according to the stored high-delay volume information and the read rate information of each system; the storage high-delay volume information at least comprises one or more than two of delay generation time, delay duration and storage volume names; each system reading rate information at least comprises one or more than two of reading rate generation time, reading rate size and system name.
Optionally, the visual display stores the performance monitoring result, at least including: acquiring and displaying a system list with delay time longer than a preset delay threshold or with read-write rate longer than a preset read-write rate threshold; based on the system list, acquiring a system selection instruction, and displaying the delay time and the read-write rate of the selected system in a designated time interval; and acquiring all delay data, performing aggregation processing to generate a cluster heat map, and displaying.
Corresponding to the storage performance monitoring method, the invention provides a storage performance monitoring system, which comprises: the data acquisition module is used for acquiring configuration information of the storage system; the data processing module is used for checking the integrity and normalization of the configuration information to obtain first data passing the checking and inputting the first data into the database; the prediction module is used for acquiring sample data from the first data, inputting the sample data into a four-element long-short-term memory neural network model obtained through pre-training, and obtaining a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation; the data analysis module is used for analyzing the reason and the range of the abnormal fluctuation based on a preset rule to obtain an analysis result; and the visualization module is used for visually displaying and storing the performance monitoring result based on the configuration information and the analysis result.
In addition, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a storage performance monitoring program which, when executed by a processor, implements the steps of the storage performance monitoring method as described above.
The beneficial effects of the invention are as follows:
(1) Compared with the prior art, the method and the device have the advantages that the configuration information of the storage system is collected, the IO performance data of the system is analyzed by utilizing the quaternary long-short-term memory neural network model to obtain the system generating abnormal fluctuation, the storage performance early warning is carried out in an intelligent analysis mode, the read-write performance of the service system can be accurately reflected, and the read-write performance of the service system can be intuitively reflected by combining visual display;
(2) Compared with the prior art, the method and the device have the advantages that the storage alarm and early warning information is automatically associated with the service system by collecting and storing configuration data and acquiring service data of the service desk, so that the efficiency of positioning the storage performance problem is prevented from being influenced; the integrity and normalization of the first data are ensured by checking the integrity and normalization of the configuration information;
(3) Compared with the prior art, the method and the device have the advantages that the IO performance data of the system are analyzed through the quaternary long-short-term memory neural network model, and the prediction accuracy of the storage performance can be effectively improved;
(4) Compared with the prior art, the method and the device have the advantages that the storage high-delay volume information and the reading rate information of each system are obtained in the process of analyzing the cause and the range of abnormal fluctuation, and the source system of the storage performance problem can be rapidly positioned according to the cause and the range of the abnormal fluctuation generated by analyzing the storage high-delay volume information and the reading rate information of each system;
(5) Compared with the prior art, the method has the advantages that the storage performance monitoring result is visually displayed, the chart can be quickly generated, the information is concise, and the storage performance problem source system can be quickly positioned conveniently; the change trend and the performance state of the selected system can be quickly known, tracking and observation can be continuously carried out, and the range with high delay is concentrated in which time period and which host computers in a certain time interval, so that the affected degree and the affected range of each host computer system in a certain optical fiber link network can be intuitively obtained.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
FIG. 1 is a flow chart of one embodiment of a storage performance monitoring method of the present invention;
FIG. 2 is a block diagram of one embodiment of a storage performance monitoring system according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, a storage performance monitoring method of the present invention includes the following steps: collecting configuration information of a storage system; checking the integrity and normalization of the configuration information to obtain first data passing the checking, and inputting the first data into a database; acquiring sample data from the first data, and inputting the sample data into a four-element long-short-term memory neural network model obtained by pre-training to obtain a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation; analyzing the reason and the range of abnormal fluctuation based on a preset rule to obtain an analysis result; and visually displaying and storing the performance monitoring result based on the configuration information and the analysis result.
According to the invention, the configuration information of the storage system is acquired, the IO performance data of the system is analyzed by utilizing the quaternary long-short-term memory neural network model, so that a system generating abnormal fluctuation is obtained, and the storage performance early warning is performed in an intelligent analysis mode, so that the read-write performance of the service system can be accurately reflected, and the read-write performance of the service system can be intuitively reflected by combining visual display.
Preferably, the predicted value refers to how much IO will arrive at the time when the IO data exceeds a threshold value by the storage system predicting the abnormal fluctuation.
In this embodiment, collecting configuration information of a storage system at least includes the following steps: collecting one or more than two of volume names, volume IDs, port WWNs (world wide names) and space sizes of a storage system through REST API and SSH protocols; collecting optical switch operation information through SSH protocol, wherein the optical switch operation information at least comprises one or two of port login WWN and interface error packet information; and acquiring one or more than two of storage access information of a host and system information, deployment service information and responsible person information registered by the ITSM service desk through SFTP.
In this embodiment, the integrity of the configuration information is checked, specifically: acquiring host information, the number of storage systems and historical data of configuration information recorded by an IT service desk, comparing the historical data with the configuration information, and checking the integrity of the configuration information; the normalization of the configuration information is checked, specifically: and checking the format and the range of the configuration information, and generating corresponding prompt information for the configuration information of the missing data and/or the configuration information of the format error.
Preferably, the corresponding prompt information is pushed to an administrator, so that the administrator can check the prompt information conveniently. And integrating and inputting the first data passing the verification into a database according to the relation and the characteristics of the data.
According to the invention, by collecting storage configuration data and acquiring service data of the service desk, storage alarm and early warning information are automatically associated with a service system, so that the efficiency of positioning the storage performance problem is prevented from being influenced; and verifying the integrity and normalization of the configuration information to ensure the integrity and normalization of the first data.
In this embodiment, the sample data is specifically a quadruple < Anchor, active 1, active 2, negative >, where anchor represents a target sample, active 1 represents a sample collected from a different host than a SAN (storage area network) and stored with the target sample, active 2 represents a sample collected from a different host than a SAN and stored with the target sample, and negative represents a sample collected from a different host than a SAN and stored with the target sample; the target samples include time, read rate, and write rate.
Meanwhile, positive1 and positive2 also represent samples that are more affected by the target sample, and negative and Anchor represent samples that are less affected by the target sample. In this embodiment, whether the samples are affected is mainly whether the samples are in the same SAN network, and the same ports are used in the same SAN network, so that in theory, the samples of the same host are affected more than the samples of the same storage.
In this embodiment, the quaternary long-short-term memory neural network model is specifically obtained through the following training steps: sample data are obtained from the first data, and data normalization processing and shift processing are carried out on the sample data to obtain training samples; taking the storage time as an X axis, and taking each metadata in the sample data as a characteristic to form a Y axis, so as to generate a quaternary data comparison graph; dividing a training sample into a training set and a testing set according to a preset data set dividing proportion, and carrying out dimension lifting processing to obtain a plurality of three-dimensional feature vectors; and (3) inputting the three-dimensional feature vector into a loss function for calculation, updating the super-parameters through back propagation, and generating a quaternary long-short-term memory neural network model.
According to the invention, the storage time is taken as the X axis, each metadata in the sample data is taken as the characteristic to form the Y axis, and the quaternary data comparison graph is generated, so that four groups of affected different data can be conveniently and intuitively compared.
Preferably, the three-dimensional feature vector represents a temporal feature of the sample data.
Because the correlation of the optical fiber link IO data before and after is lower and is influenced by holidays, the performance of the holidays IO is different from that of the workdays IO, and the performance data of the optical fiber link cannot be well fitted by using a traditional time sequence prediction algorithm, the prediction is performed through a quaternary long-short-term memory neural network model. The quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain a system generating abnormal fluctuation, and particularly a system with IO rate exceeding 1200 MB/S.
In this embodiment, the loss function takes the form of MSE loss, namely:
wherein f (x) represents a predicted value, y represents an actual value of the quaternary long-short-term memory neural network model, and n represents the number of samples.
In this embodiment, the existing data is divided into two parts, one part is used for training the quaternary long-short-period memory neural network model, the other part is used for verification, the quaternary long-short-period memory neural network model is used for generating a predicted value, and predicting when the abnormal fluctuation storage system has IO data exceeding a threshold value, the IO will reach at that time; the value used for verification in the existing data is an actual value, and the difference between the predicted value and the actual value is calculated through an MSE loss function and is used for representing the prediction accuracy of the quaternary long-short-term memory neural network model.
Preferably, n=4.
According to the invention, the IO performance data of the system is analyzed through the quaternary long-short-term memory neural network model, so that the prediction accuracy of the storage performance can be effectively improved. In this embodiment, all the first data are input into the pre-trained quaternary long-short-term memory neural network model, so as to obtain a predicted value after sample data normalization, and then the predicted value is subjected to inverse normalization, so that the predicted value can be obtained.
In the embodiment, in the process of analyzing the cause and the range of the abnormal fluctuation, acquiring the stored high-delay volume information and the read rate information of each system, and analyzing the cause and the range of the abnormal fluctuation according to the stored high-delay volume information and the read rate information of each system; the storage high-delay volume information at least comprises one or more than two of delay generation time, delay duration and storage volume names; each system reading rate information at least comprises one or more than two of reading rate generation time, reading rate size and system name.
According to the method, the system and the device, the storage high-delay volume information and the reading rate information of each system are obtained in the process of analyzing the cause and the range of abnormal fluctuation, and the source system of the storage performance problem can be rapidly positioned according to the cause and the range of the abnormal fluctuation generated by analyzing the storage high-delay volume information and the reading rate information of each system.
In this embodiment, the visual display of the storage performance monitoring result at least includes: acquiring and displaying a system list with delay time longer than a preset delay threshold or with read-write rate longer than a preset read-write rate threshold, wherein the part of the chart is fast to generate and concise in information, and a problem source system can be rapidly positioned; based on a system list, acquiring a system selection instruction, displaying the delay time and the read-write speed of the selected system in a specified time interval, and enabling the part to overview the delay and the read-write speed of the system in a certain time interval, so that the change trend and the performance state of the system can be rapidly known, and tracking and observation can be continuously carried out; and acquiring all delay data, performing aggregation processing to generate a cluster heat map, and displaying the cluster heat map, wherein the part can list which time period and which host are concentrated in a range with high delay in a certain time interval, and can intuitively obtain the affected degree and the affected range of each host system in a certain optical fiber link network.
Preferably, the preset delay threshold is 50ms. And judging the source possibly causing the congestion according to the time point generated by the roll height delay and the high reading rate time point which is coincident with two minutes before and after the time point. And finally, the data of storage, a host, optical fiber switch information, a service system, a responsible person and the like recorded on the database through the data processing layer are intuitively displayed to the affected service range of the manager.
According to the invention, the storage performance monitoring result is visually displayed, so that a chart can be quickly generated, the information is concise, and a storage performance problem source system can be quickly positioned; the change trend and the performance state of the selected system can be quickly known, tracking and observation can be continuously carried out, and the range with high delay is concentrated in which time period and which host computers in a certain time interval, so that the affected degree and the affected range of each host computer system in a certain optical fiber link network can be intuitively obtained.
As shown in fig. 2, the present invention further correspondingly provides a storage performance monitoring system, which includes: the data acquisition module 10 is used for acquiring configuration information of the storage system; the data processing module 20 is configured to verify the integrity and normalization of the configuration information, obtain first data that passes the verification, and input the first data into the database; the prediction module 30 is configured to obtain sample data from the first data, and input the sample data into a pre-trained quaternary long-short-term memory neural network model to obtain a predicted value after normalization of the sample data; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation; the data analysis module 40 is configured to analyze the cause and the range of the abnormal fluctuation based on a preset rule, and obtain an analysis result; and a visualization module 50, configured to visually display the storage performance monitoring result based on the configuration information and the analysis result.
The embodiment of the present invention also provides a computer readable storage medium, which may be a computer readable storage medium contained in the memory in the above embodiment; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer readable storage medium has stored therein at least one instruction that is loaded and executed by a processor to implement the storage performance monitoring method shown in fig. 1. The computer readable storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the device embodiments, the apparatus embodiments, and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the description of the method embodiments for relevant points.
Also, herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
While the foregoing description illustrates and describes the preferred embodiments of the present invention, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as limited to other embodiments, but is capable of use in various other combinations, modifications and environments and is capable of changes or modifications within the scope of the inventive concept, either as described above or as a matter of skill or knowledge in the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.
Claims (10)
1. A storage performance monitoring method, comprising the steps of:
collecting configuration information of a storage system;
checking the integrity and normalization of the configuration information to obtain first data passing the checking, and inputting the first data into a database;
acquiring sample data from the first data, and inputting the sample data into a four-element long-short-term memory neural network model obtained by pre-training to obtain a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation;
analyzing the reason and the range of abnormal fluctuation based on a preset rule to obtain an analysis result;
and visually displaying and storing the performance monitoring result based on the configuration information and the analysis result.
2. The storage performance monitoring method according to claim 1, wherein: the method for collecting the configuration information of the storage system at least comprises the following steps:
collecting one or more than two of stored volume names, volume IDs, port WWNs and space sizes of a storage system through REST API and SSH protocols;
collecting optical switch operation information through SSH protocol, wherein the optical switch operation information at least comprises one or two of port login WWN and interface error packet information;
and acquiring one or more than two of storage access information of a host and system information, deployment service information and responsible person information registered by the ITSM service desk through SFTP.
3. The storage performance monitoring method according to claim 1, wherein: the integrity of the configuration information is checked, specifically: acquiring host information, the number of storage systems and historical data of configuration information recorded by an IT service desk, comparing the historical data with the configuration information, and checking the integrity of the configuration information;
the normalization of the configuration information is checked, specifically: and checking the format and the range of the configuration information, and generating corresponding prompt information for the configuration information of the missing data and/or the configuration information of the format error.
4. The storage performance monitoring method according to claim 1, wherein: the sample data is specifically a quadruple < Anchor, active 1, active 2, negative >, where anchor represents a target sample, active 1 represents a sample stored with the target sample from a different host than a SAN, and active 2 represents a sample stored with the target sample from a different host than a SAN, and negative represents a sample stored with the target sample from a different host than a SAN;
the target samples include time, read rate, and write rate.
5. The storage performance monitoring method according to claim 4, wherein: the quaternary long-short-term memory neural network model is obtained specifically through the following training steps:
sample data are obtained from the first data, and data normalization processing and shift processing are carried out on the sample data to obtain training samples;
taking the storage time as an X axis, and taking each metadata in the sample data as a characteristic to form a Y axis, so as to generate a quaternary data comparison graph;
dividing a training sample into a training set and a testing set according to a preset data set dividing proportion, and carrying out dimension lifting processing to obtain a plurality of three-dimensional feature vectors;
and (3) inputting the three-dimensional feature vector into a loss function for calculation, updating the super-parameters through back propagation, and generating a quaternary long-short-term memory neural network model.
6. The storage performance monitoring method according to claim 5, wherein: the loss function takes the form of MSE loss, namely:
wherein f (x) represents a predicted value of the quaternary long-short-term memory neural network model, y represents an actual value, and n represents the number of samples.
7. The storage performance monitoring method according to claim 1, wherein: in the process of analyzing the cause and the range of the abnormal fluctuation, acquiring the stored high-delay volume information and the read rate information of each system, and analyzing the cause and the range of the abnormal fluctuation according to the stored high-delay volume information and the read rate information of each system;
the storage high-delay volume information at least comprises one or more than two of delay generation time, delay duration and storage volume names; each system reading rate information at least comprises one or more than two of reading rate generation time, reading rate size and system name.
8. The storage performance monitoring method according to claim 7, wherein: the visual display storage performance monitoring result at least comprises:
acquiring and displaying a system list with delay time longer than a preset delay threshold or with read-write rate longer than a preset read-write rate threshold;
based on the system list, acquiring a system selection instruction, and displaying the delay time and the read-write rate of the selected system in a designated time interval;
and acquiring all delay data, performing aggregation processing to generate a cluster heat map, and displaying.
9. A storage performance monitoring system, comprising:
the data acquisition module is used for acquiring configuration information of the storage system;
the data processing module is used for checking the integrity and normalization of the configuration information to obtain first data passing the checking and inputting the first data into the database;
the prediction module is used for acquiring sample data from the first data, inputting the sample data into a four-element long-short-term memory neural network model obtained through pre-training, and obtaining a predicted value of the sample data after normalization; the quaternary long-short-term memory neural network model is used for analyzing IO performance data of the system to obtain the system generating abnormal fluctuation;
the data analysis module is used for analyzing the reason and the range of the abnormal fluctuation based on a preset rule to obtain an analysis result;
and the visualization module is used for visually displaying and storing the performance monitoring result based on the configuration information and the analysis result.
10. A computer readable storage medium, wherein a storage performance monitoring program is stored on the computer readable storage medium, which when executed by a processor implements the steps of the storage performance monitoring method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311351924.1A CN117520086A (en) | 2023-10-18 | 2023-10-18 | Storage performance monitoring method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311351924.1A CN117520086A (en) | 2023-10-18 | 2023-10-18 | Storage performance monitoring method, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117520086A true CN117520086A (en) | 2024-02-06 |
Family
ID=89761563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311351924.1A Pending CN117520086A (en) | 2023-10-18 | 2023-10-18 | Storage performance monitoring method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117520086A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190334786A1 (en) * | 2018-04-30 | 2019-10-31 | Hewlett Packard Enterprise Development Lp | Predicting Workload Patterns in a Data Storage Network |
| CN110399238A (en) * | 2019-06-27 | 2019-11-01 | 浪潮电子信息产业股份有限公司 | A kind of disk failure method for early warning, device, equipment and readable storage medium storing program for executing |
| US20200160211A1 (en) * | 2018-11-21 | 2020-05-21 | Sap Se | Machine learning based database anomaly prediction |
| CN112488395A (en) * | 2020-12-01 | 2021-03-12 | 湖南大学 | Power distribution network line loss prediction method and system |
| CN113886207A (en) * | 2021-10-09 | 2022-01-04 | 济南浪潮数据技术有限公司 | Method and device for predicting performance of storage system based on convolutional neural network |
| CN114548509A (en) * | 2022-01-18 | 2022-05-27 | 湖南大学 | Multi-type load joint prediction method and system for multi-energy system |
| CN115115030A (en) * | 2022-06-24 | 2022-09-27 | 平安科技（深圳）有限公司 | System monitoring method and device, electronic equipment and storage medium |
-
2023
- 2023-10-18 CN CN202311351924.1A patent/CN117520086A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190334786A1 (en) * | 2018-04-30 | 2019-10-31 | Hewlett Packard Enterprise Development Lp | Predicting Workload Patterns in a Data Storage Network |
| US20200160211A1 (en) * | 2018-11-21 | 2020-05-21 | Sap Se | Machine learning based database anomaly prediction |
| CN110399238A (en) * | 2019-06-27 | 2019-11-01 | 浪潮电子信息产业股份有限公司 | A kind of disk failure method for early warning, device, equipment and readable storage medium storing program for executing |
| CN112488395A (en) * | 2020-12-01 | 2021-03-12 | 湖南大学 | Power distribution network line loss prediction method and system |
| CN113886207A (en) * | 2021-10-09 | 2022-01-04 | 济南浪潮数据技术有限公司 | Method and device for predicting performance of storage system based on convolutional neural network |
| CN114548509A (en) * | 2022-01-18 | 2022-05-27 | 湖南大学 | Multi-type load joint prediction method and system for multi-energy system |
| CN115115030A (en) * | 2022-06-24 | 2022-09-27 | 平安科技（深圳）有限公司 | System monitoring method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107122669B (en) | Method and device for evaluating data leakage risk | |
| US8983936B2 (en) | Incremental visualization for structured data in an enterprise-level data store | |
| US20230188452A1 (en) | Performance monitoring in a distributed storage system | |
| US8024613B2 (en) | Method and system for managing apparatus performance | |
| WO2019153487A1 (en) | System performance measurement method and device, storage medium and server | |
| CN107391359B (en) | Service testing method and device | |
| CN105488539B (en) | The predictor method and device of the generation method and device of disaggregated model, power system capacity | |
| CN107851106A (en) | It is the resource scaling of the automatic requirement drive serviced for relational database | |
| TW201732789A (en) | Disk failure prediction method and apparatus | |
| US11507489B2 (en) | Interactive model performance monitoring | |
| US10545972B2 (en) | Identification and elimination of non-essential statistics for query optimization | |
| CN113313538A (en) | User consumption capacity prediction method and device, electronic equipment and storage medium | |
| CN108810144A (en) | A kind of data transmission method, server and storage medium | |
| CN110751354B (en) | Abnormal user detection method and device | |
| CN114490375A (en) | Method, device and equipment for testing performance of application program and storage medium | |
| CN117520086A (en) | Storage performance monitoring method, system and storage medium | |
| CN114697127B (en) | Service session risk processing method based on cloud computing and server | |
| CN110263618A (en) | The alternative manner and device of one seed nucleus body model | |
| CN109726086A (en) | The method and apparatus of testing server performance | |
| CN109598525A (en) | Data processing method and device | |
| CN109992614B (en) | Data acquisition method, device and server | |
| CN113296951A (en) | Resource allocation scheme determination method and equipment | |
| CN109933604A (en) | The method and device that the patented technology of a kind of couple of rival is monitored | |
| CN113742226B (en) | Software performance test method and device, medium and electronic equipment | |
| CN112596964B (en) | Disk fault prediction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |