CN110851317A - Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment - Google Patents
Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment Download PDFInfo
- Publication number
- CN110851317A CN110851317A CN201910819729.4A CN201910819729A CN110851317A CN 110851317 A CN110851317 A CN 110851317A CN 201910819729 A CN201910819729 A CN 201910819729A CN 110851317 A CN110851317 A CN 110851317A
- Authority
- CN
- China
- Prior art keywords
- data
- iops
- fitting
- storage
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3037—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
Abstract
The invention provides a method, a device, equipment and a storage medium for predicting storage equipment IOPS performance data, which are used for acquiring the storage equipment IOPS performance data and storing the storage equipment IOPS performance data into a database; respectively storing the data into corresponding data tables according to different data types; extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array; fitting the extracted data; and displaying the prediction data obtained after the fitting treatment. The method for predicting the performance data of each IOPS of the storage equipment by the polynomial curve fitting can predict the future IOPS numerical value more accurately, further provide help for reasonably managing and using hardware equipment by most of the storage equipment through the predicted IOPS, is intuitive and strong in practicability, and can intuitively see the stored historical data and predicted data curve of each IOPS of the storage equipment in a chart after the processing.
Description
Technical Field
The invention relates to the field of computer storage systems, in particular to a method, a device, equipment and a storage medium for predicting IOPS performance data of storage equipment.
Background
With the rapid development of scientific computing and various Network applications, the amount of information generated by human beings is increasing, and the Storage of data is more and more concerned, so that the Storage component is more and more important in the whole Computer System, and the Storage is already turned to a disk array by a single disk and a tape, and further developed to the currently popular Storage Network, such as NAS (Network Storage Technologies), SAN (Storage Area Network ), iscsi (Internet Small Computer System Interface), and the like.
The demand of large-scale data application is continuously emerging, mass data and application thereof become a new development direction, data storage has generated great influence on the work and life of people, and the prediction and analysis of various performance data of storage equipment are naturally more and more important. The performance data prediction analysis of the storage device can be used for effectively monitoring the operation condition of the storage device, analyzing the quality of the storage device and being used as a basis for allocating and managing device resources in the future, so that a reasonable method for predicting the performance of the storage device is particularly important.
Generally, an IOPS (I/O per second) performance data index of a storage device, i.e., the maximum I/O number per second, can be used to measure the maximum file concurrency number that a disk, an array, or a channel can provide in an environment where a large number of small files are concurrently read and written randomly, such as many aspects of database applications. In practical situations, the IOPS statistics of the storage device are a group of fluctuating values, and it is not easy to determine the size and trend of the future statistics.
Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for predicting storage equipment IOPS performance data, wherein the large-scale data application requirements are continuously emerging, mass data and application thereof become a new development direction, data storage has great influence on the work and life of people, and the invention aims at solving the problems that the storage equipment IOPS statistical data are a group of fluctuating values and the size and trend of the future statistical data are not easy to judge.
The technical scheme of the invention is as follows:
in a first aspect, a technical solution of the present invention provides a method for predicting IOPS performance data of a storage device, including the following steps:
extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
fitting the extracted data;
and displaying the prediction data obtained after the fitting treatment.
Preferably, the step of extracting all data of the IOPS performance of the storage device in the set time period and storing the data into the corresponding array includes:
acquiring IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
and storing the data into corresponding data tables according to different data types.
Preferably, the step of fitting the extracted data includes:
and obtaining a polynomial fitting function after quantitatively analyzing the IOPS performance data of the storage equipment by calling a curve fitting function ployfit.
Preferably, the step of displaying the prediction data obtained after the fitting process includes:
and carrying in future time points through a polynomial fitting function to obtain prediction data, and displaying each IOPS actual value and each predicted value on a Web page through an open source chart display tool.
Preferably, in the step of obtaining a polynomial fitting function by calling a curve fitting function ployfit and quantitatively analyzing the IOPS performance data of the storage device, the polynomial fitting function is: p [ ] ployfit (X [ ], Y [ ], N)
wherein, i represents each predicted time point in the future time period viewed by the user, i is 1, 2, 3 … …;
p [ ]: a polynomial coefficient representing a decreasing power permutation of the fitting function polynomial expression;
x [ ]: representing each time point array corresponding to the IOPS data stored in the database;
y [ ]: actual value arrays representing the corresponding time points of the IOPS stored in the database;
n: representing the order of the fitting polynomial, and selecting a natural number more than 4 by N according to the actual situation;
forecast [ i ]: and indicating the IOPS predicted value corresponding to the predicted future time point i.
Preferably, the step of storing the data into the corresponding data tables according to different data types further includes:
and setting the storage time limit of each piece of data and automatically clearing redundant data exceeding the set time limit.
In a second aspect, the present invention provides an apparatus for predicting storage device IOPS performance data, including a data extraction module, a data fitting processing module, and a display module;
the data extraction module is used for extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
the data fitting processing module is used for fitting the extracted data;
and the display module is used for displaying the prediction data obtained after the fitting processing.
Preferably, the device also comprises a data storage module and a data classification processing module
The data storage module is used for acquiring the IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
the data classification processing module is used for respectively storing the data into corresponding data tables according to different data types and comprises a setting unit and a processing unit;
the setting unit is used for setting the storage time limit of each piece of data;
and the processing unit is used for automatically clearing the redundant data exceeding the set time limit.
In a third aspect, the present invention further provides an electronic device, including a memory and a processor, where the processor and the memory complete communication with each other through a bus; the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform a method of predicting storage device IOPS performance data as described in the first aspect.
In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a method for predicting storage device IOPS performance data according to the first aspect.
According to the technical scheme, the invention has the following advantages: the method for predicting the performance data of each IOPS of the storage equipment by the polynomial curve fitting can predict the future IOPS numerical value more accurately, further provide help for reasonably managing and using hardware equipment by most of the storage equipment through the predicted IOPS, is intuitive and strong in practicability, and can intuitively see the stored historical data and predicted data curve of each IOPS of the storage equipment in a chart after the processing. The operation is convenient, complex processing and human intervention are not needed, the user can directly check the operation, and calculation errors and extra workload caused by redundant operation of the user are avoided.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for a person skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method according to a first embodiment of the present invention.
Fig. 2 is a schematic block diagram of an apparatus according to a second embodiment of the present invention.
Fig. 3 is a schematic diagram of an electronic device connection according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
As shown in fig. 1, an embodiment of the present invention provides a method for predicting storage device IOPS performance data, including the following steps:
s1: acquiring IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
the storage device can generate some real-time performance data under normal operation, wherein various IOPS performance data such as a disk, a data channel, a created LUN and the like can be pushed to a specified performance data collection client through an SSH protocol or an FTP protocol, and the data is used as a sample to be faithfully stored in a database;
s2: respectively storing the data into corresponding data tables according to different data types;
the database is respectively stored in corresponding data tables according to different data types, one piece of information is stored in the disk IOPS table by taking time as a unit, for example, one piece of information is recorded every minute of disk IOPS data, the information comprises data pushed by storage equipment such as storage equipment identification ID, generation time, disk ID, IOPS value size and the like, the data faithfully records the real-time data size, and each piece of data is stored until redundant data exceeding the time limit is automatically cleared after the time limit (for example, 7 days) required by a user.
S3: extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
in the embodiment, a disk 1 is selected, a background extracts all data of the disk 1 within 7 days from a database, and stores corresponding sets of X [ ] and Y [ ];
s4: fitting the extracted data;
obtaining a polynomial fitting function after quantitatively analyzing IOPS performance data of the storage equipment by calling a curve fitting function ployfit; wherein the polynomial fit function: p [ ] ployfit (X [ ], Y [ ], N)
wherein, i represents each predicted time point in the future time period viewed by the user, i is 1, 2, 3 … …;
p [ ]: a polynomial coefficient representing a decreasing power permutation of the fitting function polynomial expression;
x [ ]: representing each time point array corresponding to the IOPS data stored in the database;
y [ ]: actual value arrays representing the corresponding time points of the IOPS stored in the database;
n: representing the order of the fitting polynomial, and selecting a natural number more than 4 by N according to the actual situation;
forecast [ i ]: and indicating the IOPS predicted value corresponding to the predicted future time point i.
S5: and displaying the prediction data obtained after the fitting treatment.
The predicted data is obtained by bringing future time points into the polynomial fitting function and is displayed on a Web page through open source chart display tools such as HighCharts, and users can directly see the predicted data of each IOPS.
And pushing various IOPS performance data such as a disk, a data channel, a created LUN and the like to a specified plug-in through a storage device, storing related data into a database as a collected data sample, and displaying various IOPS actual values and predicted values to a user on a Web page through an open source chart display tool such as Highhards and the like after data processing.
Example two
As shown in fig. 2, the technical solution of the present invention provides an apparatus for predicting storage device IOPS performance data, including a data extraction module, a data fitting processing module, a display module, a data storage module, and a data classification processing module;
the data storage module is used for acquiring the IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
the data classification processing module is used for respectively storing the data into corresponding data tables according to different data types and comprises a setting unit and a processing unit;
the setting unit is used for setting the storage time limit of each piece of data;
the processing unit is used for automatically clearing redundant data exceeding a set time limit;
the data extraction module is used for extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
the data fitting processing module is used for fitting the extracted data; the data fitting processing module acquires a polynomial fitting function after quantitatively analyzing IOPS performance data of the storage device by calling a curve fitting function ployfit; wherein the polynomial fit function: p [ ] ployfit (X [ ], Y [ ], N)
wherein, i represents each predicted time point in the future time period viewed by the user, i is 1, 2, 3 … …;
p [ ]: a polynomial coefficient representing a decreasing power permutation of the fitting function polynomial expression;
x [ ]: representing each time point array corresponding to the IOPS data stored in the database;
y [ ]: actual value arrays representing the corresponding time points of the IOPS stored in the database;
n: representing the order of the fitting polynomial, and selecting a natural number more than 4 by N according to the actual situation;
forecast [ i ]: and indicating the IOPS predicted value corresponding to the predicted future time point i.
And the display module is used for displaying the prediction data obtained after the fitting processing.
And predicting the stored IOPS performance data by adopting a polynomial curve fitting algorithm. The polynomial fitting function is obtained after the performance data of the disk IOPS is quantitatively analyzed by calling the curve fitting function ployfit in the MatLab, and then the predicted data is obtained and displayed by bringing the function into a future time point, so that the specific requirements of a user on the performance statistical data of the storage equipment can be more intuitively met, and help is provided for the user to reasonably manage and use the storage hardware equipment through the predicted IOPS.
EXAMPLE III
Fig. 3 is a schematic entity structure diagram of an electronic device according to an embodiment of the present invention, where the electronic device may include: a processor (processor)510, a communication Interface (Communications Interface)520, a memory (memory)530, and a bus 540, wherein the processor 510, the communication Interface 520, and the memory 530 communicate with each other via the bus 540. Bus 540 may be used for information transfer between the electronic device and the sensor. Processor 510 may call logic instructions in memory 530 to perform the following method: acquiring IOPS performance data of the storage equipment and storing the IOPS performance data into a database; respectively storing the data into corresponding data tables according to different data types; extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array; fitting the extracted data; and displaying the prediction data obtained after the fitting treatment.
Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Example four
Embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions, which cause a computer to perform a method for predicting IOPS performance data of a storage device provided in the foregoing embodiments, for example, the method includes: acquiring IOPS performance data of the storage equipment and storing the IOPS performance data into a database; respectively storing the data into corresponding data tables according to different data types; extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array; fitting the extracted data; and displaying the prediction data obtained after the fitting treatment.
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method of predicting storage device IOPS performance data, comprising the steps of:
extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
fitting the extracted data;
and displaying the prediction data obtained after the fitting treatment.
2. The method of claim 1, wherein the step of extracting all the data of the IOPS performance of the storage device within a set time period and storing the data into the corresponding array comprises:
acquiring IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
and storing the data into corresponding data tables according to different data types.
3. The method of claim 1, wherein the step of fitting the extracted data comprises:
and obtaining a polynomial fitting function after quantitatively analyzing the IOPS performance data of the storage equipment by calling a curve fitting function ployfit.
4. The method of claim 1, wherein the step of displaying the predicted data obtained after the fitting process comprises:
and carrying in future time points through a polynomial fitting function to obtain prediction data, and displaying each IOPS actual value and each predicted value on a Web page through an open source chart display tool.
5. The method according to claim 3, wherein in the step of obtaining the polynomial fitting function by calling a curve fitting function ployfit and quantitatively analyzing the IOPS performance data of the storage device, the polynomial fitting function is: p [ ] ployfit (X [ ], Y [ ], N)
wherein, i represents each predicted time point in the future time period viewed by the user, i is 1, 2, 3 … …;
p [ ]: a polynomial coefficient representing a decreasing power permutation of the fitting function polynomial expression;
x [ ]: representing each time point array corresponding to the IOPS data stored in the database;
y [ ]: actual value arrays representing the corresponding time points of the IOPS stored in the database;
n: representing the order of the fitting polynomial, and selecting a natural number more than 4 by N according to the actual situation;
forecast [ i ]: and indicating the IOPS predicted value corresponding to the predicted future time point i.
6. The method of claim 2, wherein the step of storing the IOPS performance data of the storage device into corresponding data tables according to different data types further comprises:
and setting the storage time limit of each piece of data and automatically clearing redundant data exceeding the set time limit.
7. The device for predicting the IOPS performance data of the storage equipment is characterized by comprising a data extraction module, a data fitting processing module and a display module;
the data extraction module is used for extracting all data of the IOPS performance of the storage equipment in a set time period and storing the data into a corresponding array;
the data fitting processing module is used for fitting the extracted data;
and the display module is used for displaying the prediction data obtained after the fitting processing.
8. The apparatus of claim 7, further comprising a data storage module and a data classification processing module
The data storage module is used for acquiring the IOPS performance data of the storage equipment and storing the IOPS performance data into a database;
the data classification processing module is used for respectively storing the data into corresponding data tables according to different data types and comprises a setting unit and a processing unit;
the setting unit is used for setting the storage time limit of each piece of data;
and the processing unit is used for automatically clearing the redundant data exceeding the set time limit.
9. An electronic device, comprising a memory and a processor, wherein the processor and the memory communicate with each other via a bus; the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform a method of predicting storage device IOPS performance data as claimed in any one of claims 1 to 6.
10. A non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of predicting storage device IOPS performance data as claimed in any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910819729.4A CN110851317A (en) | 2019-08-31 | 2019-08-31 | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910819729.4A CN110851317A (en) | 2019-08-31 | 2019-08-31 | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN110851317A true CN110851317A (en) | 2020-02-28 |
Family
ID=69595342
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910819729.4A Withdrawn CN110851317A (en) | 2019-08-31 | 2019-08-31 | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110851317A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111550961A (en) * | 2020-04-26 | 2020-08-18 | 青岛海尔电冰箱有限公司 | Method for predicting temperature of refrigerator compartment and intelligent refrigerator |
CN111737098A (en) * | 2020-06-06 | 2020-10-02 | 苏州浪潮智能科技有限公司 | Method and system for evaluating IOPS (input/output protection system) of storage system based on number of hard disks |
WO2021238264A1 (en) * | 2020-05-29 | 2021-12-02 | 苏州浪潮智能科技有限公司 | Random write method and apparatus |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106250306A (en) * | 2016-08-18 | 2016-12-21 | 电子科技大学 | A kind of performance prediction method being applicable to enterprise-level O&M automatization platform |
-
2019
- 2019-08-31 CN CN201910819729.4A patent/CN110851317A/en not_active Withdrawn
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106250306A (en) * | 2016-08-18 | 2016-12-21 | 电子科技大学 | A kind of performance prediction method being applicable to enterprise-level O&M automatization platform |
Non-Patent Citations (1)
Title |
---|
ZLSBM: "polyfit多项式拟合函数的用法", 《HTTPS://BLOG.CSDN.NET/ZLSBM/ARTICLE/DETAILS/73770048》 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111550961A (en) * | 2020-04-26 | 2020-08-18 | 青岛海尔电冰箱有限公司 | Method for predicting temperature of refrigerator compartment and intelligent refrigerator |
WO2021238264A1 (en) * | 2020-05-29 | 2021-12-02 | 苏州浪潮智能科技有限公司 | Random write method and apparatus |
CN111737098A (en) * | 2020-06-06 | 2020-10-02 | 苏州浪潮智能科技有限公司 | Method and system for evaluating IOPS (input/output protection system) of storage system based on number of hard disks |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11416768B2 (en) | Feature processing method and feature processing system for machine learning | |
CN110851317A (en) | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment | |
WO2019153487A1 (en) | System performance measurement method and device, storage medium and server | |
CN111143286B (en) | Cloud platform log management method and system | |
CN108900619B (en) | Independent visitor counting method and device | |
WO2020000726A1 (en) | Performance test report generating method, electronic device, and readable storage medium | |
CN107122669A (en) | A kind of method and apparatus for assessing leaking data risk | |
WO2014206131A1 (en) | Method and apparatus for report generation | |
CN109461023B (en) | Loss user retrieval method and device, electronic equipment and storage medium | |
CN106020722A (en) | Method, device and system for deduplication of repeated data of cloud storage system | |
CN105242873B (en) | The acquisition of the performance data of cloud computing system and storage method and device | |
Hildebrandt et al. | Server sounds and network noises | |
CN113987086A (en) | Data processing method, data processing device, electronic device, and storage medium | |
US9141686B2 (en) | Risk analysis using unstructured data | |
WO2022237506A1 (en) | Method, apparatus, and device for monitoring online diagnosis service, and storage medium | |
CN109408353B (en) | Storage partition utilization rate and IOPS analysis method, system and terminal equipment | |
CN114896230A (en) | Application data real-time analysis distributed big data platform | |
CN109189656A (en) | A method of storage IO PS performance data is analyzed based on standard difference algorithm | |
CN110674018A (en) | Method and device for predicting performance data of storage device and storage medium | |
CN104954280B (en) | A kind of data message processing method and device | |
CN114398343A (en) | Database abnormal key processing method, device, equipment and medium | |
CN114510708A (en) | Real-time data warehouse construction and anomaly detection method, device, equipment and product | |
US20200351284A1 (en) | Generating summaries of messages associated with assets in an enterprise system | |
CN112242201A (en) | Virus propagation risk early warning method, device and equipment based on graph calculation | |
CN111143328A (en) | Agile business intelligent data construction method, system, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200228 |
|
WW01 | Invention patent application withdrawn after publication |