CN109189655A - A method of storage device history performance data is counted based on mean value optimization algorithm - Google Patents
A method of storage device history performance data is counted based on mean value optimization algorithm Download PDFInfo
- Publication number
- CN109189655A CN109189655A CN201810896763.7A CN201810896763A CN109189655A CN 109189655 A CN109189655 A CN 109189655A CN 201810896763 A CN201810896763 A CN 201810896763A CN 109189655 A CN109189655 A CN 109189655A
- Authority
- CN
- China
- Prior art keywords
- data
- mean value
- value optimization
- client
- storage device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of method for counting storage device history performance data based on mean value optimization algorithm, includes the following steps: that storing equipment is pushed to client for the real time data of generation；Client will be stored in database after data mean value optimization processing；User selects specified time interval, designated time period to inquire all data；It is shown by figure plug-in unit in web interface.Storage equipment crosses SSH agreement or File Transfer Protocol and the real time data of generation is pushed to client, and it is primary to store equipment push per minute.
Description
Technical field
The present invention relates to computer memory system technical fields, and in particular to one kind is based on the statistics storage of mean value optimization algorithm
The method of device history performance data.
Background technique
With the fast development of scientific algorithm and various network applications, the information content that the mankind generate is more and more, what this made
The storage of data is more and more of interest by people so that storage unit it is locating in entire computer architecture status it is more next
More important, storage turns to disk array, and then develops to and store network currently popular via single disk, tape, such as
NAS (Network Storage Technologies, Network storage technology), SAN (Storage Area Network, storage
Local Area Network) and iscsi (Internet Small Computer System Interface, internet small computer
System interface) etc..Large-scale data application demand continues to bring out, and mass data and its application also become a new development side
To, data storage produces tremendous influence to the work and life of people, and wherein for the items of storage equipment
Energy data analysis is also naturally more and more important.
The History Performance Data of storage equipment is the historical record for storing equipment properties data and preserving, by dividing
Analyse History Performance Data, can with effective monitoring store equipment operation the case where, analysis storage equipment superiority and inferiority, carry out storage set
The prediction of standby future operation conditions, therefore a kind of system of reasonable statistics storage device history performance data and processing are historic
The method of energy data is particularly important.
The History Performance Data of statistics usually contain the hundreds of volumes of system, disk array, the port FC data per minute, number
Need to store general 1 year information above according to library, directly storage pressure is huge, checks that related data will also result in system response speed
Degree reduces.
Summary of the invention
It is an object of the present invention to most of storage equipment after statistical history performance data, in face of needing to check overall trend
When the mass data that occurs cause database purchase occupied space big and respond slow problem when checking, provide a kind of based on mean value
The method of optimization algorithm statistics storage device history performance data, to solve the above technical problems.
In order to achieve the above object, the technical scheme is that
A method of storage device history performance data is counted based on mean value optimization algorithm, is included the following steps:
It stores equipment and the real time data of generation is pushed to client；
Client will be stored in database after data mean value optimization processing；
User selects specified time interval, designated time period to inquire all data, extracts from the background from database during corresponding to
Interior all data, and web front end figure plug-in unit corresponding data interface is transferred to for directly showing.
Preferably, the real time data of generation is pushed in client by step storage equipment, stores equipment push per minute
Once.
Preferably, the real time data of generation is pushed in client by step storage equipment,
Storage equipment crosses SSH agreement or File Transfer Protocol and the real time data of generation is pushed to client.
Preferably, the real time data of generation is pushed in client by step storage equipment, and storage equipment generates real-time
Data include IOPS, MBPS, delay；
IOPS (I/O per second), maximum I/O number per second, for measuring the ring of the concurrent random read-write of large amount of small documents
Border stores the maximum file cocurrent number that can be provided；
Maximum bandwidth MBPS (MB per second) per second is how many million, for measuring a large amount of big document order read-writes
When, store the maximum bandwidth that can be provided；
Delay, store equipment delay refer to since receive data packet to destination port transmission data packet
Time interval.
Preferably, step client will be stored in database after data mean value optimization processing, comprising:
Tables of data is created in database, the tables of data includes IOPS table, MBPS table and time-delay table；
Client will by treated, data information be stored in corresponding data as unit of the time respectively according to data type
Table.
Preferably, step client will be stored in database after data mean value optimization processing, using mean value optimum theory, such as
Following formula (1):
Wherein, X [i]: each by storage equipment push Lai performance data strictly according to the facts, which includes IOPS, MBPS, prolongs
When three classes data；
Y [j]: the performance data of database is stored in after mean value optimization processing, i.e., by the property in diagrammatic representation to user
It can value data；
N: the time interval number that selection is defined according to user is indicated.
Preferably, time interval number includes 5 minutes, 15 minutes, 60 minutes；
N=5/15/60.
Preferably, step client will by treated, data information be stored in as unit of the time respectively according to data type
In corresponding tables of data, data information includes:
Store equipment identification id, generation time, the type of mean time interval N, the volume/disk array/FC for generating data
Port identification id information.
Preferably, step client will be stored in database after data mean value optimization processing, further includes:
The time limit that setting saves every data arrival user setting in the database removes the redundant digit beyond the time limit automatically
According to.
Preferably, user setting when be limited to 365 days.
Preferably, figure plug-in unit includes HighCharts graphical display plug-in unit.
As can be seen from the above technical solutions, the invention has the following advantages that kind mean value optimum theory carries out data processing,
Database purchase History Performance Data pressure can be effectively reduced, while can also intuitively meet what user checked statistical data
Particular demands, it is intuitive, practical, it operates conveniently, it, can by IOPS, MBPS, the delay numerical value after above-mentioned mean value optimization processing
Check that data are rung in the case where unobvious reduction precision, to reduce database purchase History Performance Data pressure and improve user
Speed is answered, can intuitively see that storage device performance increases reduced trend and fluctuating situation in the graph, when can be optional
Between section inquired, do not need human intervention data processing, can facilitate user to used storage device performance run feelings
Condition is monitored, and Products function is effectively supplemented.
In addition, design principle of the present invention is reliable, structure is simple, has very extensive application prospect.
It can be seen that compared with prior art, the present invention have substantive distinguishing features outstanding and it is significant ground it is progressive, implementation
Beneficial effect be also obvious.
Detailed description of the invention
Fig. 1 is a kind of method flow diagram that storage device history performance data is counted based on mean value optimization algorithm；
Fig. 2 is that mean value optimization processing stores device history performance statistic diagrammatic representation example.
Specific embodiment
The present invention will be described in detail with reference to the accompanying drawing and by specific embodiment, and following embodiment is to the present invention
Explanation, and the invention is not limited to following implementation.
Embodiment one
As shown in Figure 1, a kind of method that storage device history performance data is counted based on mean value optimization algorithm, including it is as follows
Step:
S1: the real time data of generation is pushed to client by storage equipment；
Wherein, storage equipment crosses SSH agreement and the real time data of generation is pushed to client, stores equipment push per minute
Once；
Storing the real time data that equipment generates includes IOPS, MBPS, delay；
IOPS (I/O per second), maximum I/O number per second, for measuring the ring of the concurrent random read-write of large amount of small documents
Border stores the maximum file cocurrent number that can be provided；
Maximum bandwidth MBPS (MB per second) per second is how many million, for measuring a large amount of big document order read-writes
When, store the maximum bandwidth that can be provided；
Delay, store equipment delay refer to since receive data packet to destination port transmission data packet
Time interval.
S2: client will be stored in database after data mean value optimization processing；
Tables of data is created in database, the tables of data includes IOPS table, MBPS table and time-delay table；
Client will by treated, data information be stored in corresponding data as unit of the time respectively according to data type
Table；Data information includes:
Store equipment identification id, generation time, the type of mean time interval N, the volume/disk array/FC for generating data
Port identification id information.
Using mean value optimum theory, such as following formula (1):
Wherein, X [i]: each by storage equipment push Lai performance data strictly according to the facts, which includes IOPS, MBPS, prolongs
When three classes data；
Y [j]: the performance data of database is stored in after mean value optimization processing, i.e., by the property in diagrammatic representation to user
It can value data；
N: the time interval number that selection is defined according to user is indicated.
S3: user selects specified time interval, designated time period to inquire all data, extracts the corresponding phase from database from the background
Interior all data, and web front end figure plug-in unit corresponding data interface is transferred to for directly showing.
Embodiment two
Storage equipment can all generate some real-time performance datas under operating normally, and real time data generates once per minute,
Including IOPS, MBPS, delay etc., these data can be pushed to specified collection historical performance by SSH agreement or File Transfer Protocol
Data client, and being stored in database after data mean value optimization processing；
Using mean value optimum theory, such as following formula (1):
Wherein, X [i]: each by storage equipment push Lai performance data strictly according to the facts, which includes IOPS, MBPS, prolongs
When three classes data；
Y [j]: the performance data of database is stored in after mean value optimization processing, i.e., by the property in diagrammatic representation to user
It can value data；
N: the time interval number that selection is defined according to user is indicated.It is set 5/15/60 minute totally three for user respectively
Kind time interval, as standard N=5/15/60, because real-time performance data is each corresponding I/O PS, MBPS per minute, prolongs
When data, corresponding mean value be optimized for database deposit optimization after 5,/15,/60 3 kinds of data.
Database will be stored in respectively in corresponding tables of data according to data type, as unit of the time, as IOPS data are equal
5/15/60 minute data after being worth optimization processing is stored in IOPS table according to the time of processing, this information includes storage equipment
Identification id, generation time, mean time interval N (N=5/15/60) type, the volume/disk array/port the FC knowledge for generating IOPS
The information such as other ID, and save after the time limit (such as 365 days) that every data is required to user it is automatic just now remove it is superfluous beyond the time limit
Remainder evidence.
Historical data specified time and the time interval of inquiry are wanted in user's input, such as 2018-07-01 00:00 ---
2018-07-17 10:36, sampling interval are N=60 minutes, are extracted from the background from database all during corresponding in each table
Data, including IOPS, MBPS, delay three classes data sample, and it is transferred to web front-end HighCharts plug-in unit corresponding data interface
For directly showing, as Fig. 2 illustrates the result example after memory device set group IP is inquired by 100.7.40.234.
By the existing free open source graphical display plug-in unit such as HighCharts, broken line displaying is set as in plug-in unit
The intuitively History Performance Data chart after the corresponding mean value optimization of page presentation.
According to above-mentioned mean value optimum theory, the storage device performances data such as IOPS, MBPS, delay of acquisition are both various numbers
According to sample, user selects to specify 5,/15,/60 3 kinds of time intervals, can check History Performance Data all in the selected period
Trend situation, such as time interval are 60 minutes, are stored in X [i] comprising 60 initial data, obtain Y after optimizing using mean value
[j] had both been comparable to database data amount reducing 60 times, in the feelings for checking History Performance Data several years several months trend easily
Under condition, mean value optimization method can't significantly reduce History Performance Data precision, can also effectively reduce database pressure
Response speed when being checked with increase system.
Description and claims of this specification and term " first ", " second ", " third " " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to
Here the sequence other than those of diagram or description is implemented.In addition, term " includes " and " having " and their any deformation,
It is intended to cover and non-exclusive includes.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (10)
1. a kind of method for counting storage device history performance data based on mean value optimization algorithm, which is characterized in that including as follows
Step:
It stores equipment and the real time data of generation is pushed to client；
Client will be stored in database after data mean value optimization processing；
User selects specified time interval, designated time period to inquire all data, extracts from the background from database during corresponding to
All data, and web front end figure plug-in unit corresponding data interface is transferred to for directly showing.
2. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 1,
It is characterized in that, the real time data of generation is pushed in client by step storage equipment, it is primary that equipment push is stored per minute.
3. a kind of side for counting storage device history performance data based on mean value optimization algorithm according to claim 1 or 2
Method, which is characterized in that the real time data of generation is pushed in client by step storage equipment,
Storage equipment crosses SSH agreement or File Transfer Protocol and the real time data of generation is pushed to client.
4. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 3,
It is characterized in that, the real time data of generation is pushed in client by step storage equipment, the real time data that storage equipment generates
Including IOPS, MBPS, delay；
IOPS (I/O per second), maximum I/O number per second, for measuring the environment of the concurrent random read-write of large amount of small documents,
Store the maximum file cocurrent number that can be provided；
Maximum bandwidth MBPS (MB per second) per second is how many million, when for measuring a large amount of big document order read-writes, is deposited
Store up the maximum bandwidth that can be provided；
Delay, store equipment delay refer to since receive data packet to destination port transmission data packet time
Interval.
5. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 4,
It is characterized in that, step client will be stored in database after data mean value optimization processing, comprising:
Tables of data is created in database, the tables of data includes IOPS table, MBPS table and time-delay table；
Client will by treated, data information be stored in corresponding tables of data as unit of the time respectively according to data type.
6. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 3,
It is characterized in that, step client will be stored in database after data mean value optimization processing, using mean value optimum theory, such as following formula
(1):
Wherein, X [i]: each by storage equipment push come performance data strictly according to the facts, the data include IOPS, MBPS, be delayed three
Class data；
Y [j]: the performance data of database is stored in after mean value optimization processing, i.e., by the performance number in diagrammatic representation to user
According to numerical value；
N: the time interval number that selection is defined according to user is indicated.
7. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 6,
It is characterized in that, time interval number includes 5 minutes, 15 minutes, 60 minutes；
N=5/15/60.
8. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 5,
It is characterized in that, step client will by treated, data information be stored in accordingly as unit of the time respectively according to data type
Tables of data in, data information includes:
Store equipment identification id, generation time, the type of mean time interval N, the volume/disk array/port FC for generating data
Identify id information.
9. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 3,
It is characterized in that, step client will be stored in database after data mean value optimization processing, further includes:
The time limit that setting saves every data arrival user setting in the database removes the redundant data beyond the time limit automatically.
10. a kind of method that storage device history performance data is counted based on mean value optimization algorithm according to claim 9,
It is characterized in that, user setting when be limited to 365 days.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810896763.7A CN109189655A (en) | 2018-08-08 | 2018-08-08 | A method of storage device history performance data is counted based on mean value optimization algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810896763.7A CN109189655A (en) | 2018-08-08 | 2018-08-08 | A method of storage device history performance data is counted based on mean value optimization algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109189655A true CN109189655A (en) | 2019-01-11 |
Family
ID=64920588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810896763.7A Pending CN109189655A (en) | 2018-08-08 | 2018-08-08 | A method of storage device history performance data is counted based on mean value optimization algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189655A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101819586A (en) * | 2010-03-31 | 2010-09-01 | 中兴通讯股份有限公司 | Intelligent sorting server and method and system for data statistics |
| CN101982957A (en) * | 2010-08-26 | 2011-03-02 | 中国石油集团川庆钻探工程有限公司 | Application method for releasing real-time data of well site and on-site video information |
| CN102541916A (en) * | 2010-12-29 | 2012-07-04 | 沈阳中科博微自动化技术有限公司 | Method for recording and managing historical data of semiconductor equipment |
| CN103678402A (en) * | 2012-09-21 | 2014-03-26 | 厦门雅迅网络股份有限公司 | Real-time data statistics method of mass data |
| CN104933203A (en) * | 2015-07-15 | 2015-09-23 | 中国科学院微电子研究所 | Data processing method and device in wireless sensor network |
| CN107402969A (en) * | 2017-06-28 | 2017-11-28 | 郑州云海信息技术有限公司 | A kind of storage performance statistical method and system |
-
2018
- 2018-08-08 CN CN201810896763.7A patent/CN109189655A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101819586A (en) * | 2010-03-31 | 2010-09-01 | 中兴通讯股份有限公司 | Intelligent sorting server and method and system for data statistics |
| CN101982957A (en) * | 2010-08-26 | 2011-03-02 | 中国石油集团川庆钻探工程有限公司 | Application method for releasing real-time data of well site and on-site video information |
| CN102541916A (en) * | 2010-12-29 | 2012-07-04 | 沈阳中科博微自动化技术有限公司 | Method for recording and managing historical data of semiconductor equipment |
| CN103678402A (en) * | 2012-09-21 | 2014-03-26 | 厦门雅迅网络股份有限公司 | Real-time data statistics method of mass data |
| CN104933203A (en) * | 2015-07-15 | 2015-09-23 | 中国科学院微电子研究所 | Data processing method and device in wireless sensor network |
| CN107402969A (en) * | 2017-06-28 | 2017-11-28 | 郑州云海信息技术有限公司 | A kind of storage performance statistical method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10552287B2 (en) | Performance metrics for diagnosing causes of poor performing virtual machines | |
| WO2020233212A1 (en) | Log record processing method, server, and storage medium | |
| US20120323923A1 (en) | Sorting Data in Limited Memory | |
| CN107515927A (en) | A kind of real estate user behavioural analysis platform | |
| CN103116661B (en) | A kind of data processing method of database | |
| CN106649670A (en) | Streaming computing-based data monitoring method and apparatus | |
| EP2946333B1 (en) | Efficient query processing using histograms in a columnar database | |
| US11210271B1 (en) | Distributed data processing framework | |
| CN106506266B (en) | Network flow analysis method based on GPU, Hadoop/Spark mixing Computational frame | |
| CN106909554B (en) | Method and device for loading database text table data | |
| CN110309110A (en) | A kind of big data log monitoring method and device, storage medium and computer equipment | |
| CN111143286A (en) | Cloud platform log management method and system | |
| CN110502583A (en) | Distributed Data Synchronization method, apparatus, equipment and readable storage medium storing program for executing | |
| CN109379305A (en) | A kind of data distributing method, device, server and storage medium | |
| CN113360554A (en) | Method and equipment for extracting, converting and loading ETL (extract transform load) data | |
| CN102779138A (en) | Hard disk access method of real time data | |
| CN110389967A (en) | Date storage method, device, server and storage medium | |
| CN113032420A (en) | Data query method and device and server | |
| CN111143158A (en) | Monitoring data real-time storage method and system, electronic equipment and storage medium | |
| US7895247B2 (en) | Tracking space usage in a database | |
| CN106407226B (en) | A kind of data processing method, backup server and storage system | |
| CN110851317A (en) | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment | |
| CN112800058A (en) | Method for realizing HBase secondary index | |
| CN108804037A (en) | The method and system of storage device History Performance Data are handled based on box figure | |
| CN109189656A (en) | A method of storage IO PS performance data is analyzed based on standard difference algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190111 |