CN105843724B - A kind of IT system monitoring state index compression analysis method - Google Patents
A kind of IT system monitoring state index compression analysis method Download PDFInfo
- Publication number
- CN105843724B CN105843724B CN201610153066.3A CN201610153066A CN105843724B CN 105843724 B CN105843724 B CN 105843724B CN 201610153066 A CN201610153066 A CN 201610153066A CN 105843724 B CN105843724 B CN 105843724B
- Authority
- CN
- China
- Prior art keywords
- time
- state
- index
- data
- sampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention relates to a kind of IT system monitoring state indexs to compress analysis method, feature few using the Status Change of equipment, repetition probability is big, design is suitble to temporally point storage state and analyzes the data record format of state, and matching compression and parser, its step are as follows: accepting device state index data；Inquire the nearest index record of the equipment；Compare the old and new's record, according to result more new historical index storehouse；Pass through history index storehouse query facility state change situation.The invention has the benefit that the present invention can be solved effectively in IT system monitoring field, the big data process performance problem that state index monitoring state is collected and processing encounters, it solves under extensive monitoring, the storage problem of mass data, the Real time Efficiency that data are read on upper layer is improved simultaneously, shortens the period of data analysis.
Description
Technical field
The present invention relates to IT systems to monitor field, and mainly a kind of IT system monitoring state index compresses analysis method, leads to
State index compression analytical technology is crossed, solves the storage problem of mass data, while improving upper layer and reading the real-time of data
Efficiency shortens the period of data analysis.
Background technique
Modern IT systems underlying basis framework is complicated, needs large number of equipment, while in order to guarantee IT system 7*24 hours
Normal operation generally requires and carries out real-time monitoring to the underlying basis framework of IT system, monitors the state of all kinds of indexs, once hair
Existing problem, can notify personnel to solve the problems, such as and repair in time, to guarantee the high availability of IT system.General device monitoring index
It is all to be stored according to time series, the time is long, and the disk space of occupancy is increasing, and the reading of data is also increasingly
Slowly.Therefore it produces state index and compresses such product technology, to solve these problems.
Summary of the invention
Present invention aim to address traditional IT monitoring systems the problem of state index is collected and processing encounters, and provides
A kind of IT system monitoring state index compression analysis method, by use state index compress technique later, it can be achieved that storage shape
State achievement data amount is greatly decreased, and solves under extensive monitoring, the storage problem of mass data, while improving upper layer reading
The Real time Efficiency for evidence of fetching shortens the period of data analysis.
The object of the present invention is achieved by the following technical solutions.This IT system monitoring state index compression analysis
Method, the feature few using the Status Change of equipment, repetition probability is big, design are suitble to temporally point storage state and analyze state
Data record format and matching compression and parser, its step are as follows:
1) accepting device state index data；
2) the nearest index record of the equipment is inquired；
3) compare the old and new's record, according to result more new historical index storehouse；
4) pass through history index storehouse query facility state change situation.
Further, the specific steps are as follows:
(1), achievement data is received, NR, NR=device identification+pointer type+index value+sampling time are labeled as；
(2), corresponding last time achievement data is inquired, is labeled as LR, LR=device identification+pointer type+index value+
Less than record, then LR is arranged as NULL mark in time started+end time+sampling number, such as inquiry；
(3), compare the index state of NR and LR, and carry out corresponding data and save operation；
(3-1), such as NR index value=LR index value, then update LR and save again, and it is as follows to update rule:
A) LR end time, LR end time=NR sampling time are updated；
B) increase LR sampling number, LR sampling number=LR sampling number+1；
(3-2), such as NR index value<>LR index value LR=NULL, then modify the LR end time, and is generated newly by NR
LR is simultaneously saved, and it is as follows to update rule:
A) LR end time, LR end time=NR sampling time are set；
B) LR is saved；
C) new LR is generated by NR；
D) LR time started, LR time started=NR sampling time are set；
E) LR end time, LR end time=NR sampling time are set；
F) setting LR sampling number is sampling number=1 1, LR；
(3-3), such as LR=NULL then generate new LR and preservation by NR, and it is as follows to update rule:
A) LR time started, LR time started=NR sampling time are set；
B) LR end time, LR end time=NR end time are set；
C) setting LR sampling number is sampling number=1 1, LR.
Further, analysis indexes Status Change is recorded by the LR in inquiry designated time period, can be quickly real
It is existing:
A) Status Change history is inquired, is sorted by the LR time started；
B) total time of certain state is analyzed, the time interval for specifying index value matching record in LR is added up to.
The invention has the benefit that the present invention can be solved effectively in IT system monitoring field, state index monitors shape
The big data process performance problem that state is collected and processing encounters solves under extensive monitoring, the storage problem of mass data,
The Real time Efficiency that data are read on upper layer is improved simultaneously, shortens the period of data analysis.
Detailed description of the invention
Fig. 1 is configuration item reconciliation flow diagram of the invention；
Fig. 2 is the primary data sample schematic diagram before the present invention；
Fig. 3 is data sample schematic diagram after applying the present invention；
Fig. 4 is using the data volume contrast schematic diagram after the present invention.
Specific embodiment
Below in conjunction with drawings and examples, the present invention will be described in detail:
State index compress technique of the present invention is related to 3 units: data receipt unit, metric history library, number altogether
According to preservation processing unit, data analysis unit.1. data receipt unit, accepting device state index data mark new record；2.
Metric history library, saves historical data and provides query interface；3. data save processing unit, compare the old and new's record, how is decision
It saves historical data；4. data analysis unit by query history library and adds up to, to provide Status Change analytical statement.
The operational flow diagram of the above-mentioned each module in the position Fig. 1；Data flow is described as indicated above；What each monitoring system provided
Achievement data first passes around label and generates new record；Secondly locating query history library record；Finally carry out record comparison；
Fig. 2 is the data sample of a network equipment port working condition before using the present invention；
Fig. 3 is the data sample after above-mentioned data compression after using the present invention；
Fig. 4 is using present invention front and back, and the income of data compression compares.
The step of present invention realizes is as follows:
1, achievement data is received, NR is labeled as.NR=device identification+pointer type+index value+sampling time
Such as: NR=switch A+port working state+offline+2015-03-10 14:42:51
2, corresponding last time achievement data is inquired, LR is labeled as.LR=device identification+pointer type+index value+is opened
Begin time+end time+sampling number.If inquired less than record, then LR is set for NULL mark.
Such as: LR=switch A+port working state+online+2015-03-05 18:22:30+2015-03-10 14:
32:51+697 times
3, compare the index state of NR and LR, and carry out corresponding data and save operation
If 1) " NR index value=LR index value ", updates LR and save again, it is as follows to update rule:
A) the LR end time is updated, LR end time=NR sampling time is made
B) increase LR sampling number, make LR sampling number=LR sampling number+1
2) " if NR index value<>LR index value or LR=NULL ", is modified the LR end time, and is generated newly by NR
LR and preservation, update rule it is as follows:
A) the LR end time is set, LR end time=NR sampling time is made
B) LR is saved
C) new LR is generated by NR
D) the LR time started is set, LR time started=NR sampling time is made
E) the LR end time is set, LR end time=NR sampling time is made
F) setting LR sampling number is sampling number=1 1, LR
If 3) " LR=NULL ", new LR and preservation are generated by NR, it is as follows to update rule:
A) the LR time started is set, LR time started=NR sampling time is made
B) the LR end time is set, LR end time=NR end time is made
C) setting LR sampling number is 1, makes sampling number=1 LR
According to the example in step 1 and 2, there is " NR index value<>LR index value " in this step, therefore generate following operation:
1) the LR end time is set, and saves former LR=switch A+port working state+online+2015-03-05 18:
22:30+2015-03-10 14:42:51+697 times
2) new LR is generated by NR, and saves new LR=switch A+port working state+offline+2015-03-10
14:42:51+2015-03-10 14:42:51+1 times
4, analysis indexes Status Change is recorded by the LR in inquiry designated time period, can be fast implemented:
A) Status Change history is inquired, is sorted by the LR time started
B) total time of certain state is analyzed, the time interval for specifying index value matching record in LR is added up to.
The present invention is not limited to the above-described embodiments, makees any variation no matter constituting in its shape or material, all to adopt
It is designed with structure provided by the present invention, is all a kind of deformation of the invention, is considered as within that scope of the present invention.
Claims (2)
1. a kind of IT system monitoring state index compresses analysis method, it is characterized in that: the Status Change using equipment is few, repeats machine
The big feature of rate, design are suitble to temporally data record format and matching pressure of the point storage state with the state of analysis
Contracting and parser, its step are as follows:
1) accepting device state index data；
2) the nearest index record of the equipment is inquired；
3) compare the old and new's record, according to result more new historical index storehouse；
4) pass through history index storehouse query facility state change situation；
Specific step is as follows:
(1), achievement data is received, NR, NR=device identification+pointer type+index value+sampling time are labeled as；
(2), corresponding last time achievement data is inquired, LR, LR=device identification+pointer type+index value+beginning are labeled as
Less than record, then LR is arranged as NULL mark in time+end time+sampling number, such as inquiry；
(3), compare the index state of NR and LR, and carry out corresponding data and save operation；
(3-1), such as NR index value=LR index value, then update LR and save again, and it is as follows to update rule:
A) LR end time, LR end time=NR sampling time are updated；
B) increase LR sampling number, LR sampling number=LR sampling number+1；
(3-2), such as NR index value<>LR index value LR=NULL, then modify the LR end time, and generate new LR simultaneously by NR
It saves, it is as follows to update rule:
A) LR end time, LR end time=NR sampling time are set；
B) LR is saved；
C) new LR is generated by NR；
D) LR time started, LR time started=NR sampling time are set；
E) LR end time, LR end time=NR sampling time are set；
F) setting LR sampling number is sampling number=1 1, LR；
(3-3), such as LR=NULL then generate new LR and preservation by NR, and it is as follows to update rule:
A) LR time started, LR time started=NR sampling time are set；
B) LR end time, LR end time=NR end time are set；
C) setting LR sampling number is sampling number=1 1, LR.
2. IT system monitoring state index according to claim 1 compresses analysis method, it is characterised in that: analysis indexes shape
State change is recorded by the LR in inquiry designated time period, can be fast implemented:
A) Status Change history is inquired, is sorted by the LR time started；
B) total time of certain state is analyzed, the time interval for specifying index value matching record in LR is added up to.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610153066.3A CN105843724B (en) | 2016-03-17 | 2016-03-17 | A kind of IT system monitoring state index compression analysis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610153066.3A CN105843724B (en) | 2016-03-17 | 2016-03-17 | A kind of IT system monitoring state index compression analysis method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105843724A CN105843724A (en) | 2016-08-10 |
| CN105843724B true CN105843724B (en) | 2019-03-15 |
Family
ID=56588135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610153066.3A Active CN105843724B (en) | 2016-03-17 | 2016-03-17 | A kind of IT system monitoring state index compression analysis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105843724B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107276849A (en) * | 2017-06-15 | 2017-10-20 | 北京奇艺世纪科技有限公司 | The method for analyzing performance and device of a kind of cluster |
| CN113688109B (en) * | 2021-08-20 | 2024-05-03 | 招商银行股份有限公司 | IT system monitoring index compression and query method, device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1553375A (en) * | 2003-05-29 | 2004-12-08 | 华为技术有限公司 | Collecting and processing method for apparatus performance accounting data |
| CN102231081A (en) * | 2011-06-14 | 2011-11-02 | 北京三博中自科技有限公司 | Energy utilization state diagnosis method for process industrial equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004049088A1 (en) * | 2002-11-22 | 2004-06-10 | Radar Hvac-Refrigeration Inc. | Refrigeration monitor |
-
2016
- 2016-03-17 CN CN201610153066.3A patent/CN105843724B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1553375A (en) * | 2003-05-29 | 2004-12-08 | 华为技术有限公司 | Collecting and processing method for apparatus performance accounting data |
| CN102231081A (en) * | 2011-06-14 | 2011-11-02 | 北京三博中自科技有限公司 | Energy utilization state diagnosis method for process industrial equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105843724A (en) | 2016-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107577588B (en) | Intelligent operation and maintenance system for mass log data | |
| CN105653427B (en) | The log monitoring method of Behavior-based control abnormality detection | |
| CN111158977A (en) | Abnormal event root cause positioning method and device | |
| EP3432520B1 (en) | Efficient storage and querying of time series metrics | |
| CN113360722B (en) | Fault root cause positioning method and system based on multidimensional data map | |
| CN106250287A (en) | A kind of log information processing means | |
| CN111177276A (en) | Spark calculation framework-based kinetic energy data processing system and method | |
| CN111860667A (en) | Method and device for determining equipment fault, storage medium and electronic device | |
| CN105843724B (en) | A kind of IT system monitoring state index compression analysis method | |
| CN114281864A (en) | Correlation analysis method for power network alarm information | |
| WO2021011817A1 (en) | Real-time analysis of in-field collected well fracturing data | |
| He et al. | Intelligent Fault Analysis With AIOps Technology | |
| CN115130847A (en) | Equipment portrait modeling method and system | |
| CN114742124A (en) | Abnormal data processing method, system and device | |
| CN105677723B (en) | A kind of data label foundation and search method for industrial signal source | |
| US11727021B2 (en) | Process control tool for processing big and wide data | |
| CN111061581A (en) | Fault detection method, device and equipment | |
| CN110909380B (en) | Abnormal file access behavior monitoring method and device | |
| CN116149896B (en) | Time sequence data abnormality detection method, storage medium and electronic device | |
| US20190340540A1 (en) | Adaptive continuous log model learning | |
| CN111367971A (en) | Financial system abnormity auxiliary analysis method and device based on data mining | |
| CN111858560A (en) | Financial data automated testing and monitoring system based on data warehouse | |
| CN106227797A (en) | A kind of processing method of massive logs information | |
| US9372603B2 (en) | Data sampling method for determining salient values in a large data stream | |
| CN112783740B (en) | Server performance prediction method and system based on time series characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191217 Address after: 311305 no.030, building 3, No.6 heting street, Qingshanhu street, Lin'an District, Hangzhou City, Zhejiang Province Co-patentee after: Beijing Guangtong software Limited by Share Ltd XinDa Patentee after: Hangzhou Youyun Software Co., Ltd. Address before: 311305, No. three, No. 6, Ting Ting Street, 030 Qingshan Lake Street, Hangzhou, Zhejiang, Ling'an Patentee before: Hangzhou Youyun Software Co., Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 311305 No. 030, building 3, No. 6, heting street, Qingshanhu street, Lin'an District, Hangzhou City, Zhejiang Province Patentee after: HANGZHOU YOUYUN SOFTWARE Co.,Ltd. Patentee after: Beijing Guangtong Youyun Technology Co.,Ltd. Address before: 311305 No. 030, building 3, No. 6, heting street, Qingshanhu street, Lin'an District, Hangzhou City, Zhejiang Province Patentee before: HANGZHOU YOUYUN SOFTWARE Co.,Ltd. Patentee before: Beijing Guangtong software Limited by Share Ltd. XinDa |