CN107608862A - Monitoring alarm method, monitoring alarm device and computer-readable recording medium - Google Patents

Monitoring alarm method, monitoring alarm device and computer-readable recording medium Download PDF

Info

Publication number
CN107608862A
CN107608862A CN201710950552.2A CN201710950552A CN107608862A CN 107608862 A CN107608862 A CN 107608862A CN 201710950552 A CN201710950552 A CN 201710950552A CN 107608862 A CN107608862 A CN 107608862A
Authority
CN
China
Prior art keywords
linear models
dynamic linear
time period
time
target component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710950552.2A
Other languages
Chinese (zh)
Other versions
CN107608862B (en
Inventor
姜兴
张春春
曾德强
王明博
冯立
骆方磊
张燕辉
孙谷飞
陆王天宇
孙建举
江雪妍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Zhongan Information Technology Service Co ltd
Original Assignee
Zhongan Information Technology Service Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongan Information Technology Service Co Ltd filed Critical Zhongan Information Technology Service Co Ltd
Priority to CN201710950552.2A priority Critical patent/CN107608862B/en
Publication of CN107608862A publication Critical patent/CN107608862A/en
Application granted granted Critical
Publication of CN107608862B publication Critical patent/CN107608862B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

Present disclosure discloses monitoring alarm method, and the monitoring alarm method includes receiving the data flow for including the historical data associated with target component to be monitored;Data stream is analyzed using dynamic linear models, to obtain for ensuing first time period and associated with target component Prediction Parameters;Target component is counted in first time period, to obtain actual parameter in first time period and associated with the target component;And by the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.Present disclosure also proposed a kind of corresponding computer-readable recording medium and corresponding monitoring alarm device.Present disclosure incorporates the history monitoring data of long term accumulation and is monitored alarm using the forecast model based on machine learning generation.The setting of the reasonable interval of the program is based on historical data, is no longer dependent on simple micro-judgment, considerably reduces manual intervention, by mistake announcement situation, O&M cost.

Description

Monitoring alarm method, monitoring alarm device and computer-readable recording medium
Technical field
Present disclosure belongs to monitoring alarm field, more particularly to a kind of monitoring alarm method, a kind of monitoring alarm device A kind of and corresponding tangible computer-readable recording medium.
Background technology
Traditional monitoring system is mainly all based on rule and is monitored alarm, and many rules as used herein are all According to the experience of operation maintenance personnel come it is being formulated and these rules use process during need O&M mistake afterwards Manual intervention is constantly carried out in journey and changes threshold value, to be applicable real-time monitoring requirement.Meanwhile these rules are universal sets Fixed excessively single, inflexible, so that announcement situation is very universal by mistake, this necessarily causes sharply increasing for O&M cost.
Fig. 1 shows the flow chart for the method for being used for monitoring alarm in the prior art, as can be seen that should from the flow chart Monitoring alarm method 100 only includes two steps, i.e., receives data flow to be monitored and people in method and step 110 before this The decision rule of work input, decision rule herein are manually entered, that is to say, that decision rule herein is entirely basis What the experience of operation maintenance personnel was configured, and adaptability is made to the decision rule by operation maintenance personnel in traffic fluctuations and repaiied Change, but such modification is only adjustment empirical and that the objective real time data of not according to is carried out, therefore such adjustment With very big randomness, and such modification also not necessarily disclosure satisfy that the demand of monitoring alarm;Next walked in method Received data stream is judged according to the decision rule inputted in rapid 120, to judge whether outputting alarm signal.By Think setting in decision rule, therefore such monitoring alarm will necessarily cause announcement situation very common scenario by mistake, this enters And inevitably result in sharply increasing for O&M cost.
The content of the invention
In the prior art, due to needing operation maintenance personnel rule of thumb to carry out given threshold, and tool is compared in the setting of such threshold value There is randomness, not science;And on the other hand the historical data for being rationally provided with close association with these threshold values is not made With causing data to waste.So present disclosure by introduce dynamic linear models come according to the historical data inputted come smart The reasonable interval of relevant parameter is predicted accurately, so as to improve the accuracy of monitoring alarm and reasonability.
The first aspect of present disclosure proposes a kind of monitoring alarm method, and the monitoring alarm method includes:
Receiving includes the data flow of the historical data associated with target component to be monitored;
The data flow is analyzed using dynamic linear models, with obtain for ensuing first time period and The Prediction Parameters associated with target component;
The target component is counted in the first time period, with obtain it is in the first time period and with The associated actual parameter of the target component;And
By the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.
Monitoring alarm method according to present disclosure is divided the data flow by introducing dynamic linear models Analysis, so as to obtain for ensuing first time period and associated with target component Prediction Parameters, using it is pre- The Prediction Parameters of survey and the target component is counted in the first time period in obtained first time period And the actual parameter associated with the target component just can accurately treat monitoring data fluctuation situation carry out science Management and the outputting alarm signal when abnormal.So it is the science and accuracy on the one hand improving prediction, the opposing party Face also greatly reduces announcement situation by mistake, finally can also reduce O&M cost.
In a kind of embodiment of present disclosure, the monitoring alarm method also includes:
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with it is to be monitored Target component associated historical data the dynamic linear models is initialized.
Dynamic linear models can be optimized in initial phase in this way, so that by history number Optimized according to the dynamic linear models of training.
In a kind of embodiment of present disclosure, the dynamic linear models includes first for second time period Dynamic linear models and the second dynamic linear models for the 3rd period, wherein,
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with described second The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain first dynamic linear models;And/or
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the described 3rd The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain second dynamic linear models.
At the beginning of just can carrying out differential dynamic linear models according to the data characteristicses of different periods in this way Beginningization, such as because daytime and evening sample data difference are obvious, the sample data daytime and evening of training pattern separate, prediction The data on daytime use the historical data training pattern on daytime, predict the data at night using the historical data training mould at night Type.
In a kind of embodiment of present disclosure, the monitoring alarm method further comprises:
By the first time period respectively with the second time period and compared with the 3rd period, and
The first time period is divided into the first time period part overlapping with the second time period and with the described 3rd Period overlapping second time period part;And
Judge whether to generate alarm signal for the first time period part and the second time period part respectively.
Even if consequently, it is possible to the period of required prediction across different dynamic linear models, according to present disclosure Monitoring alarm method also can make the accurate prediction most having for different situations, so as to the accuracy for follow-up monitoring alarm Sound assurance is provided.
In a kind of embodiment of present disclosure, the second time period be 9 points of morning at 6 points in afternoon, and 3rd period is at 6 points in afternoon under 9 points one morning.Being handled respectively with such period can be the most smart Reflect the different data volumes of different time sections accurately, so as to further improve the accuracy of monitoring alarm.
In a kind of embodiment of present disclosure, the Prediction Parameters include the Target area for the target component Between, and the Prediction Parameters are further comprised compared with the actual parameter with judging whether to generate alarm signal:
Alarm signal is not generated when the actual parameter is within the forecast interval;And
Alarm signal is generated when the actual parameter is not within the forecast interval.
By forming forecast interval, so as to judge whether actual parameter is reasonable, and decide whether to produce alarm accordingly Signal.
In a kind of embodiment of present disclosure, the historical data include system data, application performance data and/ Or business datum.
In a kind of embodiment of present disclosure, the first time period be current time point it is ensuing five minutes, Ten minutes and/or a hour.
In a kind of embodiment of present disclosure, in the last fortnight of the data flow including current time point with waiting to supervise The associated historical data of the target component of control.
The second aspect of present disclosure proposes a kind of tangible computer-readable recording medium, the storage medium bag Instruction is included, when executed so that the processor of the computer is at least used for:
Receiving includes the data flow of the historical data associated with target component to be monitored;
The data flow is analyzed using dynamic linear models, with obtain for ensuing first time period and The Prediction Parameters associated with target component;
The target component is counted in the first time period, with obtain it is in the first time period and with The associated actual parameter of the target component;And
By the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.
In a kind of embodiment of present disclosure, the storage medium includes instruction, when executed, Also so that the processor of the computer is at least used for:
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with it is to be monitored Target component associated historical data the dynamic linear models is initialized.
In a kind of embodiment of present disclosure, the dynamic linear models includes first for second time period Dynamic linear models and the second dynamic linear models for the 3rd period, the storage medium includes instruction, when the finger When order is performed, also the processor of the computer is at least used for:
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with described second The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain first dynamic linear models;And/or
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the described 3rd The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain second dynamic linear models.
The third aspect of present disclosure proposes a kind of monitoring alarm device, it is characterised in that the monitoring alarm dress Put including:
Receiving module, the receiving module, which is configured as receiving, includes the history number associated with target component to be monitored According to data flow;
Analysis module, the analysis module are configured to, with dynamic linear models and the data flow are analyzed, with Obtain for ensuing first time period and associated with target component Prediction Parameters;
Statistical module, the statistical module are configured as uniting to the target component in the first time period Meter, to obtain actual parameter in the first time period and associated with the target component;And
Judge module, the judge module be configured as by the Prediction Parameters compared with the actual parameter with judge be No generation alarm signal.
In a kind of embodiment of present disclosure, the monitoring alarm device also includes:
Initialization module, the initialization module are configured as utilizing initialization when the dynamic linear models initializes The historical data associated with target component to be monitored in predetermined amount of time before time point is to the dynamic linear models Initialized.
In a kind of embodiment of present disclosure, the dynamic linear models includes first for second time period Dynamic linear models and the second dynamic linear models for the 3rd period, wherein,
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with described second The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain first dynamic linear models;And/or
When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the described 3rd The historical data associated with target component to be monitored in corresponding period period is to the dynamic linear models Initialized to obtain second dynamic linear models.
In a kind of embodiment of present disclosure, the monitoring alarm device further comprises:
Time match module, when the time match module is configured as the first time period respectively with described second Between section and the 3rd period be compared and the first time period be divided into overlapping with the second time period One period part and the second time period part overlapping with the 3rd period;And wherein,
The judge module is configured to be directed to the first time period part and the second time period respectively Part judges whether to generate alarm signal.
Present disclosure incorporates the history monitoring data of long term accumulation and utilizes the prediction mould based on machine learning generation Type is monitored alarm.The forecast model can use dynamic linear models, Kalman filter and pattra leaves based on historical data This theorem predicts the reasonable interval of the index of subsequent time period fluctuation according to such as hour dimension.The reasonable interval of the program Setting is based on historical data, is no longer dependent on simple micro-judgment, considerably reduces manual intervention, by mistake announcement situation, O&M Cost.
Brief description of the drawings
With reference to accompanying drawing and with reference to described further below, feature, advantage and other aspects of the presently disclosed embodiments will become Must be more obvious, show some embodiments of the disclosure by way of example, and not by way of limitation herein, in the accompanying drawings:
Fig. 1 shows the schematic flow sheet of monitoring alarm method 100 of the prior art;
Fig. 2 shows the schematic flow sheet of the monitoring alarm method 200 according to present disclosure;And
Fig. 3 shows the block diagram of the monitoring alarm device 300 according to present disclosure.
Embodiment
Each exemplary embodiment of the disclosure is described in detail below with reference to accompanying drawing.Flow chart and block diagram in accompanying drawing are shown According to architectural framework in the cards, function and the operation of the method and system of the various embodiments of the disclosure.It should be noted that Each square frame in flow chart or block diagram can represent a part for a module, program segment or code, the module, program The a part of of section or code can include one or more being used to realize holding for the logic function of defined in each embodiment Row instruction.It should also be noted that in some realizations alternately, the function of being marked in square frame can also be according to different from attached The order marked in figure occurs.For example, two square frames succeedingly represented can essentially perform substantially in parallel, or it Can also perform in a reverse order sometimes, this depends on involved function.It should also be noted that flow chart And/or the combination of each square frame and flow chart in block diagram and/or the square frame in block diagram, work(as defined in performing can be used Can or the special hardware based system of operation realize, or the combination of specialized hardware and computer instruction can be used Realize.
Term as used herein " including ", " including " and similar terms are understood to the term of opening, i.e., " Including/including but not limited to ", expression can also include other guide.Term " being based on " is " being based at least partially on ".Term " One embodiment " expression " at least one embodiment ";Term " another embodiment " expression " at least one further embodiment ", etc. Deng.
It may be not discussed in detail for technology, method and apparatus known to person of ordinary skill in the relevant, but suitable In the case of, the technology, method and apparatus should be considered as part for specification.For between each unit in accompanying drawing Line, it is only for be easy to illustrate, it represents that the unit at least line both ends is in communication with each other, it is not intended that limitation does not connect It can not be communicated between the unit of line.
Before the embodiment of present disclosure is introduced, core used in the present disclosure is introduced first Heart technology, mainly there are following three, i.e. machine learning algorithm, Spark technologies and Opentsdb technologies.
Machine learning algorithm includes dynamic linear models, Kalman filter and Bayes' theorem.Monitored data kind Class is various, such as machine performance data, application performance data and traffic fluctuations data etc..Have for these data analyses very more Mode, present disclosure mainly predicts index of correlation using dynamic linear models, Kalman filter and Bayes' theorem Fluctuation whether in rational scope.Dynamic linear models (DLM) is a kind of widely used time series models.Bayes Forecasting Methodology is the classical prediction algorithm of this model.Bayesian forecasting method does not depend solely on the conventional history number of t It is predicted according to the knowledge according to model, may also include the posterior infromation of expert and the judgement of subjectivity to be predicted, this It is particularly useful for prediction accident, and historical data and prespecified model can not reflect them completely.Work as hair When now model performance is bad, goes to the experience and information of expert for help, model is improved.Bayesian forecasting method, relatively For the traditional Time Series Methods of Box-Jenkins, the advantages of having it, it need not assume that Box-Jenkins methods must The stationarity of palpus is assumed.Bayesian forecasting method provides prior distribution by the subjective experience of people so that the requirement to data volume Greatly reduce.
Next Spark technologies are introduced.The KPI achievement data magnitudes of monitoring are huge, have with certain requirement of real-time, Therefore we carry out the cleaning of data from Spark, format, machine learning model training.Spark is as big number of future generation Show up prominently according to processing engine in the very short time, and the gesture to set a prairie fire sweeps across industry.Spark mainly have three it is excellent Point.First, Spark is very handy.Because advanced API has peeled off the concern to cluster in itself, you can be absorbed in you to be done Calculating in itself, only need on the notebook computer of oneself can exploitation Spark application.Secondly, Spark quickly, supports to hand over Mutual formula uses and complicated algorithm.Finally, Spark is a utility engines, and various computings can be completed with it, including SQL query, text-processing, machine learning etc., and before Spark appearance, we generally require the various engines of study To handle these demands respectively.This three big advantage also allows Spark as a good starting point for learning big data.
Finally introduce Opentsdb technologies.Huge monitoring data, certainly will need distributed, and the data efficiently accessed are deposited As support, present disclosure selects Opentsdb as core memory component for storage.Opentsdb is a distribution, expansible Time series databases (TSDB) HBase top.Opentsdb is to solve a common demand:Storage, index And the index system (application program) collected from computer system (network equipment, operation) is provided, and it is readily accessible to this data And it can draw.Due to HBase scalability, Opentsdb allows you to collect thousands of to come from tens thousand of individual main frames and application program Index, with high-speed (every several seconds).Opentsdb never deletes or reduced sample data, can easily store number 10000000000 data points.
Understand that present disclosure incorporates the history monitoring data of long term accumulation and utilized based on content described above Forecast model based on machine learning generation is monitored alarm.The forecast model can use dynamic linear based on historical data Model, Kalman filter and Bayes' theorem predict the reasonable of the index of subsequent time period fluctuation according to such as hour dimension Section.The setting of the reasonable interval of the program is based on historical data, is no longer dependent on simple micro-judgment, considerably reduces Manual intervention, miss announcement situation, O&M cost.
The schematic flow sheet according to the monitoring alarm method 200 described by present disclosure is described below with reference to Fig. 2. From the point of view of specific, the monitoring alarm method 200 according to present disclosure comprises the following steps, and first, is received in method and step 210 Include the data flow of the historical data associated with target component to be monitored;Then, it is sharp in ensuing method and step 220 The data flow is analyzed with dynamic linear models, to obtain for ensuing first time period and and target component Associated Prediction Parameters;The target component is counted in the first time period in method and step 230, to obtain Obtain in the first time period and actual parameter associated with the target component;And obtaining Prediction Parameters and reality After the parameter of border, the Prediction Parameters are alerted compared with the actual parameter with judging whether to generate in method and step 240 Signal.
Monitoring alarm method according to present disclosure is divided the data flow by introducing dynamic linear models Analysis, so as to obtain for ensuing first time period and associated with target component Prediction Parameters, using it is pre- The Prediction Parameters of survey and the target component is counted in the first time period in obtained first time period And the actual parameter associated with the target component just can accurately treat monitoring data fluctuation situation carry out science Management and the outputting alarm signal when abnormal.So it is the science and accuracy on the one hand improving prediction, the opposing party Face also greatly reduces announcement situation by mistake, finally can also reduce O&M cost.
Alternatively, utilized when the monitoring alarm method is additionally included in the dynamic linear models initialization and initialize time point The historical data associated with target component to be monitored in predetermined amount of time before is carried out to the dynamic linear models Initialization.The step of initialization, does not show that in fig. 2.Can be in initial phase to dynamic with such initialization mode Linear model optimizes, so that the dynamic linear models by historical data training optimizes.Still optionally further, energy Different dynamic linear models is enough set for the different periods, when dynamic linear models as escribed above includes being used for second Between section the first dynamic linear models and the second dynamic linear models for the 3rd period, wherein, in dynamic linear models During initialization using in the period corresponding with the second time period in the predetermined amount of time before initializing time point with The associated historical data of target component to be monitored is initialized to the dynamic linear models to obtain the first dynamic line Property model;And/or
When dynamic linear models initializes using in the predetermined amount of time before initializing time point with the 3rd time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain the second dynamic linear models.
Second time period herein is for example corresponding to 9 points of morning at 6 points in afternoon, and the 3rd period was at 6 points in afternoon Under 9 points one morning.The difference of different time sections can accurately be reflected the most by being handled respectively with such period Data volume, so as to further improve monitoring alarm accuracy.
But first time period may be across second time period and the 3rd period, now, the monitoring alarm method Further comprise the first time period respectively with the second time period and compared with the 3rd period, and will The first time period is divided into the first time period part overlapping with the second time period and overlapping with the 3rd period Second time period part;And judge whether to give birth to for the first time period part and the second time period part respectively Into alarm signal.
Even if consequently, it is possible to the period of required prediction across different dynamic linear models, according to present disclosure Monitoring alarm method also can make the accurate prediction most having for different situations, so as to the accuracy for follow-up monitoring alarm Sound assurance is provided.
At the beginning of just can carrying out differential dynamic linear models according to the data characteristicses of different periods in this way Beginningization, such as because daytime and evening sample data difference are obvious, the sample data daytime and evening of training pattern separate, prediction The data on daytime use the historical data training pattern on daytime, predict the data at night using the historical data training mould at night Type.
Alternatively, the Prediction Parameters include the forecast interval for the target component, and the forecast interval is for example by moving The average value and variance at the state linear model place of prediction are calculated, and by the Prediction Parameters and the actual parameter ratio Further comprise compared with to judge whether to generate alarm signal:Do not generated when the actual parameter is within the forecast interval Alarm signal;And generate alarm signal when the actual parameter is not within the forecast interval.Predicted by being formed Section, so as to judge whether actual parameter is reasonable, and decide whether to produce alarm signal accordingly.
As described above, historical data includes system data, application performance data and/or business datum.Alternatively, described One period was current time point ensuing five minutes, ten minutes and/or one hours.Be merely exemplary herein rather than limit Property processed, the first time period of other times length is also feasible.Alternatively, the data flow is included two before current time point The historical data associated with target component to be monitored in week.
In addition, above-mentioned collocation method can realize that this is deposited by way of tangible computer-readable recording medium Storage media includes instruction, when executed so that the processor of computer is at least used for:Receive include with it is to be monitored The associated historical data of target component data flow;The data flow is analyzed using dynamic linear models, to obtain Prediction Parameters ensuing first time period and associated with target component must be used for;To institute in the first time period State target component to be counted, to obtain actual parameter in the first time period and associated with the target component; And by the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.
Alternatively, above-mentioned collocation method can be realized by computer program product.Computer program product can be with Including computer-readable recording medium, containing the computer-readable program instructions for performing various aspects of the disclosure. Computer-readable recording medium can keep and store the tangible device that the instruction that equipment uses is performed by instruction.Calculate Machine readable storage medium storing program for executing can for example be but not limited to storage device electric, magnetic storage apparatus, light storage device, electromagnetism storage are set Standby, semiconductor memory apparatus or above-mentioned any appropriate combination.The more specifically example of computer-readable recording medium is (non- Exhaustive list) include:It is portable computer diskette, hard disk, random access memory (RAM), read-only storage (ROM), erasable Formula programmable read only memory (EPROM or flash memory), static RAM (SRAM), the read-only storage of Portable compressed disk Device (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example it is stored thereon with beating for instruction Hole card or groove internal projection structure and above-mentioned any appropriate combination.Computer-readable recording medium used herein above It is not construed as instantaneous signal in itself, the electromagnetic wave of such as radio wave or other Free propagations, passes through waveguide or other biographies The electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of defeated media transmission or the electric signal transmitted by electric wire.
In addition, present disclosure also proposed a kind of monitoring alarm device, Fig. 3 shows the monitoring according to present disclosure The structural representation of alarm device, it can be seen that the monitoring alarm device 300 according to present disclosure includes:Receive Module 310, the receiving module 310 are configured as receiving the number for including the historical data associated with target component to be monitored According to stream;Analysis module 320, the analysis module 320 are configured to, with dynamic linear models and the data flow are analyzed, To obtain for ensuing first time period and associated with target component Prediction Parameters;Statistical module 330, the system Meter module 330 is configured as counting the target component in the first time period, to obtain the very first time In the section and actual parameter associated with the target component;And judge module 340, the judge module 340 are configured For the Prediction Parameters are judged whether to generate alarm signal compared with the actual parameter.Although mould will be analyzed in figure 3 Block 320 and statistical module 330 are shown as two independent modules, but those skilled in the art is it is to be appreciated that when a mould When block can realize the function of analysis module 320 and statistical module 330, it is also feasible that the two modules are embodied as into a module 's.
The monitoring alarm method according to present disclosure is described below in conjunction with specific computer system.According to the disclosure The monitoring alarm method of content include data acquisition phase, the data cleansing stage, phase data memory, data analysis phase and The last output control stage.Each stage is described below in conjunction with specifically used computer system, but is described below It is being merely exemplary and nonrestrictive.
The main three parts data of data acquisition:Part I includes system data, such as central processing unit, memory, hard Disk, input and output and logining such as publish at the data;Part II includes application performance data, for example, request call take data, Jvm data etc.;Part III includes business datum, such as business processing success miss data, business interface processing data etc..Day The unified dynamic using from more than the data collecting system (such as warden systems) ground, system support each item data of will collection Collection, and send data to the processing module (such as kafka modules) specified.
Next need to carry out data cleansing to the data gathered.In the part, acquired monitoring data usually without Method is directly used, it is necessary to which these data cleansings to be first converted to the data format that can be used directly, and systematic unity uses at present Json forms, distributed Stream Processing is carried out by Spark and Jstorm, and the data handled are sent to the place more than Manage module (such as kafka modules).
In addition, cleaned and initial data can also carry out data storage.For example, from processing module (such as Kafka modules) in obtain data can be stored respectively in ES (elasticsearch) modules and hdfs modules, hdfs number of modules Permanently stored according to as filing data and off-line analysis data, and ES module datas as implement inquiry, statistics it Data can store 1 month to 6 months according to data type.It can be stored in Opentsdb in ES by the statistics of each dimension Data, for machine learning model training and threshold determination, while it can also store the model of training generation.
Next data will be analyzed, now, the training mission of machine learning model is using submission Spark tasks (Job) mode timing obtains newest data to update iterative model from Opentsdb, and updated model is stored Opentsdb.Refree systems are that the system from Opentsdb by obtaining newest model from the threshold determination system ground And statistics, carry out threshold determination, threshold determination.
It is finally output control, in this process, output control module supports the data that come into force to threshold determination to alert, Storage, call the functions such as other systems API.These data can be used for the displaying of monitoring data and monitoring effect.
Below in conjunction with specific embodiment, object is including 15 business banks and including Tengxun and branch in this embodiment The payment mechanism of the Internet type of Fu Baodeng Third-party payments mechanism, therefore mechanism of exchange is 16.With mobile Internet Development, each business bank and Ali Tengxun these internet industrys giant enter on-line payment business scope, client one after another After either Claims Resolution etc. is saved from damage in purchase declaration form, many payment method selections are there is, up to now, support online hand over Easy bank is mainly state-owned four big rows and 11 large-scale joint-stock commercial banks, the Alipay and Tengxun for Ali Wechat pays us and also supported.Due to each banking independence, we just certainly will for each bank transaction health whether Separated monitoring, in addition, Alipay and wechat pay below still corresponding each bank commerce bank, but due to us not It can accurately take and which specific bank belonged to, therefore temporarily by both a kind of transaction channel of playback in this monitoring.
For each bank, we just need the data cases returned according to bank transaction in the past period, judged The bank transaction channel whether still in desired extent, and one section exceed it is contemplated that normal level, should just have one from The mechanism of dynamic alarm, notice carry out necessary processing to related director.
For a transaction of generation, no matter success or not, we have carried out the processing of print log, each daily record We are by the storage in real time of the stream process such as kafka modules technology to data storing platform as ES modules in case follow-up monitoring Calling.
Each daily record mainly includes following information:
Transaction categories are mainly comprising payment, gathering;
Client IP and host name, the IP address and host information of client;
Exchange hour, the time that order occurs;
Order number and the amount of money, the order number and the amount of money of transaction;
Stateful transaction, mainly successfully fail and Sorry, your ticket has not enough value these three;And
Routing Number, the code of each bank.
In addition, daily record is not repeating herein also comprising information necessary to a complete order.
For above-mentioned data, according to type of transaction, which bank of family merchandises success or not and belongs to as we Three dimensions, count total number of deals that each bank occur within each period respectively and successfully fail and remaining sum not Foot number of deals, thus obtain one it is using time as index, have 16*2*3+16*2=128 arrange time series, represent respectively The situation of each transaction channel, it is contemplated that the independence of the transaction interface of each bank, therefore, we are it is not intended that different silver The relevance of transaction data in the ranks.
Next the algorithm model of dynamic linear models used in present disclosure is simply introduced.State space mould The method that type provides the analysis and predicted time sequence data of very abundant, is widely used in statistics, economics and letter Number processing etc. field.Dynamic linear models (Dynamic Linear Model) is one kind special in state-space model, it With very big flexibility, can read positioning soon has a most important row in multiple characteristics, passes through Kalman filtering Device (Kalman Filter), moreover it is possible to build likelihood function and simulate unobservable state, for one non-stationary of simulation Time series data is highly effective.
Dynamic linear equation can describe simply by following a pair of equation groups:
yt=Ftθt+vt,vt~N (0, Vt)
θt=Gtθt-1+wt,wt~N (0, Wt)
ytIt is in θtThe observation at place, this text is we assume that be scalar or a vector;
θt=(θ1,12,1,…,θp,1) be t parameter;
FtIt is the coefficient matrix of a coefficient 1x p dimension;
GtIt is the transformed matrix of a p x p dimension;
vtAnd wtIt is observation error and model error respectively, it is that 0 variance is V that they obey average respectivelytAnd WtNormal state point Cloth;
It is, in general, that dynamic linear models can be provided (pdfs) by following two conditional probability density functions:f(yt│θt) With g (θt│θt-1), if trying to achieve posterior probability p (θt│Dt), wherein Dt={ y1,y2,…,yt, you can by Bayes' theorem, Can inverse obtain corresponding probability density function, and provide observation ytReasonable layout section.
If it should be noted that although being referred to the equipment for drying or sub-device of equipment in detailed descriptions above, but it is this Division is merely exemplary rather than enforceable.In fact, in accordance with an embodiment of the present disclosure, two or more above-described dresses The feature and function put can embody in one apparatus.Conversely, the feature and function of an above-described device can be with It is further divided into being embodied by multiple devices.
Embodiment of the disclosure alternative embodiment is the foregoing is only, is not limited to embodiment of the disclosure, for For those skilled in the art, embodiment of the disclosure can have various modifications and variations.It is all in embodiment of the disclosure Within spirit and principle, any modification for being made, equivalence replacement, improvement etc., the protection of embodiment of the disclosure should be included in Within the scope of.
Although describe embodiment of the disclosure by reference to some specific embodiments, it should be appreciated that, the disclosure Embodiment is not limited to disclosed specific embodiment.Embodiment of the disclosure be intended to appended claims spirit and In the range of included various modifications and equivalent arrangements.Scope of the following claims meets broadest explanation, so that comprising All such modifications and equivalent structure and function.

Claims (16)

  1. A kind of 1. monitoring alarm method, it is characterised in that the monitoring alarm method includes:
    Receiving includes the data flow of the historical data associated with target component to be monitored;
    The data flow is analyzed using dynamic linear models, to obtain for ensuing first time period and and mesh Mark the associated Prediction Parameters of parameter;
    The target component is counted in the first time period, with obtain it is in the first time period and with it is described The associated actual parameter of target component;And
    By the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.
  2. 2. monitoring alarm method according to claim 1, it is characterised in that the monitoring alarm method also includes:
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with mesh to be monitored The associated historical data of mark parameter initializes to the dynamic linear models.
  3. 3. monitoring alarm method according to claim 2, it is characterised in that the dynamic linear models includes being used for second The first dynamic linear models of period and the second dynamic linear models for the 3rd period, wherein,
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with second time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain first dynamic linear models;And/or
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the 3rd time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain second dynamic linear models.
  4. 4. monitoring alarm method according to claim 3, it is characterised in that the monitoring alarm method further comprises:
    By the first time period respectively with the second time period and compared with the 3rd period, and
    By the first time period be divided into the first time period part overlapping with the second time period and with the 3rd time The overlapping second time period part of section;And
    Judge whether to generate alarm signal for the first time period part and the second time period part respectively.
  5. 5. monitoring alarm method according to claim 3, it is characterised in that the second time period is 9 points of morning under 6 points of noon, and the 3rd period be at 6 points in afternoon under 9 points one morning.
  6. 6. monitoring alarm method according to claim 1, it is characterised in that the Prediction Parameters include being used for the target The forecast interval of parameter, and the Prediction Parameters are entered one compared with the actual parameter to judge whether to generate alarm signal Step includes:
    Alarm signal is not generated when the actual parameter is within the forecast interval;And
    Alarm signal is generated when the actual parameter is not within the forecast interval.
  7. 7. monitoring alarm method according to claim 1, it is characterised in that the historical data includes system data, answered With performance data and/or business datum.
  8. 8. monitoring alarm method according to claim 1, it is characterised in that the first time period is that current time point connects down Five minutes, ten minutes and/or one hours come.
  9. 9. monitoring alarm method according to claim 1, it is characterised in that the data flow is included two before current time point The historical data associated with target component to be monitored in week.
  10. 10. a kind of tangible computer-readable recording medium, the storage medium includes instruction, when executed, So that the processor of the computer is at least used for:
    Receiving includes the data flow of the historical data associated with target component to be monitored;
    The data flow is analyzed using dynamic linear models, to obtain for ensuing first time period and and mesh Mark the associated Prediction Parameters of parameter;
    The target component is counted in the first time period, with obtain it is in the first time period and with it is described The associated actual parameter of target component;And
    By the Prediction Parameters compared with the actual parameter with judge whether generate alarm signal.
  11. 11. computer-readable recording medium according to claim 10, the storage medium includes instruction, when the instruction When being performed, also the processor of the computer is at least used for:
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with mesh to be monitored The associated historical data of mark parameter initializes to the dynamic linear models.
  12. 12. computer-readable recording medium according to claim 10, when the dynamic linear models includes being used for second Between section the first dynamic linear models and the second dynamic linear models for the 3rd period, the storage medium include referring to Order, when executed, also the processor of the computer is at least used for:
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with second time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain first dynamic linear models;And/or
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the 3rd time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain second dynamic linear models.
  13. 13. a kind of monitoring alarm device, it is characterised in that the monitoring alarm device includes:
    Receiving module, the receiving module are configured as receiving including the historical data associated with target component to be monitored Data flow;
    Analysis module, the analysis module are configured to, with dynamic linear models and the data flow are analyzed, to obtain For ensuing first time period and associated with target component Prediction Parameters;
    Statistical module, the statistical module are configured as counting the target component in the first time period, with Obtain actual parameter in the first time period and associated with the target component;And
    Judge module, the judge module are configured as the Prediction Parameters compared with the actual parameter to judge whether to give birth to Into alarm signal.
  14. 14. monitoring alarm device according to claim 13, it is characterised in that the monitoring alarm device also includes:
    Initialization module, the initialization module, which is configured as utilizing when the dynamic linear models initializes, initializes time point The historical data associated with target component to be monitored in predetermined amount of time before is carried out to the dynamic linear models Initialization.
  15. 15. monitoring alarm device according to claim 13, it is characterised in that the dynamic linear models includes being used for the The first dynamic linear models of two periods and the second dynamic linear models for the 3rd period, wherein,
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with second time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain first dynamic linear models;And/or
    When the dynamic linear models initializes using in the predetermined amount of time before initializing time point with the 3rd time The historical data associated with target component to be monitored in the section corresponding period is carried out to the dynamic linear models Initialize to obtain second dynamic linear models.
  16. 16. monitoring alarm device according to claim 15, it is characterised in that the monitoring alarm device further wraps Include:
    Time match module, the time match module be configured as by the first time period respectively with the second time period When being compared with the 3rd period and the first time period being divided into first overlapping with the second time period Between section part and the second time period part overlapping with the 3rd period;And wherein,
    The judge module is configured to be directed to the first time period part and the second time period part respectively Judge whether to generate alarm signal.
CN201710950552.2A 2017-10-13 2017-10-13 Monitoring alarm method, monitoring alarm device and computer readable storage medium Active CN107608862B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710950552.2A CN107608862B (en) 2017-10-13 2017-10-13 Monitoring alarm method, monitoring alarm device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710950552.2A CN107608862B (en) 2017-10-13 2017-10-13 Monitoring alarm method, monitoring alarm device and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN107608862A true CN107608862A (en) 2018-01-19
CN107608862B CN107608862B (en) 2020-10-27

Family

ID=61068809

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710950552.2A Active CN107608862B (en) 2017-10-13 2017-10-13 Monitoring alarm method, monitoring alarm device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN107608862B (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108710099A (en) * 2018-05-24 2018-10-26 广州供电局有限公司 Capacitance type potential transformer monitoring alarm method and system
CN109039727A (en) * 2018-07-24 2018-12-18 中国银行股份有限公司 Message queue monitoring method and device based on deep learning
CN109582529A (en) * 2018-09-29 2019-04-05 阿里巴巴集团控股有限公司 A kind of setting method and device of alarm threshold value
CN109709389A (en) * 2018-11-30 2019-05-03 珠海派诺科技股份有限公司 For electric instrument distributed mass real time data sampling alarm method and system
CN109800127A (en) * 2019-01-03 2019-05-24 众安信息技术服务有限公司 A kind of system fault diagnosis intelligence O&M method and system based on machine learning
CN110647070A (en) * 2019-09-05 2020-01-03 国家计算机网络与信息安全管理中心 Power environment monitoring system for super-large-scale data center
CN110658905A (en) * 2019-09-23 2020-01-07 珠海格力电器股份有限公司 Early warning method, early warning system and early warning device for equipment running state
CN110673549A (en) * 2019-09-05 2020-01-10 国家计算机网络与信息安全管理中心 Data center intelligent monitoring system and method based on distributed architecture
CN111159524A (en) * 2019-12-30 2020-05-15 北京同邦卓益科技有限公司 Monitoring method and system, and storage medium
WO2020125290A1 (en) * 2018-12-18 2020-06-25 中兴通讯股份有限公司 Data processing method, system, and storage medium
CN111597094A (en) * 2020-05-20 2020-08-28 神州数码融信软件有限公司 IT system monitoring method and device
CN111679952A (en) * 2020-06-08 2020-09-18 中国银行股份有限公司 Alarm threshold generation method and device
CN113220542A (en) * 2021-04-01 2021-08-06 深圳市云网万店科技有限公司 Early warning method and device for computing task, computer equipment and storage medium
CN114253808A (en) * 2021-12-21 2022-03-29 奇安信科技集团股份有限公司 Object state monitoring method and device
CN117539665A (en) * 2024-01-09 2024-02-09 珠海金智维信息科技有限公司 Efficient processing method for alarm event and computer readable storage medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050091005A1 (en) * 2003-10-27 2005-04-28 Jean-Francois Huard Computer performance estimation system configured to take expected events into consideration
CN104811344A (en) * 2014-01-23 2015-07-29 阿里巴巴集团控股有限公司 Network dynamic service monitoring method and apparatus
CN105204971A (en) * 2015-08-28 2015-12-30 浙江大学 Dynamic monitoring interval adjustment method based on Naive Bayes classification technology
CN105956734A (en) * 2016-04-15 2016-09-21 广东轩辕网络科技股份有限公司 Method and system for dynamically setting performance index threshold of IT equipment
CN106126391A (en) * 2016-06-28 2016-11-16 北京百度网讯科技有限公司 System monitoring method and apparatus
CN106250306A (en) * 2016-08-18 2016-12-21 电子科技大学 A kind of performance prediction method being applicable to enterprise-level O&M automatization platform
CN106407082A (en) * 2016-09-30 2017-02-15 国家电网公司 Method and device for alarming information system
CN106683340A (en) * 2016-12-15 2017-05-17 歌尔股份有限公司 User behavior monitoring method and wearable equipment
CN106844138A (en) * 2016-12-14 2017-06-13 北京奇艺世纪科技有限公司 O&M warning system and method
CN106909487A (en) * 2017-01-18 2017-06-30 北京盛世全景科技股份有限公司 It is applied to the method for early warning and device of information system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050091005A1 (en) * 2003-10-27 2005-04-28 Jean-Francois Huard Computer performance estimation system configured to take expected events into consideration
CN104811344A (en) * 2014-01-23 2015-07-29 阿里巴巴集团控股有限公司 Network dynamic service monitoring method and apparatus
CN105204971A (en) * 2015-08-28 2015-12-30 浙江大学 Dynamic monitoring interval adjustment method based on Naive Bayes classification technology
CN105956734A (en) * 2016-04-15 2016-09-21 广东轩辕网络科技股份有限公司 Method and system for dynamically setting performance index threshold of IT equipment
CN106126391A (en) * 2016-06-28 2016-11-16 北京百度网讯科技有限公司 System monitoring method and apparatus
CN106250306A (en) * 2016-08-18 2016-12-21 电子科技大学 A kind of performance prediction method being applicable to enterprise-level O&M automatization platform
CN106407082A (en) * 2016-09-30 2017-02-15 国家电网公司 Method and device for alarming information system
CN106844138A (en) * 2016-12-14 2017-06-13 北京奇艺世纪科技有限公司 O&M warning system and method
CN106683340A (en) * 2016-12-15 2017-05-17 歌尔股份有限公司 User behavior monitoring method and wearable equipment
CN106909487A (en) * 2017-01-18 2017-06-30 北京盛世全景科技股份有限公司 It is applied to the method for early warning and device of information system

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108710099A (en) * 2018-05-24 2018-10-26 广州供电局有限公司 Capacitance type potential transformer monitoring alarm method and system
CN108710099B (en) * 2018-05-24 2021-08-31 广东电网有限责任公司广州供电局 Monitoring and alarming method and system for capacitor voltage transformer
CN109039727A (en) * 2018-07-24 2018-12-18 中国银行股份有限公司 Message queue monitoring method and device based on deep learning
CN109039727B (en) * 2018-07-24 2021-08-06 中国银行股份有限公司 Deep learning-based message queue monitoring method and device
CN109582529A (en) * 2018-09-29 2019-04-05 阿里巴巴集团控股有限公司 A kind of setting method and device of alarm threshold value
CN109709389A (en) * 2018-11-30 2019-05-03 珠海派诺科技股份有限公司 For electric instrument distributed mass real time data sampling alarm method and system
CN109709389B (en) * 2018-11-30 2021-09-28 珠海派诺科技股份有限公司 Distributed high-capacity real-time data sampling and alarming method and system for power instrument
WO2020125290A1 (en) * 2018-12-18 2020-06-25 中兴通讯股份有限公司 Data processing method, system, and storage medium
CN109800127A (en) * 2019-01-03 2019-05-24 众安信息技术服务有限公司 A kind of system fault diagnosis intelligence O&M method and system based on machine learning
CN110647070A (en) * 2019-09-05 2020-01-03 国家计算机网络与信息安全管理中心 Power environment monitoring system for super-large-scale data center
CN110673549A (en) * 2019-09-05 2020-01-10 国家计算机网络与信息安全管理中心 Data center intelligent monitoring system and method based on distributed architecture
CN110658905A (en) * 2019-09-23 2020-01-07 珠海格力电器股份有限公司 Early warning method, early warning system and early warning device for equipment running state
CN110658905B (en) * 2019-09-23 2023-08-04 珠海格力电器股份有限公司 Early warning method, early warning system and early warning device for equipment operation state
CN111159524A (en) * 2019-12-30 2020-05-15 北京同邦卓益科技有限公司 Monitoring method and system, and storage medium
CN111597094A (en) * 2020-05-20 2020-08-28 神州数码融信软件有限公司 IT system monitoring method and device
CN111679952A (en) * 2020-06-08 2020-09-18 中国银行股份有限公司 Alarm threshold generation method and device
CN111679952B (en) * 2020-06-08 2023-09-19 中国银行股份有限公司 Alarm threshold generation method and device
CN113220542A (en) * 2021-04-01 2021-08-06 深圳市云网万店科技有限公司 Early warning method and device for computing task, computer equipment and storage medium
CN114253808A (en) * 2021-12-21 2022-03-29 奇安信科技集团股份有限公司 Object state monitoring method and device
CN117539665A (en) * 2024-01-09 2024-02-09 珠海金智维信息科技有限公司 Efficient processing method for alarm event and computer readable storage medium
CN117539665B (en) * 2024-01-09 2024-04-12 珠海金智维信息科技有限公司 Efficient processing method for alarm event and computer readable storage medium

Also Published As

Publication number Publication date
CN107608862B (en) 2020-10-27

Similar Documents

Publication Publication Date Title
CN107608862A (en) Monitoring alarm method, monitoring alarm device and computer-readable recording medium
Mehtab et al. Stock price prediction using CNN and LSTM-based deep learning models
Lux Estimation of agent-based models using sequential Monte Carlo methods
CN107730087A (en) Forecast model training method, data monitoring method, device, equipment and medium
CN110175788A (en) A kind of smart city energy cloud platform
CN108537544A (en) A kind of transaction system method for real-time monitoring and its monitoring system
CN109583904A (en) Training method, impaired operation detection method and the device of abnormal operation detection model
CN108446972A (en) Bank's Supervision of credit method, apparatus and fund position manage system
CN104115178A (en) Methods and systems for predicting market behavior based on news and sentiment analysis
Zhang et al. Deep stock ranker: A LSTM neural network model for stock selection
Duman et al. A novel and successful credit card fraud detection system implemented in a turkish bank
JP2023537879A (en) A System and Method for Next Basket Recommendation by Dynamic Attribute Modeling
CN110009502B (en) Financial data analysis method, device, computer equipment and storage medium
WO2020024718A1 (en) Method and device for predicting foreign exchange transaction volume
Kornevs et al. Cloud computing evaluation based on financial metrics
CN106407305A (en) Data mining system and method
Scherer et al. On the practical art of state definitions for Markov decision process construction
CN110544052A (en) method and device for displaying relationship network diagram
Huang et al. Hourly water demand forecasting using a hybrid model based on mind evolutionary algorithm
Skitmore et al. Bidding models: testing the stationarity assumption
CN115660728B (en) Air ticket sales order prediction method and device, electronic equipment and storage medium
US20140136280A1 (en) Predictive Tool Utilizing Correlations With Unmeasured Factors Influencing Observed Marketing Activities
CN116956994A (en) Service platform capacity expansion prediction method and device
EP3493082A1 (en) A method of exploring databases of time-stamped data in order to discover dependencies between the data and predict future trends
US11915180B2 (en) Systems and methods for identifying an officer at risk of an adverse event

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1249941

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20240320

Address after: Room 1179, W Zone, 11th Floor, Building 1, No. 158 Shuanglian Road, Qingpu District, Shanghai, 201702

Patentee after: Shanghai Zhongan Information Technology Service Co.,Ltd.

Country or region after: China

Address before: 518052 Room 201, building A, 1 front Bay Road, Shenzhen Qianhai cooperation zone, Shenzhen, Guangdong

Patentee before: ZHONGAN INFORMATION TECHNOLOGY SERVICE Co.,Ltd.

Country or region before: China

TR01 Transfer of patent right