CN105204971A - Dynamic monitoring interval adjustment method based on Naive Bayes classification technology - Google Patents

Dynamic monitoring interval adjustment method based on Naive Bayes classification technology Download PDF

Info

Publication number
CN105204971A
CN105204971A CN201510542115.8A CN201510542115A CN105204971A CN 105204971 A CN105204971 A CN 105204971A CN 201510542115 A CN201510542115 A CN 201510542115A CN 105204971 A CN105204971 A CN 105204971A
Authority
CN
China
Prior art keywords
monitoring
data
interval
classification
supervision interval
Prior art date
Application number
CN201510542115.8A
Other languages
Chinese (zh)
Other versions
CN105204971B (en
Inventor
尹建伟
陈怡东
赵新奎
李莹
邓水光
Original Assignee
浙江大学
苏州龙唐信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江大学, 苏州龙唐信息科技有限公司 filed Critical 浙江大学
Priority to CN201510542115.8A priority Critical patent/CN105204971B/en
Publication of CN105204971A publication Critical patent/CN105204971A/en
Application granted granted Critical
Publication of CN105204971B publication Critical patent/CN105204971B/en

Links

Abstract

The invention discloses a dynamic monitoring interval adjustment method based on a Naive Bayes classification technology. The dynamic monitoring interval adjustment method comprises the following steps: acquiring monitoring information of a host unit from the host unit for monitoring a target node through a monitoring system client Agent, storing the acquired monitoring information, preprocessing the data, and culling abnormal and invalid data; setting a classification feature set, and obtaining a classification result; and processing the classification result of the monitoring information of the current time point. According to the method provided by the invention, the monitoring problem in a complicated cloud computing environment can be solved, a precise and efficient monitoring solution is provided, and the running efficiency of a monitoring system is improved; and therefore, the dynamic monitoring interval adjustment method can be applicable to a load change caused by elastic stretching in the cloud computing environment, and a monitoring interval can be effectively adjusted.

Description

A kind of dynamic monitoring interval regulation method based on Naive Bayes Classification technology

Technical field

The present invention relates to cloud computing intelligent supervisory system technical field, particularly relate to a kind of dynamic monitoring interval regulation method based on Naive Bayes Classification technology.

Background technology

Cloud monitoring is along with the operation of whole cloud computing system, the operation performed is not only add to delete monitoring objective, obtain these shirtsleeve operations of data, after cloud monitoring obtains data, also need to make some process operation to the data obtained, allow user can for the demand self-defining data process behavior of oneself.The historical data of monitoring can regard a kind of daily record data of system information as, log analysis can be utilized to obtain history run state and the trend of system, thus make further decision-making.

Cloud computing system is a large-scale system normally, and for OpenStack, in system, each assembly can regard independently module as, has information interaction frequently between module and module.In cloud computing system, information is mutual all along with network service, and network bandwidth resources is very important resource in cloud computing system.The cloud monitoring introduced is above collected in the process of client metric, no matter is when Pull or Push mode, needs definition monitoring period granularity, and timing effectively stably obtains the data of monitoring.While acquisition in-depth monitoring data, need the high efficiency of the utilization of resources ensureing supervisory system, therefore need a strategy weighed to while obtaining effective monitoring data, the resource consumption rate of system is reduced.

System environments characteristic changeable in cloud environment, traditional static monitoring mode can not meet the demand of new environment, cloud monitoring needs can the monitoring period interval of dynamically adjustment System, while bringing less burden to system, can ensure the effective monitoring of system and stable operation.Monitoring period interval is configured in static file and makes it operationally not make a change, in the process of system cloud gray model, client and service end can be run according to this configuration information all the time, Agent timing can run collection information on client node, also can carry out data communication according to the time interval of configuration between service end and client.Therefore, when the metric data index monitored is very large time, bring certain performance burden will to monitoring objective node.For the design architecture of supervisory system, consider the design of extensibility, concrete monitor task is all performed by plug-in unit usually.When frequently on client node when operation monitoring plug-in unit, if some plug-in units operationally can consume certain system resource, as obtained JVM service data, or during the data of Tomcat and so on, need to do certain analysis to acquisition data, the burden brought to client node like this can not be ignored.

Cloud monitoring can change the configuration of its own system dynamically, operationally can adapt to current certain loads to be configured the adjustment of file.

Summary of the invention

For above-mentioned technological deficiency, the object of the invention is to there is information redundancy and the low inferior problem of the utilization of resources for the fixing monitoring period gap model of traditional monitoring software, system monitoring configuration feature is dynamically changed when adding operation in cloud supervisory system, propose a kind of dynamic monitoring interval regulation method based on Naive Bayes Classification technology, according to history monitor message, trend prediction is made to current state, and the result according to prediction carrys out decision-making system state, the monitoring of the monitoring period of dynamic conditioning is carried out with this, effectively can adjust supervision interval, optimize information transmission frequency, thus reach the Effec-tive Function of system.

In order to solve the problems of the technologies described above, technical scheme of the present invention is as follows:

Based on a dynamic monitoring interval regulation method for Naive Bayes Classification technology, comprise the steps:

1.1) obtained the monitor message of main frame by supervisory system client Agent from monitoring objective node host, choose CPU usage and memory usage information;

1.2) monitor message obtained is stored according to the mode of timestamp, main frame, monitor data value, and pre-service is carried out to data, rejecting abnormalities invalid data;

1.3) set characteristic of division set f, which includes four characteristic informations: CPU usage, CPU rate of change, memory usage, internal memory rate of change four eigenwerts, the category set in setting sorter, comprises 5 kinds of different behavior classifications to supervision interval adjustment; According to the set of monitoring objective Feature Selection reasonable data as initialization sample, initialization sample is brought in sorter and trains; The value calculated comprises statistical magnitude of all categories, total classification sample size, the statistical value of each eigenwert under different classes of; Add up above data and be used for prior probability p (c) of each classification calculated in model-naive Bayesian, eigenwert belongs to the Probability p (fi|c) of a certain classification; When classifying to current point in time monitor message, monitor data being substituted into sorter and classifies, obtaining classification results;

1.4) by step 1.3) classification results to current point in time monitor message that obtains, corresponding change is taked to supervision interval; The particle size range size of setting supervision interval is 1 ~ 20, and initialization supervision interval is defined as 5, and according to classifying, the result obtained can take increase 1 to supervision interval, increases 2, constant, reduces 1, reduce the behavior of 2, but maximum supervision interval can not more than 20; Supervision interval is changed by supervisory system dynamic-configuration function, and enter next monitoring period, the operation of supervision interval change behavior can be defined as i supervision interval cycle, and make sort operation is front i control point data can be added in sorter to carry out repetitive exercise at every turn;

1.5) by step 1.4) carry out sample size after iteration and can constantly expand, choosing a certain size threshold value and control sample size, after sorter obtains better stablizing effect, is conventional training sample data by Sample preservation.

Beneficial effect of the present invention is: method provided by the present invention can tackle the monitoring problem in cloud computing complex environment, provides and accurately monitors solution efficiently, improves supervisory control system running efficiency, monitor the advantage brought to cloud primarily of following some:

1. dynamically change monitoring objective configuration

2. adapt to load change in monitoring objective operational process, dynamically change monitoring and run interval

3. there is higher accuracy relative to traditional interval optimization method and other dynamic change methods

4. in system state, add change in resources rate factor, more reasonably predicting monitoring behavior

5. effectively decrease redundancy monitor message obtain behavior, enable supervisory system Effec-tive Function adapt to cloud computing environment Elastic stretch cause load change, effectively adjust supervision interval.

Accompanying drawing explanation

Fig. 1 is the category set definition in Naive Bayes Classification;

Fig. 2 is supervision interval adjust structure figure;

Fig. 3 is dynamic monitoring interval adjustment process flow diagram.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is described further.

The present invention is the basic data utilized in monitor message: CPU and memory information, carries out prediction classification, the method design supervision interval adjust structure of Fig. 2 to system state, performs corresponding supervision interval adjustment operation according to classification results.First the definition that some are relevant is given below.

CPU usage and memory usage, because these two indexs also exist polytype data in systems in which, such as use in the information of the top instruction display in Linux system, index in CPU mono-hurdle mainly comprises as follows: user model (user), system model (system), idle utilization rate (idle), during I/O waits for (iowait) etc.Internal storage data one hurdle in Top instruction shows total (memory amount), used (having used), free (idle available), the options such as buffers (cache size).

The present invention defines CPU and memory usage is:

C use=100%-C idle M u s e = M t o t a l - M f r e e M t o t a l

Namely in system, the CPU time of all consumption all counts among CPU usage, only has idle component as available CPU resource, and internal memory then considers idle available resources, and remainder is all thought of as and uses.

Naive Bayesian probability model is a conditional probability model, its most basic form as shown by the equation:

P ( A | B ) = P ( B | A ) P ( A ) P ( B )

Wherein P (A|B) represent when known occurs at event B, time A generation probability, this formula is the kernel model of whole model-naive Bayesian.First define the eigenwert of item to be sorted, defined CPU usage and memory usage, when considering system cloud gray model trend value change difference also should add, add rate of change two:

CPU rate of change: C d e l t a = C u s e ( t ) - C u s e ( t - 1 ) C u s e ( t - 1 )

Internal memory rate of change: M d e l t a = M u s e ( t ) - M u s e ( t - 1 ) M u s e ( t - 1 )

The item f={C to be sorted that namely defined feature vector occurs use, C delta, M use, M delta.For monitoring period interval, proper vector is all relatively independent for its impact produced above, because the classification value of arbitrary characteristics is if the situation that will increase, monitoring period interval just should increase for this service, otherwise will can not be guaranteed for this kind of monitoring resource health status.

Definition category set: C={-2 ,-1,0, the numerical value positive number in 1,2}, C set represents increase n monitoring period unit, and negative number representation reduces n chronomere, and 0 expression keeps monitoring period granularity constant.

Above-mentioned definition is applied to Bayes' theorem and then obtains following equation:

p ( c | f ) = p ( c ) p ( f | c ) p ( f )

Wherein c illustrates each in category set C, namely calculates the probability size that it is classified as a certain classification c when certain feature f occurs.P (c), c ∈ C} then illustrates the prior probability size of each classification c in training sample, and this can obtain by adding up training sample:

p ( c ) = C o u n t ( c ) T , c ∈ C

Because proper vector is all independently, so launch to obtain according to new probability formula:

p ( f | c ) = p ( ( C u s e , C d e l t a , M u s e , M d e l t a ) | c ) = Π i = 1 4 p ( f i | c ) , c ∈ C

Wherein f irepresent i-th in f feature.

Be constant for certain given feature for all categories p (f) in C, therefore can indicate:

p ( c | f ) ∝ p ( c ) Π i = 1 4 p ( f i | c )

Know by after the above-mentioned derivation of equation, result p (c|f) will be made when denominator is constant to maximize, as long as then make molecule maximize, draw decision function:

m a x ( p ( c ) Π i = 1 4 p ( f i | c ) ) , c ∈ C

The classification range interval that the value drawn according to decision function drops on obtains the result of classifying.Fig. 1 gives the scope of classifying in system.As can be seen from Figure 1, one has been divided into 5 classes, and does not take common normal distribution or the situation of decile, and the foundation choosing classification results in the present invention is as follows:

1. for CPU, the usual CPU of the system that normal load is not high operates in less than 10%, under the intermediateness that what under real scene, system seldom can be stable operate in as 30%-40%, the situation that load uprises can have been there is by illustrative system so become time the CPU usage worked as in system is more than 50%, may be there is the situation that card pauses in system, and the load value of the CPU of system also can rise significantly.So the time interval monitored just should be reduced when system appears in these classification time.

2. the impact of internal memory on classification is then different with CPU, for internal memory change usually than CPU come slow because the usual consumption internal memory wanted in a period of time slowly of service, or RAM leakage is also such situation.And for utilization rate, the memory system taking 75% also can normally work, and too much influence can not be brought to QoS.

3. and when indices is all smaller time, can think system stable operation, and the time interval of monitoring should tune up accordingly, do not need to obtain monitor data frequently because systems stabilisation is run, the reduction that the probability of system errors also can be relative.Add rate of change factor as feature, stable load represents that system state is good, and can consider to increase supervision interval, when system change rate is uprushed or changed frequent, then system should need more accurate monitor message.

Concrete dynamic change supervision interval process is as follows

One, obtains the monitor message of main frame from monitoring objective node host by supervisory system client Agent, Agent adopts card format to perform at destination node, and mode and the monitor service end of integrated use Pull/Push carry out data transmission.

2nd, stores according to the mode of timestamp, main frame, monitor data value the monitor message obtained, and defines CPU usage, CPU rate of change above monitor data comprises, memory usage, internal memory rate of change, with main frame association store, timestamp then adopts the unix timestamp of standard to store.

3rd. defining classification characteristic set f, which includes four characteristic informations: CPU usage, CPU rate of change, memory usage, internal memory rate of change four eigenwerts.Category set in defining classification device, includes 5 kinds of different behavior classifications to supervision interval adjustment.According to the set of monitoring objective Feature Selection reasonable data as initialization sample, raw data in sample is carried out initialization, monitor message numerical value substitutes in Fig. 1 tries to achieve classified information, then by corresponding grouped data stored in database, supervision interval that in sample data, known current point in time should be taked changes behavior.The statistics form stored is feature f ifor the statistical magnitude of a certain concrete classification c can be expressed as f ithe form of [c], brings into initialization sample in sorter and trains.Calculative value comprises statistical magnitude Count (c) of all categories, total classification sample size Sum (c), the statistical value of each eigenwert under different classes of and above-mentioned f i[c].Add up above data and be used for prior probability p (c) of each classification calculated in model-naive Bayesian, eigenwert belongs to the Probability p (f of a certain classification i| c).When current point in time monitor message is classified, monitor data is substituted into sorter and classifies, obtain classification results.

4th. the classification results to current point in time monitor message obtained by step 3, corresponding change is taked to supervision interval.The particle size range size of this method definition supervision interval is 1-20, initialization supervision interval is defined as 5, and according to classifying, the result obtained can take increase 1 to supervision interval, increases 2, constant, reduce 1, reduce the behavior of 2, but maximum supervision interval can not more than 20, supervision interval is changed by supervisory system dynamic-configuration function, and circulation enters next monitoring period, as shown in Figure 3, method is described below idiographic flow:

1. sample data described in initialization step three, carries out initialization training by sample data to sorter, if initial supervision interval interval=5, the behavior of definition supervision interval change operation is m supervision interval cycle and m*interval.

2. pair current point in time is classified, and front m control point data is added in sorter to carry out repetitive exercise and add monitoring historical trend information and predict the state judging current system.

3. the training aids after pair iteration calculates classification prior probability p (c), calculates eigenwert respective classes Probability p (f i| c), try to achieve current m a x ( p ( c ) Π i = 1 4 p ( f i | c ) ) , The value of c ∈ C, show that current monitor point classification results is set to classify.

4. according to classification results adjustment monitoring interval time, and limit supervision interval bounds according to definition:

interval=max(1,interval+ca),classify≤0

interval=min(threshold,interval+ca),classify>0

5. the training aids after current iteration is applied in the cyclic process of next adjustment.

It should be noted that in step 4 that after carrying out iteration, sample size can constantly expand, and chooses a certain size threshold value and controls sample size, after sorter obtains better stablizing effect, by Sample preservation for conventional training sample data can obtain good result.

The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, without departing from the inventive concept of the premise; can also make some improvements and modifications, these improvements and modifications also should be considered as in scope.

Claims (1)

1., based on a dynamic monitoring interval regulation method for Naive Bayes Classification technology, it is characterized in that, comprise the steps:
1.1) obtained the monitor message of main frame by supervisory system client Agent from monitoring objective node host, choose CPU usage and memory usage information;
1.2) monitor message obtained is stored according to the mode of timestamp, main frame, monitor data value, and pre-service is carried out to data, rejecting abnormalities invalid data;
1.3) set characteristic of division set f, which includes four characteristic informations: CPU usage, CPU rate of change, memory usage, internal memory rate of change four eigenwerts, the category set in setting sorter, comprises 5 kinds of different behavior classifications to supervision interval adjustment; According to the set of monitoring objective Feature Selection reasonable data as initialization sample, initialization sample is brought in sorter and trains; The value calculated comprises statistical magnitude of all categories, total classification sample size, the statistical value of each eigenwert under different classes of; Add up above data and be used for prior probability p (c) of each classification calculated in model-naive Bayesian, eigenwert belongs to the Probability p (fi|c) of a certain classification; When classifying to current point in time monitor message, monitor data being substituted into sorter and classifies, obtaining classification results;
1.4) by step 1.3) classification results to current point in time monitor message that obtains, corresponding change is taked to supervision interval; The particle size range size of setting supervision interval is 1 ~ 20, and initialization supervision interval is defined as 5, and according to classifying, the result obtained can take increase 1 to supervision interval, increases 2, constant, reduces 1, reduce the behavior of 2, but maximum supervision interval can not more than 20; Supervision interval is changed by supervisory system dynamic-configuration function, and enter next monitoring period, the operation of supervision interval change behavior can be defined as i supervision interval cycle, and make sort operation is front i control point data can be added in sorter to carry out repetitive exercise at every turn;
1.5) by step 1.4) carry out sample size after iteration and can constantly expand, choosing a certain size threshold value and control sample size, after sorter obtains better stablizing effect, is conventional training sample data by Sample preservation.
CN201510542115.8A 2015-08-28 2015-08-28 A kind of dynamic supervision interval method of adjustment based on Naive Bayes Classification technology CN105204971B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510542115.8A CN105204971B (en) 2015-08-28 2015-08-28 A kind of dynamic supervision interval method of adjustment based on Naive Bayes Classification technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510542115.8A CN105204971B (en) 2015-08-28 2015-08-28 A kind of dynamic supervision interval method of adjustment based on Naive Bayes Classification technology

Publications (2)

Publication Number Publication Date
CN105204971A true CN105204971A (en) 2015-12-30
CN105204971B CN105204971B (en) 2018-03-16

Family

ID=54952665

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510542115.8A CN105204971B (en) 2015-08-28 2015-08-28 A kind of dynamic supervision interval method of adjustment based on Naive Bayes Classification technology

Country Status (1)

Country Link
CN (1) CN105204971B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106095639A (en) * 2016-05-30 2016-11-09 中国农业银行股份有限公司 A kind of cluster subhealth state method for early warning and system
CN106161140A (en) * 2016-06-28 2016-11-23 中国联合网络通信集团有限公司 Determine method, monitor node and the group system of monitored node duty
CN106357469A (en) * 2016-11-16 2017-01-25 郑州云海信息技术有限公司 Dynamic adjustment method and device of resource monitoring mode
CN106407012A (en) * 2016-09-30 2017-02-15 青岛海信移动通信技术股份有限公司 Sampling method and device for load of central processing unit
CN106713423A (en) * 2016-12-06 2017-05-24 上海斐讯数据通信技术有限公司 Distributed data processing method and device for cloud access point controller
CN107483292A (en) * 2017-09-11 2017-12-15 电子科技大学 Dynamic monitoring and controlling method for cloud platform
CN107608862A (en) * 2017-10-13 2018-01-19 众安信息技术服务有限公司 Monitoring alarm method, monitoring alarm device and computer-readable recording medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6691067B1 (en) * 1999-04-07 2004-02-10 Bmc Software, Inc. Enterprise management system and method which includes statistical recreation of system resource usage for more accurate monitoring, prediction, and performance workload characterization
CN101281486A (en) * 2007-04-04 2008-10-08 英业达股份有限公司 Method for monitoring system environment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6691067B1 (en) * 1999-04-07 2004-02-10 Bmc Software, Inc. Enterprise management system and method which includes statistical recreation of system resource usage for more accurate monitoring, prediction, and performance workload characterization
CN101281486A (en) * 2007-04-04 2008-10-08 英业达股份有限公司 Method for monitoring system environment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
杨勇 等: "支持大规模云服务平台的敏捷弹性伸缩技术", 《华中科技大学学报》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106095639A (en) * 2016-05-30 2016-11-09 中国农业银行股份有限公司 A kind of cluster subhealth state method for early warning and system
CN106161140A (en) * 2016-06-28 2016-11-23 中国联合网络通信集团有限公司 Determine method, monitor node and the group system of monitored node duty
CN106161140B (en) * 2016-06-28 2019-07-02 中国联合网络通信集团有限公司 Determine method, monitoring node and the group system of monitored node working condition
CN106407012A (en) * 2016-09-30 2017-02-15 青岛海信移动通信技术股份有限公司 Sampling method and device for load of central processing unit
CN106357469B (en) * 2016-11-16 2019-05-28 郑州云海信息技术有限公司 A kind of dynamic adjusting method and device of monitoring resource mode
CN106357469A (en) * 2016-11-16 2017-01-25 郑州云海信息技术有限公司 Dynamic adjustment method and device of resource monitoring mode
CN106713423A (en) * 2016-12-06 2017-05-24 上海斐讯数据通信技术有限公司 Distributed data processing method and device for cloud access point controller
CN106713423B (en) * 2016-12-06 2019-11-29 上海斐讯数据通信技术有限公司 The processing method and processing device of distributed data in a kind of cloud access base site controller
CN107483292A (en) * 2017-09-11 2017-12-15 电子科技大学 Dynamic monitoring and controlling method for cloud platform
CN107608862A (en) * 2017-10-13 2018-01-19 众安信息技术服务有限公司 Monitoring alarm method, monitoring alarm device and computer-readable recording medium
CN107608862B (en) * 2017-10-13 2020-10-27 众安信息技术服务有限公司 Monitoring alarm method, monitoring alarm device and computer readable storage medium

Also Published As

Publication number Publication date
CN105204971B (en) 2018-03-16

Similar Documents

Publication Publication Date Title
He et al. Non-intrusive load disaggregation using graph signal processing
Lipton et al. Optimal thresholding of classifiers to maximize F1 measure
Yao et al. Beyond parity: Fairness objectives for collaborative filtering
US10198339B2 (en) Correlation-based analytic for time-series data
Haque et al. Sand: Semi-supervised adaptive novel class detection and classification over data stream
Wang et al. A hybrid wind speed forecasting model based on phase space reconstruction theory and Markov model: A case study of wind farms in northwest China
Masud et al. Addressing concept-evolution in concept-drifting data streams
CN106557778B (en) General object detection method and device, data processing device and terminal equipment
CN102930063B (en) Feature item selection and weight calculation based text classification method
Gao et al. Classifying data streams with skewed class distributions and concept drifts
Smith et al. Federated multi-task learning
EP2871577B1 (en) Complex event processing (CEP) based system for handling performance issues of a CEP system and corresponding method
US10187415B2 (en) Cognitive information security using a behavioral recognition system
Hamm et al. Crowd-ml: A privacy-preserving learning framework for a crowd of smart devices
CN103401939B (en) Load balancing method adopting mixing scheduling strategy
Wang et al. Efficient learning by directed acyclic graph for resource constrained prediction
US8078913B2 (en) Automated identification of performance crisis
CN105095866B (en) A kind of quick Activity recognition method and system
Chen et al. Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach
CN104168318A (en) Resource service system and resource distribution method thereof
Bifet et al. New ensemble methods for evolving data streams
CN104038392A (en) Method for evaluating service quality of cloud computing resources
US10067805B2 (en) Technologies for offloading and on-loading data for processor/coprocessor arrangements
CN103020288B (en) Method for classifying data stream under a kind of dynamic data environment
CN102111337A (en) Method and system for task scheduling

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
GR01 Patent grant