CN110837452A - Method for detecting application program abnormity - Google Patents
Method for detecting application program abnormity Download PDFInfo
- Publication number
- CN110837452A CN110837452A CN201810939884.5A CN201810939884A CN110837452A CN 110837452 A CN110837452 A CN 110837452A CN 201810939884 A CN201810939884 A CN 201810939884A CN 110837452 A CN110837452 A CN 110837452A
- Authority
- CN
- China
- Prior art keywords
- application program
- data
- classification
- data set
- reading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method for detecting application program abnormity, which is characterized by comprising the following steps: the method comprises the following steps of 1, collecting log quantity X of the normal operation process of an application program, reading and writing times Y of a database, reading and writing times Z of a cache and access times W of other programs; step 2, dividing the data collected in the step 1 into three parts according to time periods; step 3, acquiring N groups of first data sets T1, second data sets T2 and third data sets T3; step 4, respectively using a linear regression classification algorithm to obtain three classification models for N groups of T1, T2 and T3; and 5, collecting and collecting data generated in the running process of the application program at regular time, and substituting the currently collected data into the classification model of the corresponding time period according to time to obtain a classification result. Compared with the prior art, the invention does not need an additional monitoring program which runs for a long time, thereby reducing the system load; the application program does not need to add an additional functional module to realize the communication with the monitoring program.
Description
Technical Field
The invention relates to a method for detecting application program abnormity.
Background
The existing methods for detecting application program exceptions mainly include the following three methods:
firstly, an application program reports the running state of the application program to another monitoring program periodically in the running process;
secondly, a heartbeat is maintained between the application program and another monitoring program, and the monitoring program periodically detects the state of the application program;
thirdly, the monitoring program directly checks the state of the application program through a scanning port or a process monitoring mode;
the first approach has the disadvantages that: an additional long-running monitoring program is needed; the application program needs to add an additional functional module to complete the reported task, so that the load of the application program is increased, and meanwhile, the error probability is increased; when the application program carries out multitasking, the subprogram responsible for reporting the state is independent from the main program, namely the state of the main program is often not monitored;
the second approach has the disadvantages that: the heartbeat of the application program and the main program is greatly influenced by network fluctuation; after the application program is abnormal, a large amount of system garbage can be caused because heartbeat connection cannot be timely recovered;
the third approach has the disadvantages that: the monitoring program cannot find the problems of the application program in time, and the real-time performance is poor.
In addition, in the existing monitoring mode, when the application program is in a false death state, because the output of the application program is not deeply analyzed, the general monitoring program monitors that the program still normally runs.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a method for detecting an application program exception with high detection accuracy and small system load for the above prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows: a method of detecting application exceptions, comprising: comprises the following steps
Step 1, collecting the following data generated in the normal operation process of an application program: the method comprises the following steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of a cache used by the application program, and access times W of other programs to the application program are determined;
step 2, dividing the data collected in the step 1 into three parts according to time 0-8 points, 8-20 points, 20-24 points, and respectively obtaining a first data set T1, a second data set T2 and a third data set T3;
step 3, repeatedly executing the step 1 and the step 2 to obtain N groups of first data sets T1, second data sets T2 and third data sets T3, wherein N is a natural number;
step 4, respectively using a linear regression classification algorithm, a KNN classification algorithm or a decision tree classification algorithm to obtain a first classification model, a second classification model and a third classification model for the N groups of first data set T1, second data set T2 and third data set T3;
and 5, collecting and collecting the following data generated in the running process of the application program at regular time: the method comprises the steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of the application program using a cache and access times W of other programs to the application program are obtained, then according to time, currently collected data are substituted into a classification model of a corresponding time period to obtain a classification result, if the classification result is less than or equal to 0.5, the output application program is abnormal, and if the classification result is greater than 0.5, the output application program runs normally.
Compared with the prior art, the invention has the advantages that: an extra monitoring program which runs for a long time is not needed, so that the system load is reduced; the application program does not need to add an additional functional module to realize the communication with the monitoring program.
Detailed Description
The method for detecting the application program exception comprises the following steps
Step 1, collecting the following data generated in the normal operation process of an application program: the method comprises the following steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of a cache used by the application program, and access times W of other programs to the application program are determined;
step 2, dividing the data collected in the step 1 into three parts according to time 0-8 points, 8-20 points, 20-24 points, and respectively obtaining a first data set T1, a second data set T2 and a third data set T3;
step 3, repeatedly executing the step 1 and the step 2 to obtain N groups of first data sets T1, second data sets T2 and third data sets T3, wherein N is a natural number which is more than or equal to 3;
step 4, respectively obtaining a first classification model, a second classification model and a third classification model for the N groups of the first data set T1, the second data set T2 and the third data set T3 by using a linear regression classification algorithm;
and 5, collecting and collecting the following data generated in the running process of the application program at regular time: the method comprises the steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of the application program using a cache and access times W of other programs to the application program are obtained, then according to time, currently collected data are substituted into a classification model of a corresponding time period to obtain a classification result, if the classification result is less than or equal to 0.5, the output application program is abnormal, and if the classification result is greater than 0.5, the output application program runs normally.
The linear regression classification algorithm used in the step 4 may also be replaced by a KNN classification algorithm or a decision tree classification algorithm, and the linear regression classification algorithm and the KNN classification algorithm or the decision tree classification algorithm are both conventional classification algorithms.
If a linear regression classification algorithm is used, the first classification model is obtained as follows:
step 1, forming a new matrix M (1, X, Y, Z, W) by using eigenvalues X, Y, Z, W in N groups of first data sets T1, taking the new matrix M and h (θ) as two variables in a linear equation, and obtaining the following thread equation: h (theta) ═ thetaTM, theta are parameter matrices, thetaTRepresents a transposition of θ;
and 2, processing h (theta) to obtain a probability model s (theta), wherein s (theta) is 1/(1+ e)-h(0))；
Step 3, defining a loss function
m represents the record number of the characteristic value X or the characteristic value Y or the characteristic value Z or the characteristic value W in the first data set T1;
yirepresenting the state value of the ith row of feature vectors in the first data set T1 on the server on which the application is running, yiEqual to 0 or 1, yiThe query can be directly inquired on a server operated by the application program;
xi: representative of the ith row of bits in the first data set T1The value of the eigenvector;
h0(xi): representing the predicted value of the feature vector of the ith row in the first data set T1;
and 4, selecting theta corresponding to the minJ (theta), wherein theta is a parameter matrix trained by the user, obtaining a final model h (theta), substituting h (theta) into a probability model s (theta), and taking the probability model s (theta) as a first classification model.
The second classification model and the third classification model are obtained in the same manner as the first classification model.
Claims (1)
1. A method of detecting application exceptions, comprising: comprises the following steps
Step 1, collecting the following data generated in the normal operation process of an application program: the method comprises the following steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of a cache used by the application program, and access times W of other programs to the application program are determined;
step 2, dividing the data collected in the step 1 into three parts according to time 0-8 points, 8-20 points, 20-24 points, and respectively obtaining a first data set T1, a second data set T2 and a third data set T3;
step 3, repeatedly executing the step 1 and the step 2 to obtain N groups of first data sets T1, second data sets T2 and third data sets T3, wherein N is a natural number;
step 4, respectively using a linear regression classification algorithm, a KNN classification algorithm or a decision tree classification algorithm to obtain a first classification model, a second classification model and a third classification model for the N groups of first data set T1, second data set T2 and third data set T3;
and 5, collecting and collecting the following data generated in the running process of the application program at regular time: the method comprises the steps that log quantity X of an application program, reading and writing times Y of the application program to a database, reading and writing times Z of the application program using a cache and access times W of other programs to the application program are obtained, then according to time, currently collected data are substituted into a classification model of a corresponding time period to obtain a classification result, if the classification result is less than or equal to 0.5, the output application program is abnormal, and if the classification result is greater than 0.5, the output application program runs normally.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810939884.5A CN110837452A (en) | 2018-08-17 | 2018-08-17 | Method for detecting application program abnormity |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810939884.5A CN110837452A (en) | 2018-08-17 | 2018-08-17 | Method for detecting application program abnormity |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110837452A true CN110837452A (en) | 2020-02-25 |
Family
ID=69574253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810939884.5A Pending CN110837452A (en) | 2018-08-17 | 2018-08-17 | Method for detecting application program abnormity |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110837452A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105204971A (en) * | 2015-08-28 | 2015-12-30 | 浙江大学 | Dynamic monitoring interval adjustment method based on Naive Bayes classification technology |
| CN106844161A (en) * | 2017-02-20 | 2017-06-13 | 重庆邮电大学 | Abnormal monitoring and Forecasting Methodology and system in a kind of carrier state stream calculation system |
| CN107291911A (en) * | 2017-06-26 | 2017-10-24 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| CN108228442A (en) * | 2016-12-14 | 2018-06-29 | 华为技术有限公司 | A kind of detection method and device of abnormal nodes |
-
2018
- 2018-08-17 CN CN201810939884.5A patent/CN110837452A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105204971A (en) * | 2015-08-28 | 2015-12-30 | 浙江大学 | Dynamic monitoring interval adjustment method based on Naive Bayes classification technology |
| CN108228442A (en) * | 2016-12-14 | 2018-06-29 | 华为技术有限公司 | A kind of detection method and device of abnormal nodes |
| CN106844161A (en) * | 2017-02-20 | 2017-06-13 | 重庆邮电大学 | Abnormal monitoring and Forecasting Methodology and system in a kind of carrier state stream calculation system |
| CN107291911A (en) * | 2017-06-26 | 2017-10-24 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108415789B (en) | Node fault prediction system and method for large-scale hybrid heterogeneous storage system | |
| CN110928718B (en) | Abnormality processing method, system, terminal and medium based on association analysis | |
| US8209567B2 (en) | Message clustering of system event logs | |
| US7388482B2 (en) | Method for the machine learning of frequent chronicles in an alarm log for the monitoring of dynamic systems | |
| CN113887616A (en) | Real-time abnormity detection system and method for EPG (electronic program guide) connection number | |
| EP3871120A1 (en) | Apparatus and method for detecting an anomaly among successive events and computer program product therefor | |
| CN105955214A (en) | Batch process fault detection method based on sample timing sequence and neighborhood similarity information | |
| CN111444072A (en) | Client abnormality identification method and device, computer equipment and storage medium | |
| WO2021012861A1 (en) | Method and apparatus for evaluating data query time consumption, and computer device and storage medium | |
| WO2023115856A1 (en) | Task exception alert method and apparatus | |
| CN112905380A (en) | System anomaly detection method based on automatic monitoring log | |
| CN114416783A (en) | Method and device for evaluating dynamic cost of OLAP (on-line analytical processing) query engine | |
| CN108171271B (en) | Early warning method and system for equipment degradation | |
| CN110837452A (en) | Method for detecting application program abnormity | |
| WO2024007580A1 (en) | Power equipment parallel fault diagnosis method and apparatus based on hybrid clustering | |
| CN117172093A (en) | Method and device for optimizing strategy of Linux system kernel configuration based on machine learning | |
| Guan et al. | A method of false alarm recognition based on k-nearest neighbor | |
| CN105897503B (en) | Hadoop cluster bottleneck detection method based on resource information gain | |
| Li et al. | Event block identification and analysis for effective anomaly detection to build reliable HPC systems | |
| CN109978038B (en) | Cluster abnormity judgment method and device | |
| CN113296994A (en) | Fault diagnosis system and method based on domestic computing platform | |
| CN114386311A (en) | Operation and maintenance abnormal data enhancement method and equipment based on key performance indexes | |
| Nehme | Database, heal thyself | |
| CN111475380A (en) | Log analysis method and device | |
| CN117194049B (en) | Cloud host intelligent behavior analysis method and system based on machine learning algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200225 |
|
| RJ01 | Rejection of invention patent application after publication |