CN112508316B - Self-adaptive abnormality determination method and device in real-time abnormality detection system - Google Patents
Self-adaptive abnormality determination method and device in real-time abnormality detection system Download PDFInfo
- Publication number
- CN112508316B CN112508316B CN201910872258.3A CN201910872258A CN112508316B CN 112508316 B CN112508316 B CN 112508316B CN 201910872258 A CN201910872258 A CN 201910872258A CN 112508316 B CN112508316 B CN 112508316B
- Authority
- CN
- China
- Prior art keywords
- residual
- sequence
- real
- value
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005856 abnormality Effects 0.000 title claims abstract description 58
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 title claims abstract description 36
- 230000002159 abnormal effect Effects 0.000 claims abstract description 33
- 208000018910 keratinopathic ichthyosis Diseases 0.000 claims description 82
- 230000006870 function Effects 0.000 claims description 16
- 230000003044 adaptive effect Effects 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 17
- 238000012423 maintenance Methods 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
Abstract
The embodiment of the invention provides a self-adaptive abnormality judgment method and device in a real-time abnormality detection system. The method comprises the following steps: determining a threshold reference value according to residual indexes of the historical KPI data; determining a real-time adjustment reference value according to residual indexes of current KPI data; adaptively adjusting an abnormality judgment threshold according to the threshold reference value and the real-time adjustment reference value; and automatically carrying out abnormality judgment according to the abnormality judgment threshold. The embodiment of the invention can automatically set the KPI dynamic threshold, effectively reflect the abnormal threshold dynamic change caused by the real-time dynamic change of the system, and improve the accuracy of abnormal judgment.
Description
Technical Field
The invention relates to the technical field of system operation and maintenance, in particular to a self-adaptive abnormality judgment method and device in a real-time abnormality detection system.
Background
In the era of digitalization and informatization, various industries need a plurality of complicated and diversified large-scale software and hardware system supports. Meanwhile, the deployment, operation and maintenance of the systems all need professional operation and maintenance personnel, but the traditional operation and maintenance means are difficult to efficiently support a large-scale system operation and maintenance scene. In recent years, under the promotion of academia and industry, intelligent operation and maintenance (Artificial Intelligence for IT Operations, AIOps) is rapidly developed, and by applying artificial intelligence technology to the operation and maintenance field and combining a big data analysis system and a machine learning algorithm, the method continuously excavates, learns, refines and summarizes rules from massive operation and maintenance data (system logs, monitoring information, application information and the like), and provides a new solution for the operation and maintenance of a plurality of complex and diverse large-scale software and hardware systems at present. Wherein key performance indicator (Key Performance Indicator, KPI) anomaly detection is one of the underlying core technologies of intelligent operation and maintenance. When a KPI of a system exhibits anomalies (e.g., bumps, dips, jitters), it is often meant that the application associated with it has suffered some potential failure, such as network failure, server failure, configuration errors, etc. However, for a practical large-scale software and hardware system, a plurality of sub-modules are usually included, and each sub-module has a plurality of different types of KPIs to reflect the running states of the sub-modules. Such as a CDN (Content Delivery Network ) system, which is currently widely used, mainly includes functional modules such as CDN edge servers, load balancing, scheduling, and the like. For CDN edge servers, there are many different types of KPIs reflecting their performance, such as hit rate, first packet response time at hit, abnormal status code duty cycle at hit, etc. The objective of KPI anomaly detection is to monitor all KPIs that reflect the operational status of an actual large software and hardware system. When KPI indicators are abnormal (e.g., sudden increases, sudden decreases, jitter), it is often meant that a potential failure has occurred. For the abnormality determination of KPIs, the methods commonly used in the industry at present are mainly abnormality determination methods based on historical data, including fixed threshold methods based on experience, rada criteria based on n-tai distribution, and classification methods based on label-making sample learning.
The fixed threshold method is widely used by the primary anomaly detection system in the industry because of the advantages of simple algorithm, low cost and easy implementation. The operation and maintenance personnel set an abnormality threshold for different KPIs according to own experience, such as CPU utilization rate, and the operation and maintenance personnel set 90% as an abnormality judgment threshold according to own subjective experience, and display abnormality once the abnormality judgment threshold is greater than 90%.
The Laida criterion is also widely used by the primary anomaly detection system in the industry because of the advantages of simple algorithm, low cost and easy implementation. The Laida criterion is also called as 3 sigma criterion, it is to assume that a group of detection data only contains random error, calculate it to obtain standard deviation, and determine a section according to a certain probability, and consider that the error exceeding the section is not random error but coarse error, and the data containing the error should be removed. And 3σ applies when there are more sets of data. The 3 sigma principle is that the probability of the numerical value distribution in (mu-sigma, mu+sigma) is 0.6827, the probability of the numerical value distribution in (mu-2 sigma, mu+2 sigma) is 0.9545, the probability of the numerical value distribution in (mu-3 sigma, mu+3 sigma) is 0.9973, and the value of Y is almost entirely concentrated in the (mu-3 sigma, mu+3 sigma) interval, and the probability of exceeding the range is only less than 0.3%, so that the data can be judged as abnormal.
A classification method based on marking sample learning is to mark a part of historical KPIs into two types of abnormal and normal data by manpower, then to perform classification learning by using a supervision type machine learning method such as a Support Vector Machine (SVM) algorithm, an artificial neural network algorithm and the like, and then to apply the learned model to real-time data to perform abnormality judgment.
The three methods are seriously dependent on historical data, and can not effectively reflect abnormal threshold dynamic changes caused by the real-time dynamic changes of KPIs, so that the problems of poor monitoring performance and more false alarms are caused. Specifically, the fixed threshold method relies heavily on expert experience, the monitoring performance is poor, and false alarms are more. The Laida criterion method is only aimed at the curve of the KPI conforming to the positive distribution, and is difficult to adapt to the characteristics of the KPI curve such as diversified characteristics and real-time variation of abnormal probability, thereby causing a large number of false alarms. Typically, such KPIs can be on the order of millions for a large software and hardware system running on-line. The classification method based on the marking sample learning increases the manual marking cost on one hand, and can not effectively reflect the abnormal threshold dynamic change caused by the real-time dynamic change of the KPI on the other hand.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a self-adaptive anomaly determination method and device in a real-time anomaly detection system.
The embodiment of the invention provides a self-adaptive abnormality judgment method in a real-time abnormality detection system, which comprises the following steps:
determining a threshold reference value according to residual indexes of the historical KPI data;
determining a real-time adjustment reference value according to residual indexes of current KPI data;
adaptively adjusting an abnormality judgment threshold according to the threshold reference value and the real-time adjustment reference value;
and automatically carrying out abnormality judgment according to the abnormality judgment threshold.
The embodiment of the invention provides a self-adaptive abnormality judgment device in a real-time abnormality detection system, which comprises:
the first determining unit is used for determining a threshold value reference value according to residual indexes of the historical KPI data;
the second determining unit is used for determining a real-time adjustment reference value according to the residual index of the current KPI data;
the adjusting unit is used for adaptively adjusting an abnormality judgment threshold value according to the threshold value reference value and the real-time adjustment reference value;
and the judging unit is used for automatically judging the abnormality according to the abnormality judging threshold value.
The embodiment of the invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the self-adaptive abnormality judgment method in the real-time abnormality detection system when executing the program.
The embodiment of the invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the adaptive anomaly determination method in the real-time anomaly detection system.
According to the self-adaptive anomaly determination method and device in the real-time anomaly detection system, the anomaly determination threshold value is obtained through calculation according to the KPI historical data threshold value reference value and the real-time data dynamic threshold value adjustment reference value, and in the anomaly detection process, the anomaly determination threshold value is self-adaptively adjusted according to the system dynamic change, so that the KPI dynamic threshold value can be automatically set, the anomaly threshold value dynamic change caused by the system real-time dynamic change can be effectively reflected, and the anomaly determination accuracy is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of an adaptive anomaly determination method in a real-time anomaly detection system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for adaptive anomaly determination in a real-time anomaly detection system according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of KPI threshold anomaly determination according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an adaptive anomaly determination device in a real-time anomaly detection system according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flow chart illustrating an adaptive anomaly determination method in a real-time anomaly detection system according to an embodiment of the present invention.
As shown in fig. 1, the method specifically comprises the following steps:
s11, determining a threshold value reference value according to residual indexes of historical KPI data;
specifically, the embodiment of the invention can be applied to a real-time abnormality detection system of a CDN system, wherein the CDN system mainly comprises functional modules such as a CDN edge server, load balancing, scheduling and the like. For CDN edge servers, there are many different types of KPIs reflecting their performance, such as hit rate, first packet response time at hit, abnormal status code duty cycle at hit, etc.
The historical KPI data comprises actual values and predicted values of the historical KPIs, a historical KPI time sequence is formed according to time, and the residual indexes comprise average absolute errors, average absolute percentage errors, symmetrical average absolute errors and the like which are automatically calculated according to the actual values and the predicted values of the historical KPIs, and differences between the actual values and the predicted values of the KPI time sequence are captured from different angles by different residual indexes. According to the embodiment of the invention, the threshold reference value determined according to the residual index of the historical KPI is a proper threshold value obtained by automatic learning from the historical data, and the historical data information of the KPI can be reflected.
S12, determining a real-time adjustment reference value according to the residual error index of the current KPI data;
specifically, the current KPI data comprises actual values and predicted values of KPIs of a current designated period, a current KPI time sequence is formed according to time, and likewise, the residual indexes comprise average absolute errors, average absolute percentage errors, symmetrical average absolute errors and the like which are automatically calculated according to the actual values and the predicted values of the current KPIs, and the difference between the actual values and the predicted values of the current KPI time sequence is captured from different angles by different residual indexes. Considering that some key performance indexes sometimes fluctuate due to the dynamic property of a CDN system, the adjustment reference value determined according to the residual error index of the current KPI is a real-time KPI threshold adjustment reference value, and abnormal threshold dynamic change caused by the real-time dynamic change of the KPI can be reflected.
S13, adaptively adjusting an abnormality judgment threshold according to the threshold reference value and the real-time adjustment reference value;
specifically, the embodiment of the invention combines the KPI historical data threshold reference value with the real-time data dynamic threshold adjustment reference value to obtain a final self-adaptive abnormality judgment threshold value, wherein the abnormality judgment threshold value is used as the threshold value of online abnormality detection. In the abnormality detection process, the threshold adjustment reference value is dynamically adjusted according to the real-time dynamic information of the system, and finally the online threshold is adjusted in a self-adaptive mode.
According to the self-adaptive anomaly determination method in the real-time anomaly detection system, the anomaly determination threshold value is obtained by calculating the threshold value reference value of the KPI historical data and the dynamic threshold value adjustment reference value of the real-time data, and in the anomaly detection process, the anomaly determination threshold value is self-adaptively adjusted according to the dynamic change of the system, so that the dynamic change of the KPI can be automatically set, the dynamic change of the anomaly threshold value caused by the real-time dynamic change of the system is effectively reflected, and the anomaly determination accuracy is improved.
On the basis of the above embodiment, step S11 specifically includes:
calculating a residual sequence of each residual index according to the actual value and the predicted value of the historical KPI time sequence;
continuously moving the maximum value in the residual sequence into an abnormal set until the center distance between the abnormal set and the residual set is greater than a first threshold value and the abnormal proportion corresponding to the residual set is less than a second threshold value; wherein the set of anomalies and the set of residual residuals is the residual sequence;
and determining the maximum value in the residual error set as the threshold reference value.
Specifically, according to the actual value and the predicted value of the historical KPI time series, all residual indexes such as average absolute error, average absolute percentage error, symmetrical average absolute error, average absolute proportion error, expected average absolute percentage error and the like are calculated, residual sequences of different indexes are generated, and the difference between the actual value and the predicted value of the KPI time series can be captured from different angles.
The following threshold value reference value automatic acquisition algorithm is respectively carried out on the residual sequences of all residual indexes:
continuously moving the maximum value in the current residual sequence into an abnormal set, and when the center distance between the abnormal set and the residual set is far enough and the proportion of the residual set meeting the occurrence of the abnormality is smaller than a certain threshold value, considering that the current residual index can be used for abnormality judgment, and at the moment, the residual set is a normal set, and automatically acquiring the maximum value in the normal set, namely the determined threshold value reference value.
Further, the residual sequences of the five residual indexes respectively obtain five threshold reference values, and the threshold reference values are determined from different angles.
On the basis of the above embodiment, step S12 specifically includes:
calculating a residual sequence of each residual index according to the actual value and the predicted value of the current KPI time sequence;
determining the real-time adjustment reference value according to the mean value and the variance of the residual sequence, wherein the real-time adjustment reference value meets the following conditions:
and removing all residuals which are larger than the real-time adjustment reference value in the residual sequence from the residual sequence, so that the variation of the mean value and the variance of the residual sequence is maximum.
Specifically, according to the actual value and the predicted value of the current KPI time sequence, all residual indexes such as average absolute error, average absolute percentage error, symmetrical average absolute error, average absolute proportion error, expected average absolute percentage error and the like are calculated, residual sequences of different indexes are generated, and the difference between the actual value and the predicted value of the KPI time sequence can be captured from different angles.
The method comprises the steps of respectively calculating the mean value and the variance of a residual sequence aiming at the residual sequence of each residual index, and carrying out a heuristic algorithm for adjusting a reference value in real time according to the mean value and the variance, wherein the heuristic algorithm has the meaning that: if all residuals in the sequence of residuals above a certain threshold are removed from the sequence, the average and variance of the sequence of residuals will change relatively much, then the threshold is the most reasonable threshold.
Further, the residual sequences of the five residual indexes respectively obtain five threshold real-time adjustment reference values, and the real-time adjustment reference values are determined from different angles.
On the basis of the foregoing embodiment, the determining the real-time adjustment reference value according to the mean and the variance of the residual sequence includes:
determining candidate real-time adjustment reference values according to the mean value and the variance of the residual sequence;
constructing an objective function according to the variation of the mean and variance of the residual sequence, and determining a candidate real-time adjustment reference value which enables the objective function to be maximum as a final real-time adjustment reference value;
the objective function is:
s.t.
wherein ,adjusting the reference value vector for the candidate in real time,/->Mu is the mean of the residual sequence, sigma is the variance of the residual sequence,/i>Step for real-time adjustment of reference value for selectionA long vector; epsilon is a real-time adjustment reference value for maximizing the objective function;
for the smoothed residual sequence, +.>l is the length of a time sequence window for calculating residual errors, and t is the current moment;
the residual sequenceResidual error greater than the real-time adjustment reference value constitutes an outlier sequence +.>
The sequence of outliersThe successive outliers in (a) constitute an outlier sequence +.>
For the abnormality sequence +.>Element number of (2), ->For the abnormal interval sequence->The number of elements in the matrix.
Specifically, the KPI time series residuals are noted asSmoothing the residual by using an exponentially weighted moving average to obtain a smoothed residual +.>Selecting a group of candidate real-time adjustment reference value vectors according to the mean value and variance of the residual sequence> For the selected step size, the value can be +.> The candidate real-time adjustment reference value which makes the objective function maximum is calculated as the determined real-time adjustment reference value.
Further, the objective function is constructed by removing all residuals in the residual sequence, which are larger than the real-time adjustment reference value, from the residual sequence according to the meaning of a heuristic algorithm, so that the variation of the mean and the variance of the residual sequence is maximum.
Further, the method comprises the steps of,indicating that all are greater than +.>After the residual error of a certain threshold value in the sequence is removed from the sequence, the variation of the average value of the residual error sequence; />Indicating that all are greater than +.>After removing the residual of a certain threshold value from the sequence, the variance of the residual sequence is changed.
On the basis of the above embodiment, the adjustment formula of the abnormality determination threshold is:
ths＝pε+(1-p)ths em
wherein ths is the abnormality determination threshold, ε is the real-time adjustment reference value, p is the weight of ε, ths em Is the threshold reference value.
Specifically, wherein ths is em The method is a more stable threshold value automatically calculated according to the historical data set, and is more general; epsilon is an adjustment reference value calculated in real time by adopting a heuristic algorithm according to the current data, and the dynamic characteristics of the system can be reflected. The two are combined together through a certain weight to obtain the final abnormality judgment threshold value, so that the method has higher accuracy.
Further, the residual sequences of the five residual indexes respectively obtain five abnormal judgment thresholds, and the abnormal judgment is carried out from different angles.
Fig. 2 is a flow chart illustrating an adaptive anomaly determination method in a real-time anomaly detection system according to another embodiment of the present invention.
As shown in fig. 2, the method specifically comprises the following steps:
calculating threshold reference value ths from historical KPI data em ；
Calculating a real-time adjustment reference value epsilon according to the real-time KPI data;
according to the threshold reference value ths em And real-time adjusting the reference value epsilon to calculate the final online threshold;
and carrying out abnormality judgment according to the final online threshold value.
Fig. 3 shows a KPI threshold abnormality determination schematic diagram provided by an embodiment of the invention.
As shown in fig. 3, threshold is a set KPI threshold, and a region exceeding threshold is an abnormal region.
Fig. 4 is a schematic structural diagram of an adaptive anomaly determination device in a real-time anomaly detection system according to an embodiment of the present invention.
As shown in fig. 4, the apparatus includes: a first determination unit 41, a second determination unit 42, an adjustment unit 43, and a determination unit 44, wherein:
the first determining unit 41 is configured to determine a threshold reference value according to a residual index of the historical KPI data;
the second determining unit 42 is configured to determine a real-time adjustment reference value according to a residual index of the current KPI data;
the adjusting unit 43 is configured to adaptively adjust an abnormality determination threshold according to the threshold reference value and the real-time adjustment reference value;
the determining unit 44 is configured to automatically perform abnormality determination according to the abnormality determination threshold.
According to the self-adaptive anomaly determination device in the real-time anomaly detection system, the anomaly determination threshold value is obtained by calculation according to the KPI historical data threshold value reference value and the real-time data dynamic threshold value adjustment reference value, and in the anomaly detection process, the anomaly determination threshold value is self-adaptively adjusted according to the system dynamic change, so that the KPI dynamic threshold value can be automatically set, the anomaly threshold value dynamic change caused by the system real-time dynamic change can be effectively reflected, and the anomaly determination accuracy is improved.
On the basis of the above-described embodiment, the first determination unit 41 includes:
the first calculation module is used for calculating a residual sequence of each residual index according to the actual value and the predicted value of the historical KPI time sequence;
the processing module is used for continuously moving the maximum value in the residual sequence into an abnormal set until the center distance between the abnormal set and the residual set is greater than a first threshold value and the abnormal proportion corresponding to the residual set is less than a second threshold value; wherein the set of anomalies and the set of residual residuals is the residual sequence;
and the first determining module is used for determining that the maximum value in the residual error set is the threshold value reference value.
On the basis of the above embodiment, the first determining unit 42 includes:
the second calculation module is used for calculating a residual sequence of each residual index according to the actual value and the predicted value of the current KPI time sequence;
the second determining module is configured to determine the real-time adjustment reference value according to the mean value and the variance of the residual sequence, where the real-time adjustment reference value satisfies:
and removing all residuals which are larger than the real-time adjustment reference value in the residual sequence from the residual sequence, so that the variation of the mean value and the variance of the residual sequence is maximum.
On the basis of the above embodiment, the second determining module includes:
the first determining submodule is used for determining candidate real-time adjustment reference values according to the mean value and the variance of the residual sequence;
the second determining submodule is used for constructing an objective function according to the variation of the mean value and the variance of the residual sequence and determining a candidate real-time adjustment reference value which enables the objective function to be maximum as a final real-time adjustment reference value;
the objective function is:
s.t.
wherein ,adjusting the reference value vector for the candidate in real time,/->Mu is the mean of the residual sequence, sigma is the variance of the residual sequence,/i>Step size vector for selecting real-time adjustment reference value; epsilon is a real-time adjustment reference value for maximizing the objective function;
for the smoothed residual sequence, +.>l is the length of a time sequence window for calculating residual errors, and t is the current moment;
the residual sequenceResidual error greater than the real-time adjustment reference value constitutes an outlier sequence +.>
The sequence of outliersThe successive outliers in (a) constitute an outlier sequence +.>
For the abnormality sequence +.>Element number of (2), ->For the abnormal interval sequence->The number of elements in the matrix.
On the basis of the above embodiment, the adjustment formula of the abnormality determination threshold is:
ths＝pε+(1-p)ths em
wherein ths is the abnormality determination threshold, ε is the real-time adjustment reference value, p is the weight of ε, ths em Is the threshold reference value.
The adaptive anomaly determination device in the real-time anomaly detection system according to the present embodiment may be used to execute the above-described method embodiments, and the principle and technical effects are similar, and are not repeated here.
Fig. 5 illustrates a physical schematic diagram of an electronic device, as shown in fig. 5, which may include: processor (processor) 51, communication interface (Communications Interface) 52, memory (memory) 53 and communication bus 54, wherein processor 51, communication interface 52, memory 53 accomplish the communication between each other through communication bus 54. Processor 51 may invoke logic instructions in memory 53 to perform the methods provided by the various embodiments described above.
Further, the logic instructions in the memory 53 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, embodiments of the present invention also provide a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the method provided by the above embodiments.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. An adaptive anomaly determination method in a real-time anomaly detection system, the method comprising:
determining a threshold reference value according to residual indexes of the historical KPI data; the historical KPI data comprises actual values and predicted values of historical KPIs, a historical KPI time sequence is formed according to time, and the historical KPIs comprise hit rates of edge servers of CDN systems, first packet response time during hit and abnormal state code duty ratios during hit; the residual indexes comprise any combination of one or at least two of average absolute error, average absolute percentage error, symmetrical average absolute error, average absolute proportion error and expected average absolute percentage error which are automatically calculated according to the actual value and the predicted value of the historical KPI time sequence, and different residual indexes capture the difference between the actual value and the predicted value of the historical KPI time sequence from different angles;
determining a real-time adjustment reference value according to residual indexes of current KPI data;
adaptively adjusting an abnormality judgment threshold according to the threshold reference value and the real-time adjustment reference value;
automatically performing abnormality judgment according to the abnormality judgment threshold;
the determining the threshold reference value according to the residual error index of the historical KPI data comprises the following steps:
calculating a residual sequence of each residual index according to the actual value and the predicted value of the historical KPI time sequence;
continuously moving the maximum value in the residual sequence into an abnormal set until the center distance between the abnormal set and the residual set is greater than a first threshold value and the abnormal proportion corresponding to the residual set is less than a second threshold value; wherein the set of anomalies and the set of residual residuals is the residual sequence;
determining the maximum value in the residual error set as the threshold reference value;
the determining the real-time adjustment reference value according to the residual error index of the current KPI data comprises the following steps:
calculating a residual sequence of each residual index according to the actual value and the predicted value of the current KPI time sequence;
determining the real-time adjustment reference value according to the mean value and the variance of the residual sequence, wherein the real-time adjustment reference value meets the following conditions:
removing all residuals in the residual sequence, which are larger than the real-time adjustment reference value, from the residual sequence, so that the variation of the mean value and the variance of the residual sequence is maximum;
the determining the real-time adjustment reference value according to the mean value and the variance of the residual sequence comprises:
determining candidate real-time adjustment reference values according to the mean value and the variance of the residual sequence;
constructing an objective function according to the variation of the mean and variance of the residual sequence, and determining a candidate real-time adjustment reference value which enables the objective function to be maximum as a final real-time adjustment reference value;
the residual sequence includes: any combination of one or at least two of a mean absolute error sequence, a mean absolute percentage error sequence, a symmetric mean absolute error sequence, a mean absolute proportional error sequence, and a desired mean absolute percentage error sequence.
2. The adaptive anomaly determination method in a real-time anomaly detection system of claim 1, wherein the objective function is:
；
wherein ,adjusting the reference value vector for the candidate in real time,/->，/>For the mean value of the residual sequence, +.>For the variance of the residual sequence, +.>Step size vector for selecting real-time adjustment reference value; />Adjusting a reference value in real time to maximize the objective function;
for the smoothed residual sequence, +.>，/>For calculating the time series window length of the residual, +.>Is the current moment;
the residual sequenceResidual error greater than the real-time adjustment reference value constitutes an outlier sequence +.>；
The sequence of outliersThe successive outliers in (a) constitute an outlier sequence +.>；
For the abnormality sequence +.>Element number of (2), ->For the abnormal interval sequence->The number of elements in the matrix.
3. The adaptive anomaly determination method in a real-time anomaly detection system of claim 1, wherein the adjustment formula for the anomaly determination threshold is:
；
wherein ,determining a threshold value for said abnormality,>adjusting a reference value for said real time,/->Is->Weight of->Is the threshold reference value.
4. An adaptive anomaly determination device in a real-time anomaly detection system, the device comprising:
the first determining unit is used for determining a threshold value reference value according to residual indexes of the historical KPI data; the historical KPI data comprises actual values and predicted values of historical KPIs, a historical KPI time sequence is formed according to time, and the historical KPIs comprise hit rates of edge servers of CDN systems, first packet response time during hit and abnormal state code duty ratios during hit; the residual indexes comprise any combination of one or at least two of average absolute error, average absolute percentage error, symmetrical average absolute error, average absolute proportion error and expected average absolute percentage error which are automatically calculated according to the actual value and the predicted value of the historical KPI time sequence, and different residual indexes capture the difference between the actual value and the predicted value of the historical KPI time sequence from different angles;
the second determining unit is used for determining a real-time adjustment reference value according to the residual index of the current KPI data;
the adjusting unit is used for adaptively adjusting an abnormality judgment threshold value according to the threshold value reference value and the real-time adjustment reference value;
a judging unit for automatically judging the abnormality according to the abnormality judging threshold;
the first determination unit includes:
the first calculation module is used for calculating a residual sequence of each residual index according to the actual value and the predicted value of the historical KPI time sequence;
the processing module is used for continuously moving the maximum value in the residual sequence into an abnormal set until the center distance between the abnormal set and the residual set is greater than a first threshold value and the abnormal proportion corresponding to the residual set is less than a second threshold value; wherein the set of anomalies and the set of residual residuals is the residual sequence;
a first determining module, configured to determine a maximum value in the residual set as the threshold reference value;
the second determination unit includes:
the second calculation module is used for calculating a residual sequence of each residual index according to the actual value and the predicted value of the current KPI time sequence;
the second determining module is configured to determine the real-time adjustment reference value according to the mean value and the variance of the residual sequence, where the real-time adjustment reference value satisfies:
removing all residuals in the residual sequence, which are larger than the real-time adjustment reference value, from the residual sequence, so that the variation of the mean value and the variance of the residual sequence is maximum;
the second determining module includes:
the first determining submodule is used for determining candidate real-time adjustment reference values according to the mean value and the variance of the residual sequence;
the second determining submodule is used for constructing an objective function according to the variation of the mean value and the variance of the residual sequence and determining a candidate real-time adjustment reference value which enables the objective function to be maximum as a final real-time adjustment reference value;
the residual sequence includes: any combination of one or at least two of a mean absolute error sequence, a mean absolute percentage error sequence, a symmetric mean absolute error sequence, a mean absolute proportional error sequence, and a desired mean absolute percentage error sequence.
5. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the adaptive anomaly determination method in the real-time anomaly detection system of any one of claims 1 to 3 when the program is executed by the processor.
6. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the adaptive anomaly determination method in the real-time anomaly detection system of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910872258.3A CN112508316B (en) | 2019-09-16 | 2019-09-16 | Self-adaptive abnormality determination method and device in real-time abnormality detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910872258.3A CN112508316B (en) | 2019-09-16 | 2019-09-16 | Self-adaptive abnormality determination method and device in real-time abnormality detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112508316A CN112508316A (en) | 2021-03-16 |
| CN112508316B true CN112508316B (en) | 2023-08-08 |
Family
ID=74923872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910872258.3A Active CN112508316B (en) | 2019-09-16 | 2019-09-16 | Self-adaptive abnormality determination method and device in real-time abnormality detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112508316B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113380008B (en) * | 2021-05-12 | 2022-07-08 | 四川新网银行股份有限公司 | Dynamic threshold value adjusting method based on number of hits and hit rate |
| CN113099476B (en) * | 2021-05-13 | 2022-12-06 | 中国联合网络通信集团有限公司 | Network quality detection method, device, equipment and storage medium |
| CN114978956B (en) * | 2022-04-11 | 2024-04-09 | 北京邮电大学 | Method and device for detecting abnormal mutation points of performance of intelligent city network equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014001947A1 (en) * | 2012-06-28 | 2014-01-03 | Koninklijke Philips N.V. | A method of estimating the position of a device and an apparatus implementing the same |
| CN107832855A (en) * | 2017-09-14 | 2018-03-23 | 北京中恒博瑞数字电力科技有限公司 | Line loss multi-source diagnostic method and system based on correlation analysis |
| CN108460144A (en) * | 2018-03-14 | 2018-08-28 | 西安华光信息技术有限责任公司 | A kind of coal equipment fault early-warning system and method based on machine learning |
| CN109213654A (en) * | 2018-07-05 | 2019-01-15 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| WO2019012726A1 (en) * | 2017-07-14 | 2019-01-17 | Kabushiki Kaisha Toshiba | Abnormality detection device, abnormality detection method, and non-transitory computer readable medium |
-
2019
- 2019-09-16 CN CN201910872258.3A patent/CN112508316B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014001947A1 (en) * | 2012-06-28 | 2014-01-03 | Koninklijke Philips N.V. | A method of estimating the position of a device and an apparatus implementing the same |
| WO2019012726A1 (en) * | 2017-07-14 | 2019-01-17 | Kabushiki Kaisha Toshiba | Abnormality detection device, abnormality detection method, and non-transitory computer readable medium |
| CN107832855A (en) * | 2017-09-14 | 2018-03-23 | 北京中恒博瑞数字电力科技有限公司 | Line loss multi-source diagnostic method and system based on correlation analysis |
| CN108460144A (en) * | 2018-03-14 | 2018-08-28 | 西安华光信息技术有限责任公司 | A kind of coal equipment fault early-warning system and method based on machine learning |
| CN109213654A (en) * | 2018-07-05 | 2019-01-15 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112508316A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112508316B (en) | Self-adaptive abnormality determination method and device in real-time abnormality detection system | |
| Huong et al. | Detecting cyberattacks using anomaly detection in industrial control systems: A federated learning approach | |
| CN111652496B (en) | Running risk assessment method and device based on network security situation awareness system | |
| WO2022068645A1 (en) | Database fault discovery method, apparatus, electronic device, and storage medium | |
| CN111309565B (en) | Alarm processing method and device, electronic equipment and computer readable storage medium | |
| CN114978956B (en) | Method and device for detecting abnormal mutation points of performance of intelligent city network equipment | |
| US20210081501A1 (en) | System and method for automated insight curation and alerting | |
| CN110011879B (en) | Sensor network safety real-time online monitoring system based on parallel filtering | |
| CN111342988A (en) | Situation awareness-based network security early warning method and device | |
| CN108549981A (en) | A method of improving high-volume concurrent service flow services quality | |
| CN110647086B (en) | Intelligent operation and maintenance monitoring system based on operation big data analysis | |
| CN117216713A (en) | Fault delimiting method, device, electronic equipment and storage medium | |
| CN116663747A (en) | Intelligent early warning method and system based on data center infrastructure | |
| CN108282360B (en) | Fault detection method for long-term and short-term prediction fusion | |
| CN115878171A (en) | Middleware configuration optimization method, device, equipment and computer storage medium | |
| KR20210046423A (en) | Method and Apparatus for Security Management Based on Machine Learning | |
| CN109412885A (en) | Detection method and device | |
| CN111866924B (en) | Performance index monitoring method and device, computing equipment and computer storage medium | |
| US11337087B2 (en) | Methods, systems and computer readable media for predicting risk in network elements using machine learning | |
| CN113656452A (en) | Method and device for detecting abnormal index of call chain, electronic equipment and storage medium | |
| CN113535522A (en) | Abnormal condition detection method, device and equipment | |
| CN111722977A (en) | System inspection method and device and electronic equipment | |
| CN112231127A (en) | Electronic device and method for analyzing reliability of equipment | |
| CN112070283A (en) | Server operation health degree prediction method and system based on machine learning | |
| CN111080118B (en) | Quality evaluation method and system for new energy grid-connected data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |