CN113918433A - Adaptive intelligent network equipment performance index abnormity detection device and method - Google Patents
Adaptive intelligent network equipment performance index abnormity detection device and method Download PDFInfo
- Publication number
- CN113918433A CN113918433A CN202111135607.7A CN202111135607A CN113918433A CN 113918433 A CN113918433 A CN 113918433A CN 202111135607 A CN202111135607 A CN 202111135607A CN 113918433 A CN113918433 A CN 113918433A
- Authority
- CN
- China
- Prior art keywords
- data
- fluctuation type
- performance
- quartile
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3447—Performance evaluation by modeling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a self-adaptive intelligent network equipment performance index abnormity detection device and a method, wherein the device comprises: the data access module is used for accessing the performance data acquired by the network equipment; the data preprocessing module is used for processing the abnormal value of the data and standardizing the data processed by the abnormal value; the wave type classification module judges whether the performance index data belongs to a periodic fluctuation type, a flat slow fluctuation type or an irregular fluctuation type; an abnormality detection module for performing abnormality detection based on the waveform classification; and the result feedback module is used for acquiring the judgment results of all the performance indexes, summarizing the judgment results and feeding back the judgment results to operation and maintenance personnel. The device and the method can automatically judge the fluctuation type of the performance index, and implement a corresponding unsupervised anomaly detection scheme aiming at the performance data of different fluctuation types.
Description
Technical Field
The invention relates to the field of network equipment index abnormity detection, in particular to a self-adaptive intelligent network equipment performance index abnormity detection device and method.
Background
For the performance indexes collected by the network equipment, the performance indexes collected by different network equipment are different, and the fluctuation conditions of different indexes of the same equipment are different. For example: the inflow and outflow rate index regularly fluctuates in a period of 24 hours, the time series data of the CPU utilization rate fluctuate violently and irregularly, and the time series data of the temperature are relatively smooth.
In the prior art, the fluctuation types of different indexes need to be analyzed independently for realizing the abnormal detection of the indexes of the network equipment, and then a corresponding abnormal detection scheme is constructed for the indexes of the type. This approach requires a lot of human involvement and is less mobile. Meanwhile, the anomaly detection scheme is mainly divided into an unsupervised scheme and a supervised scheme, the supervised scheme depends on a large amount of labeled data, and on one hand, a large amount of index data needs to be accumulated, and on the other hand, a large amount of manpower needs to be spent on searching and labeling the anomaly data. In an actual service scene, the application limiting conditions of the supervision scheme are more, and the applicability is poorer.
Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides an adaptive apparatus and method for detecting abnormality of performance index of intelligent network device, which can automatically determine the fluctuation type of the performance index, and implement corresponding unsupervised abnormality detection schemes for performance data of different fluctuation types.
In order to achieve the purpose, the invention adopts the following technical scheme:
in an embodiment of the present invention, an adaptive apparatus for detecting abnormal performance index of an intelligent network device is provided, including:
the data access module is used for accessing the performance data acquired by the network equipment;
the data preprocessing module is used for processing the abnormal value of the data and standardizing the data processed by the abnormal value;
the wave type classification module judges whether the performance index data belongs to a periodic fluctuation type, a flat slow fluctuation type or an irregular fluctuation type;
an abnormality detection module for performing abnormality detection based on the waveform classification;
and the result feedback module is used for acquiring the judgment results of all the performance indexes, summarizing the judgment results and feeding back the judgment results to operation and maintenance personnel.
Further, the data in the data access module includes, but is not limited to, device cpu utilization, memory utilization, and ingress and egress traffic.
Further, the data acquisition time in the data access module is kept consistent.
Further, the abnormal value processing in the data preprocessing module adopts a box plot method, an upper quartile, a lower quartile and an upper and lower quartile difference of the performance index are calculated, the sum of the upper quartile and the 1.5 times of the quartile difference is used as an upper limit, the difference of the lower quartile and the 1.5 times of the quartile difference is used as a lower limit, the index value larger than the upper limit is adjusted to be an upper limit value, and the index value smaller than the lower limit is adjusted to be a lower limit value.
Further, the normalization in the data preprocessing module adopts mean variance normalization, the mean and the variance are respectively calculated for each column of performance index values, and the mean of each column of performance index values is subtracted from the index values of each column, and then the mean of each column of performance index values is divided by the variance of each column of performance index values.
Further, the wave pattern classification module trains a resnet deep network classification model by using historical data, and the steps are as follows:
step one, collecting historical performance data of network equipment, and carrying out category marking according to a periodic fluctuation type, a smooth fluctuation type and an irregular fluctuation type;
step two, constructing a resnet depth network classification model with one-dimensional input data;
and step three, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
Further, the anomaly detection module includes:
the periodic fluctuation type anomaly detection module learns the characteristics of historical data and predicts future data by using an fbprophet model, and judges whether the periodic time sequence data is abnormal or not according to the difference degree of the predicted data and the real-time data;
the steady fluctuation type anomaly detection module is used for evaluating the real-time data by using the historical data to calculate the reference value as an evaluation basis;
the irregular fluctuation type anomaly detection module is used for calculating the anomaly degree by using an LOF algorithm through characteristic construction of performance data, and further performing anomaly definition on real-time data.
In an embodiment of the present invention, a method for detecting abnormal performance indicators of an adaptive intelligent network device is further provided, including:
s01, accessing performance data collected by network equipment;
s02, carrying out abnormal value processing on the data, and standardizing the data after the abnormal value processing;
s03, judging whether the performance index data belongs to a periodic fluctuation type, a smooth fluctuation type or an irregular fluctuation type;
s04, carrying out abnormity detection according to waveform classification;
and S05, acquiring the judgment results of the performance indexes, summarizing the judgment results, and feeding back the judgment results to operation and maintenance personnel.
Further, the data in S01 includes, but is not limited to, device cpu utilization, memory utilization, and ingress and egress traffic.
Further, the data acquisition time in S01 is kept consistent.
In S02, the abnormal value processing is performed by using a box plot method to calculate the upper quartile, the lower quartile, and the upper and lower level differences of the performance index, using the sum of the upper quartile and the 1.5-fold difference as an upper limit, the difference of the lower quartile and the 1.5-fold difference as a lower limit, adjusting the index value larger than the upper limit to the upper limit, and adjusting the index value smaller than the lower limit to the lower limit.
Further, the normalization in S02 is performed by means of mean variance normalization, and a mean and a variance are calculated for each column of performance index values, and the mean of each column of performance index values is subtracted by the variance of each column of performance index values.
Further, the S03 trains a resnet deep network classification model by using the historical data, and includes the following steps:
s031, collect the historical performance data of the network equipment, and carry on the classification label according to periodic fluctuation type, gentle fluctuation type and irregular fluctuation type;
s032, constructing a resnet depth network classification model with one-dimensional input data;
s033, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
Further, the S04 includes:
s041, detecting the periodic fluctuation type abnormality, learning the characteristics of historical data and predicting future data by using an fbprophet model, and judging whether the periodic time sequence data is abnormal or not according to the difference degree of the predicted data and the real-time data;
s042, detecting a steady fluctuation type anomaly, and evaluating real-time data by using historical data to calculate a reference value as an evaluation basis;
and S043, detecting irregular fluctuation type abnormity, constructing the characteristics of the performance data, calculating the abnormity degree by using an LOF algorithm, and further performing abnormity definition on the real-time data.
In an embodiment of the present invention, a computer device is further provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the foregoing adaptive intelligent network device performance index anomaly detection method is implemented.
In an embodiment of the present invention, a computer-readable storage medium is further provided, where a computer program for executing the adaptive intelligent network device performance index abnormality detection method is stored in the computer-readable storage medium.
Has the advantages that:
1. the invention can automatically judge the fluctuation type of the performance index without a large amount of human participation and has stronger mobility.
2. The anomaly detection method is an unsupervised method, so that a large amount of index data accumulation is not needed, and a large amount of manpower is not needed for searching and marking the anomaly data. Therefore, in an actual service scene, wider application conditions exist, and the applicability is stronger.
Drawings
FIG. 1 is a schematic structural diagram of an apparatus for detecting abnormal performance indicators of an adaptive intelligent network device according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for detecting abnormal performance indicators of an adaptive intelligent network device according to an embodiment of the present invention;
FIG. 3 is a diagram of a resnet deep network classification model;
fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The principles and spirit of the present invention will be described below with reference to several exemplary embodiments, which should be understood to be presented only to enable those skilled in the art to better understand and implement the present invention, and not to limit the scope of the present invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, the invention provides a self-adaptive intelligent network equipment performance index abnormity detection device and method, which can automatically judge the fluctuation type of the performance index and implement a corresponding unsupervised abnormity detection scheme aiming at performance data of different fluctuation types.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.
Fig. 1 is a schematic structural diagram of an apparatus for detecting performance index abnormality of an adaptive intelligent network device according to an embodiment of the present invention. As shown in fig. 1, includes:
a data access module 110, configured to access performance data collected by a network device;
the data comprises performance indexes such as the CPU utilization rate of the equipment, the memory utilization rate, the inflow and outflow flow and the like.
Description of the data:
accessing the characteristic dimension of the index, different equipment models, equipment software and hardware versions, equipment service types and other equipment attributes can cause different collected performance indexes, and the number of the performance indexes can be any value;
the time granularity of the indexes is accessed, the acquisition time of each performance index is kept consistent, for example, performance data is acquired every 5 minutes;
accessing historical data of the indexes, wherein the time of the historical data is 2 weeks, and the part of data is used for calculating the abnormal detection judgment indexes;
accessing real-time data of the index, the real-time data being used for anomaly detection
Longitudinally splicing the historical data and the real-time data of the performance indexes into continuous time sequence data, wherein the last index value of each performance index sequence is the real-time data; and transversely splicing the performance index data by taking the acquisition time as an index, wherein each column is the data of one performance index.
The data preprocessing module 120 is used for performing abnormal value processing on the data and standardizing the data after the abnormal value processing;
the data preprocessing comprises abnormal value processing and standardization, wherein the abnormal value processing is carried out on the data, and then the data after the abnormal value processing is carried out is standardized.
The abnormal value processing adopts a box-line graph method to calculate an upper quartile, a lower quartile and an upper and lower quartile difference of the performance index, the sum of the upper quartile and the 1.5 times of the quartile difference is used as an upper limit, the difference of the lower quartile and the 1.5 times of the quartile difference is used as a lower limit, the index value larger than the upper limit is adjusted to be the upper limit, and the index value smaller than the lower limit is adjusted to be the lower limit.
The above-mentioned quantile differences are the differences between the upper quartile and the lower quartile.
The normalization was performed using mean variance normalization. The mean and variance are calculated separately for each column of performance indicator values, and then the mean of its column is subtracted separately from the indicator value of each column and then divided by the variance of its column.
The waveform classification module 130 determines that the performance index data belongs to a periodic fluctuation type, a flat slow fluctuation type or an irregular fluctuation type; the waveform classification module 130 is used to determine which of the above three types the unknown performance indicator data belongs to.
Firstly, historical performance data are collected and classified, and a resnet deep network classification model is trained by using the historical data.
And then, inputting the time sequence data of the performance index into a trained resnet deep network classification model to obtain the probability value of each fluctuation type of the performance index, and taking the fluctuation type of the maximum probability value as the fluctuation type of the performance index.
The training steps of the resnet deep network classification model are as follows:
step one, collecting historical performance data of network equipment, and carrying out category marking according to a periodic fluctuation type, a smooth fluctuation type and an irregular fluctuation type;
step two, constructing a resnet depth network classification model with one-dimensional input data:
the model body is composed of three convolution blocks: an identity block and two conv blocks, both of which are formed of convolution blocks of three different convolution kernel sizes, connected by residual edges. The only difference between these two blocks is that the residual edges of the conv block need to be convolved once separately.
The volume block is conv + bn + relu.
After the vector is input into the model, the vector passes through a convolution block, then the features are extracted through three convolution blocks, and finally the output is obtained through an average pooling layer and a full connection layer.
The vector refers to a one-dimensional vector of performance index time series data.
The schematic diagram of the algorithm model is shown in FIG. 3:
the loss function adopts a cross entropy loss function Cross EntropyLoss commonly used by classification tasks, and the optimizer adopts a random gradient descent SGD (gradient descent) to update model parameters, wherein the SGD is a random gradient descent method.
And step three, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
An anomaly detection module 140 for performing anomaly detection according to the waveform classification; the abnormality detection module 140 includes:
the periodic fluctuation type abnormality detection module 141 learns the historical data characteristics and predicts future data by using the fbprophet model, and determines whether the periodic time series data is abnormal or not according to the difference degree between the predicted data and the real-time data.
First, the fbprophet model is trained with partial historical data using performance indicator data. The input data are historical 13 days of performance index data, time points corresponding to the performance indexes and holiday characteristics. And using mse as a loss function, carrying out gradient reduction on the loss function once and again to obtain an optimal model parameter, and storing the optimal model parameter for predicting performance data.
Then, prediction is performed using a trained fbprophet model. Inputting the moment one day before the current time as a predicted starting time point, setting the prediction length as one day, and predicting by using a trained fbprophet model to obtain a predicted value and a prediction upper limit and a prediction lower limit of performance index data.
And finally, judging whether the data at the moment is abnormal or not according to the difference between the real-time performance data and the predicted data. If the real-time data are within the prediction upper and lower limit intervals, the data are considered to be abnormal, otherwise, the data are abnormal.
The stationary fluctuation type abnormality detection module 142 evaluates the real-time data by calculating a reference value using the historical data as an evaluation basis.
Firstly, the average value and the variance are calculated by using the historical data of the performance index, according to the 3sigma principle, the sum of the average value and the 3 times of variance is used as the upper limit, and the difference of the average value and the 3 times of variance is used as the lower limit, so that the upper and lower limit intervals of the historical data can be obtained.
And then, judging whether the data at the moment is abnormal or not according to the difference between the real-time performance data and the upper and lower limit intervals of the historical data. If the real-time data are in the upper and lower limit intervals of the historical data, the data are considered to be abnormal, otherwise, the data are abnormal.
The irregular fluctuation type anomaly detection module 143 performs feature construction on the performance data, calculates the anomaly degree using an LOF algorithm, and then performs anomaly definition on the real-time data.
Firstly, performing feature construction on historical data and real-time data of performance indexes, wherein the constructed features comprise: the ring ratio data of the previous hour, the homonymy data of the previous day, the homonymy data of the previous week, the moving average, the weighted moving average, the low-frequency wavelet characteristic value, the high-frequency wavelet characteristic value and the like.
And then, calculating the local density and the abnormal score of all performance indexes by using the performance index data of the built characteristics and adopting a local abnormal factor algorithm LOF. For a sample point p, if the anomaly score for that point is less than 1, it indicates that p is in a relatively dense region, unlike an anomaly point. If the anomaly score is much greater than 1, it indicates that p is distant from other points, and is likely to be an anomaly.
And finally, judging the abnormal score of the real-time data, if the abnormal score of the real-time data is less than 1, determining that the data is not abnormal, otherwise, determining that the data is abnormal.
And the result feedback module 150 acquires the judgment results of the performance indexes, summarizes the judgment results and feeds the judgment results back to the operation and maintenance personnel. And if the abnormal detection judgment results of the individual indexes at the moment are not abnormal, returning the information of normal detection. Otherwise, returning that the abnormity exists and returning the abnormal performance index information so as to facilitate the operation and maintenance personnel to check.
It should be noted that although several modules of the adaptive intelligent network device performance indicator anomaly detection apparatus are mentioned in the above detailed description, such partitioning is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module according to embodiments of the invention. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
For a more clear explanation of the foregoing adaptive intelligent network device performance index abnormality detection apparatus, a specific embodiment is described below, but it should be noted that this embodiment is only for better explaining the present invention and should not be construed as an undue limitation to the present invention.
Device label information:
911020000000000030590993 data collection completion:
completing the preprocessing of the abnormalDetection-INFO-Data loading completed Data:
the abrormaldetection-INFO-Preprocessing is completed feature name: SmfAvgIpv4AddrNum
The classification is as follows: periodic fluctuation type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfAvgIpv4AddrNum
abnormalDetection-INFO-Type:cyclicity
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfMaxPduSessNum
The classification is as follows: periodic fluctuation type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfMaxPduSessNum
abnormalDetection-INFO-Type:cyclicity
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessBuiFailNumNetworkErr
The classification is as follows: smooth wave type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：
SmfSessBuiFailNumNetworkErr
abnormalDetection-INFO-Type:stationary
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessBuiFailNumRej
The classification is as follows: irregular fluctuation pattern
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfSessBuiFailNumRej
abnormalDetection-INFO-Type:random
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessBuiFailNumRetErrcode
The classification is as follows: smooth wave type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：
SmfSessBuiFailNumRetErrcode
abnormalDetection-INFO-Type:stationary
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessBuiReqNum
The classification is as follows: periodic fluctuation type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfSessBuiReqNum
abnormalDetection-INFO-Type:cyclicity
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessBuiSuccnum
The classification is as follows: periodic fluctuation type
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfSessBuiSuccNum
abnormalDetection-INFO-Type:cyclicity
abnormalDetection-INFO-Result:normal
The feature name is as follows: SmfSessModiFailNum
The classification is as follows: irregular fluctuation pattern
And (4) testing results: is normal
abnormalDetection-INFO-Feature：SmfSessModiFailNum
abnormalDetection-INFO-Type:random
abnormalDetection-INFO-Result:normal
And (5) summarizing all characteristic results of the equipment:
the abnormal detection-INFO-The results of The retrieval are summarized and returned:
{'deviceID':'911020000000000030590993','result':'normal'}
based on the same invention concept, the invention also provides a self-adaptive intelligent network equipment performance index abnormity detection method. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 2 is a flowchart illustrating a method for detecting abnormal performance indicators of an adaptive intelligent network device according to an embodiment of the present invention. As shown in fig. 2, the method includes:
s01, accessing performance data collected by network equipment;
s02, carrying out abnormal value processing on the data, and standardizing the data after the abnormal value processing;
s03, judging whether the performance index data belongs to a periodic fluctuation type, a smooth fluctuation type or an irregular fluctuation type;
s04, carrying out abnormity detection according to waveform classification;
and S05, acquiring the judgment results of the performance indexes, summarizing the judgment results, and feeding back the judgment results to operation and maintenance personnel.
The data in S01 includes, but is not limited to, device cpu utilization, memory utilization, and ingress and egress traffic.
The data acquisition time in S01 is kept consistent.
In the processing of the abnormal value in S02, a box plot method is used to calculate the upper quartile, the lower quartile, and the upper and lower level differences of the performance index, the sum of the upper quartile and the level difference of 1.5 times is used as an upper limit, the difference of the lower quartile and the level difference of 1.5 times is used as a lower limit, the index value greater than the upper limit is adjusted to the upper limit, and the index value less than the lower limit is adjusted to the lower limit.
In the step S02, the normalization is performed by means of mean variance normalization, the mean and the variance are calculated for each column of performance index values, and the mean of each column is subtracted from each index value of each column and then divided by the variance of each column.
The S03 training a resnet deep network classification model by using historical data comprises the following steps:
s031, collect the historical performance data of the network equipment, and carry on the classification label according to periodic fluctuation type, gentle fluctuation type and irregular fluctuation type;
s032, constructing a resnet depth network classification model with one-dimensional input data;
s033, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
The S04 includes:
s041, detecting the periodic fluctuation type abnormality, learning the characteristics of historical data and predicting future data by using an fbprophet model, and judging whether the periodic time sequence data is abnormal or not according to the difference degree of the predicted data and the real-time data;
s042, detecting a steady fluctuation type anomaly, and evaluating real-time data by using historical data to calculate a reference value as an evaluation basis;
and S043, detecting irregular fluctuation type abnormity, constructing the characteristics of the performance data, calculating the abnormity degree by using an LOF algorithm, and further performing abnormity definition on the real-time data.
It should be noted that although the operations of the method of the present invention have been described in the above embodiments and the accompanying drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the operations shown must be performed, to achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Based on the aforementioned inventive concept, as shown in fig. 4, the present invention further provides a computer device 200, which includes a memory 210, a processor 220, and a computer program 230 stored on the memory 210 and operable on the processor 220, wherein the processor 220 implements the aforementioned adaptive intelligent network device performance index abnormality detection method when executing the computer program 230.
Based on the above inventive concept, the present invention further provides a computer readable storage medium storing a computer program for executing the foregoing adaptive intelligent network device performance index abnormality detection method.
The device and the method for detecting the performance index abnormity of the self-adaptive intelligent network equipment can automatically judge the fluctuation type of the performance index, do not need a large amount of human participation and have stronger mobility; firstly, a large amount of index data accumulation is not needed, and secondly, a large amount of manpower is not needed to be spent on searching and labeling abnormal data. Therefore, in an actual service scene, wider application conditions exist, and the applicability is stronger.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The limitation of the protection scope of the present invention is understood by those skilled in the art, and various modifications or changes which can be made by those skilled in the art without inventive efforts based on the technical solution of the present invention are still within the protection scope of the present invention.
Claims (16)
1. An adaptive intelligent network equipment performance index anomaly detection device is characterized by comprising:
the data access module is used for accessing the performance data acquired by the network equipment;
the data preprocessing module is used for processing the abnormal value of the data and standardizing the data processed by the abnormal value;
the wave type classification module judges whether the performance index data belongs to a periodic fluctuation type, a flat slow fluctuation type or an irregular fluctuation type;
an abnormality detection module for performing abnormality detection based on the waveform classification;
and the result feedback module is used for acquiring the judgment results of all the performance indexes, summarizing the judgment results and feeding back the judgment results to operation and maintenance personnel.
2. The apparatus of claim 1, wherein the data in the data access module includes but is not limited to device cpu utilization, memory utilization, and ingress and egress traffic.
3. The apparatus of claim 1, wherein the data access module keeps data acquisition time consistent.
4. The apparatus of claim 1, wherein the abnormal value processing in the data preprocessing module uses a box plot method to calculate the upper quartile, the lower quartile, and the upper and lower quartile differences of the performance index, the sum of the upper quartile and the 1.5 times of the quartile difference is used as an upper limit, the difference of the lower quartile and the 1.5 times of the quartile difference is used as a lower limit, the index value greater than the upper limit is adjusted to the upper limit, and the index value smaller than the lower limit is adjusted to the lower limit.
5. The apparatus of claim 1, wherein the normalization in the data preprocessing module is performed by mean variance normalization, and the mean and variance are calculated for each row of performance metric values, and the mean of each row is subtracted from the metric value of each row and then divided by the variance of the row.
6. The apparatus according to claim 1, wherein the waveform classification module trains a resnet deep network classification model using historical data, and comprises the following steps:
step one, collecting historical performance data of network equipment, and carrying out category marking according to a periodic fluctuation type, a smooth fluctuation type and an irregular fluctuation type;
step two, constructing a resnet depth network classification model with one-dimensional input data;
and step three, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
7. The apparatus of claim 1, wherein the anomaly detection module comprises:
the periodic fluctuation type anomaly detection module learns the characteristics of historical data and predicts future data by using an fbprophet model, and judges whether the periodic time sequence data is abnormal or not according to the difference degree of the predicted data and the real-time data;
the steady fluctuation type anomaly detection module is used for evaluating the real-time data by using the historical data to calculate the reference value as an evaluation basis;
the irregular fluctuation type anomaly detection module is used for calculating the anomaly degree by using an LOF algorithm through characteristic construction of performance data, and further performing anomaly definition on real-time data.
8. A self-adaptive intelligent network equipment performance index abnormity detection method is characterized by comprising the following steps:
s01, accessing performance data collected by network equipment;
s02, carrying out abnormal value processing on the data, and standardizing the data after the abnormal value processing;
s03, judging whether the performance index data belongs to a periodic fluctuation type, a smooth fluctuation type or an irregular fluctuation type;
s04, carrying out abnormity detection according to waveform classification;
and S05, acquiring the judgment results of the performance indexes, summarizing the judgment results, and feeding back the judgment results to operation and maintenance personnel.
9. The method of claim 8, wherein the data in S01 includes but is not limited to device cpu utilization, memory utilization, and ingress and egress traffic.
10. The method of claim 8, wherein the data collection time in S01 is kept consistent.
11. The method as claimed in claim 8, wherein the abnormal value processing in S02 is performed by using a box plot method to calculate the upper quartile, the lower quartile and the upper and lower quartile differences of the performance index, using the sum of the upper quartile and the 1.5 times of the quartile difference as an upper limit, using the difference of the lower quartile and the 1.5 times of the quartile difference as a lower limit, adjusting the index value greater than the upper limit to the upper limit, and adjusting the index value less than the lower limit to the lower limit.
12. The method of claim 8, wherein the normalization in S02 is performed by mean variance normalization, wherein the mean and variance are calculated for each row of performance metric values, and the mean of each row is subtracted from the metric value of each row and then divided by the variance of the row.
13. The method of claim 8, wherein the step S03 of training a resnet deep network classification model using historical data includes the following steps:
s031, collect the historical performance data of the network equipment, and carry on the classification label according to periodic fluctuation type, gentle fluctuation type and irregular fluctuation type;
s032, constructing a resnet depth network classification model with one-dimensional input data;
s033, training a resnet depth network classification model by using the marked historical performance data of the network equipment, obtaining an optimal model parameter after the optimizer performs random gradient reduction on the loss function for one time, and storing the optimal model parameter to obtain the trained resnet depth network classification model.
14. The method of claim 8, wherein the step S04 includes:
s041, detecting the periodic fluctuation type abnormality, learning the characteristics of historical data and predicting future data by using an fbprophet model, and judging whether the periodic time sequence data is abnormal or not according to the difference degree of the predicted data and the real-time data;
s042, detecting a steady fluctuation type anomaly, and evaluating real-time data by using historical data to calculate a reference value as an evaluation basis;
and S043, detecting irregular fluctuation type abnormity, constructing the characteristics of the performance data, calculating the abnormity degree by using an LOF algorithm, and further performing abnormity definition on the real-time data.
15. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1-4 when executing the computer program.
16. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for performing the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111135607.7A CN113918433A (en) | 2021-09-27 | 2021-09-27 | Adaptive intelligent network equipment performance index abnormity detection device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111135607.7A CN113918433A (en) | 2021-09-27 | 2021-09-27 | Adaptive intelligent network equipment performance index abnormity detection device and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113918433A true CN113918433A (en) | 2022-01-11 |
Family
ID=79236329
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111135607.7A Pending CN113918433A (en) | 2021-09-27 | 2021-09-27 | Adaptive intelligent network equipment performance index abnormity detection device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113918433A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978956A (en) * | 2022-04-11 | 2022-08-30 | 北京邮电大学 | Method and device for detecting abnormal performance mutation points of network equipment in smart city |
| CN115061605A (en) * | 2022-08-18 | 2022-09-16 | 深圳东昇射频技术有限公司 | Test curve marking method, device, equipment and storage medium |
-
2021
- 2021-09-27 CN CN202111135607.7A patent/CN113918433A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978956A (en) * | 2022-04-11 | 2022-08-30 | 北京邮电大学 | Method and device for detecting abnormal performance mutation points of network equipment in smart city |
| CN114978956B (en) * | 2022-04-11 | 2024-04-09 | 北京邮电大学 | Method and device for detecting abnormal mutation points of performance of intelligent city network equipment |
| CN115061605A (en) * | 2022-08-18 | 2022-09-16 | 深圳东昇射频技术有限公司 | Test curve marking method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10877863B2 (en) | Automatic prediction system for server failure and method of automatically predicting server failure | |
| CN111222290B (en) | Multi-parameter feature fusion-based method for predicting residual service life of large-scale equipment | |
| CN113918433A (en) | Adaptive intelligent network equipment performance index abnormity detection device and method | |
| CN110119787B (en) | Working condition detection method and equipment for rotary mechanical equipment | |
| CN111262750B (en) | Method and system for evaluating baseline model | |
| CN115412455A (en) | Server multi-performance index abnormity detection method and device based on time sequence | |
| CN113760670A (en) | Cable joint abnormity early warning method and device, electronic equipment and storage medium | |
| CN111538311A (en) | Flexible multi-state self-adaptive early warning method and device for mechanical equipment based on data mining | |
| CN115858794B (en) | Abnormal log data identification method for network operation safety monitoring | |
| CN115392782A (en) | Method and system for monitoring and diagnosing health state of process system of nuclear power plant | |
| CN111901134B (en) | Method and device for predicting network quality based on recurrent neural network model (RNN) | |
| CN115310650A (en) | Low-complexity high-precision time sequence multi-step prediction method and system | |
| CN114138601A (en) | Service alarm method, device, equipment and storage medium | |
| CN115705282A (en) | Cell network anomaly detection method and device and computer readable storage medium | |
| CN110995461B (en) | Network fault diagnosis method | |
| CN116030955B (en) | Medical equipment state monitoring method and related device based on Internet of things | |
| CN117010442A (en) | Equipment residual life prediction model training method, residual life prediction method and system | |
| US20230161653A1 (en) | Method of managing system health | |
| CN117048524A (en) | Method and device for detecting vehicle faults, vehicle and storage medium | |
| CN116415989A (en) | Gigabit potential customer prediction method, gigabit potential customer prediction device, computer equipment and storage medium | |
| CN112925806B (en) | Method, system, medium and equipment for extracting performance degradation characteristic parameters based on association rule | |
| CN109978038B (en) | Cluster abnormity judgment method and device | |
| CN114202110A (en) | Service fault prediction method and device based on RF-XGBOOST | |
| CN115858606A (en) | Method, device and equipment for detecting abnormity of time series data and storage medium | |
| CN113283157A (en) | System, method, terminal and medium for predicting life cycle of intelligent stamping press part |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |