CN116545867A - Method and device for monitoring abnormal performance index of network element of communication network - Google Patents
Method and device for monitoring abnormal performance index of network element of communication network Download PDFInfo
- Publication number
- CN116545867A CN116545867A CN202310306155.7A CN202310306155A CN116545867A CN 116545867 A CN116545867 A CN 116545867A CN 202310306155 A CN202310306155 A CN 202310306155A CN 116545867 A CN116545867 A CN 116545867A
- Authority
- CN
- China
- Prior art keywords
- performance index
- historical data
- index
- scored
- performance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 38
- 238000004891 communication Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012544 monitoring process Methods 0.000 title claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 66
- 238000001514 detection method Methods 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 10
- 238000011156 evaluation Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 230000005856 abnormality Effects 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims 2
- 230000003044 adaptive effect Effects 0.000 abstract description 2
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 239000013585 weight reducing agent Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000007774 longterm Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/149—Network analysis or design for prediction of maintenance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a method and a device for monitoring the abnormal performance index of a network element of a communication network, wherein the method inputs the historical data of each performance index acquired from the network element in a historical acquisition period into a training model, invokes an algorithm matched with a current prediction scene from an algorithm library to train the historical data, outputs a plurality of prediction models to be scored to select index thresholds of the performance indexes corresponding to the optimal prediction model prediction from the prediction models, acquires real-time data of a first performance index acquired in the current acquisition period, carries out abnormal detection on the real-time data according to the first index thresholds of the first performance index, and if the real-time data is within the first threshold, the performance index corresponding to the real-time data is not abnormal, otherwise, the real-time data is abnormal. Therefore, the invention periodically collects the historical data of the performance index, so that the calculated index threshold is dynamic, the prediction accuracy is improved, the adaptive algorithm can be called for training according to the current prediction scene, the requirements of various service scenes are met, and the flexibility is improved.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method and an apparatus for monitoring performance indicators of network elements of a communications network.
Background
In the process of rapid development of communication service and massive construction of network engineering, the structure of a communication network is more and more complex, the function division of network elements is more and more refined, and the requirement on network quality is more and more enhanced, so that whether the performance index of the network element of the communication network is abnormal or not can be rapidly and accurately monitored to ensure the stability and reliability of network operation, and the network element is important for operation and maintenance work.
At present, most operators estimate a trend value aiming at a certain network element type index through inputting a certain type parameter and operation of a specific function prediction model, so that abnormal monitoring of the network element performance index of a communication network is realized, but the existing monitoring technology is limited to fitting prediction of a simple function, the input parameter and the prediction model are only operated aiming at a specific service function, so that the prediction model is too single, service limitation exists, other service scenes cannot be compatible, the input parameter is a static parameter, the trend value obtained through operation is a static threshold value, a large number of alarms are easily touched by mistake, and therefore abnormal monitoring accuracy of the network element performance index of the communication network is insufficient.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the invention provides a method and a device for monitoring abnormal performance indexes of a communication network element, which improve the accuracy and the flexibility of monitoring the abnormal performance indexes of the communication network element.
In order to solve the technical problems, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for monitoring abnormal performance indexes of network elements of a communication network, including:
acquiring historical data of each performance index acquired from a network element in a historical acquisition period, inputting the historical data into a training model, respectively training the historical data by calling an algorithm matched with a current prediction scene from an algorithm library, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored;
predicting an index threshold value of a corresponding performance index through the optimal prediction model;
acquiring real-time data of a first performance index acquired in a current acquisition period, performing anomaly detection on the real-time data according to a first index threshold of the first performance index, if the real-time data is within the first index threshold, no anomaly exists in the performance index corresponding to the real-time data, otherwise, the anomaly exists in the performance index corresponding to the real-time data.
The invention has the advantages that the historical data of each performance index in the network element is periodically collected, the data input into the training model is ensured to belong to dynamic data, thus ensuring the calculated index threshold of the performance index to be dynamic, improving the accuracy of judging whether the performance index is abnormal or not, reducing the false alarm rate, and training the historical data according to the current prediction scene and the adaptive algorithm thereof when training the historical data, the diversity of the algorithm in the algorithm library breaks the limitation on the service scene, meets the requirements of various service scenes and improves the flexibility.
Optionally, the step of inputting the historical data into a training model, calling an algorithm adapted to the current prediction scene from an algorithm library to train the historical data respectively, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored includes:
performing first label processing on the historical data according to a first label rule to generate historical data with a first label;
clustering the historical data with the first labels according to a clustering rule to generate performance index sets, and carrying out second label processing on each performance index set according to region aggregations to generate the performance index sets with the first labels and the second labels;
and for each performance index set, respectively training the performance index sets by calling an algorithm matched with the current prediction scene from an algorithm library, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored.
According to the description, when the historical data of the performance indexes are trained, the historical data are not subjected to single performance indexes, but are subjected to clustering treatment after subjected to first label treatment, so that a performance index set is formed, and a second label treatment is performed on the performance index set according to regional aggregation, and the performance index set with the first label and the second label is formed, so that the performance index set is trained, the calculated amount of training and the redundancy of a training model are reduced, and the training efficiency is further improved.
Optionally, the acquiring the real-time data of the first performance index acquired in the current acquisition period further includes:
judging whether an optimal prediction model corresponding to the first label and the second label of the first performance index exists, if so, acquiring a first index threshold value predicted by the corresponding optimal prediction model, otherwise, acquiring historical data of the first performance index acquired in a historical acquisition period for training, electing and predicting, and acquiring the first index threshold value of the first performance index.
According to the description, whether the optimal prediction model corresponding to the first label and the second label of the performance index exists can be judged, when the optimal prediction model exists, the first index threshold value predicted by the corresponding optimal prediction model is directly obtained, repeated operation is avoided, the prediction efficiency is improved, otherwise, the historical data of the first performance index is trained, elected and predicted, and the corresponding first index threshold value is obtained.
Optionally, the invoking the algorithm adapted to the current predicted scene from the algorithm library to train the historical data respectively includes:
establishing a parameter matrix based on parameters related to each algorithm in the algorithm library and preset parameter ranges;
and calling an algorithm matched with the current prediction scene from an algorithm library, and calling corresponding parameters from the parameter matrix according to the algorithm to train the historical data.
According to the description, a parameter matrix is established for the parameters related to each algorithm and the set parameter ranges in the algorithm library, so that when the algorithm is called to train the historical data, the corresponding parameters can be quickly called from the parameter matrix to train, and the training efficiency is improved.
Optionally, before the first label processing is performed on the history data according to the first label rule, the method includes:
judging whether null values exist in the historical data, if so, filling the null values through a preset filling rule to obtain filled historical data;
judging whether the filled historical data has abnormal values or not, and if so, replacing the abnormal values through a preset replacement rule to obtain the historical data without the abnormality.
According to the description, before the first label processing is performed on the historical data, not only the null value in the historical data is filled, but also the abnormal value is replaced, so that the trend and training of the historical data are prevented from being influenced by the null value and the abnormal value, and the training accuracy is improved.
Optionally, before inputting the history data into the training model, the method includes:
judging whether the historical data is structured data conforming to the parameter characteristics of the training model, if not, analyzing the historical data of unstructured data according to the parameter characteristics of the training model, and carrying out standardization processing on the analyzed historical data according to a preset rule to obtain the standardized historical data.
According to the description, the historical data input into the training model for training is structured data which accords with the parameter characteristics of the training model, and the historical data which does not accord with the parameter characteristics of the training model and is not structured data can be analyzed and normalized, so that the training effectiveness is ensured.
Optionally, the selecting the best prediction model from the plurality of prediction models to be scored includes:
taking the mean square error, the average absolute error and the average absolute value percentage error as evaluation indexes;
calculating an index score of each predictive model to be scored based on the evaluation index;
calculating the average score of the index scores, and calculating the comprehensive score of each prediction model to be scored according to the average score and the weight of each prediction model to be scored;
and taking the prediction model with the highest comprehensive score to be scored as an optimal prediction model.
According to the description, for the election of the optimal prediction model, the prediction model to be scored is scored based on diversified evaluation indexes, and the weight of each prediction model to be scored is taken into consideration, so that the election of the optimal prediction model is more comprehensive and accurate.
Optionally, the method further comprises:
acquiring error feedback of the performance index set in real time, reducing weights of optimal prediction models with the error feedback, and selecting a new optimal prediction model from a plurality of prediction models to be scored again;
and/or
Updating the training model according to a preset updating period to obtain a new optimal prediction model;
and/or
And acquiring the error feedback times of each performance index set in preset time in real time, and updating the training model of the corresponding performance index set to obtain a new optimal prediction model if the error feedback times exceed a time threshold.
According to the description, the error feedback of the performance index set is obtained in real time, namely the optimal prediction model with the error feedback is subjected to weight reduction according to each error feedback, so that a new optimal prediction model is reelected, the prediction accuracy is improved, the training model is updated according to the preset updating period and/or the error feedback times of the performance index set in the preset time, the new optimal prediction model is reelected, and the accuracy and the dynamics of the prediction result of the optimal prediction model are ensured.
Optionally, the acquiring the historical data of each performance index acquired in the network element in the historical acquisition period includes:
and if the historical data of each performance index fails to be acquired from the network element for the first time in the historical acquisition period, the historical data of each performance index is subjected to supplementary acquisition again, and if the number of supplementary acquisition exceeds a supplementary acquisition threshold value, the historical data is marked as a null value.
According to the description, in the history acquisition period, the failure of first acquiring the history data of each performance index of the network element can give a supplementary acquisition opportunity, so that special situations and accidental events are avoided, and the acquisition efficiency is improved.
In a second aspect, the present invention provides an apparatus for monitoring performance index anomalies of a communication network element, including a memory, a processor and a computer program stored in the memory and executable on the processor, where the processor implements a method for monitoring performance index anomalies of a communication network element according to the first aspect when executing the computer program.
The technical effects corresponding to the device for monitoring the abnormal performance index of the network element of the communication network provided in the second aspect refer to the related description of the method for monitoring the abnormal performance index of the network element of the communication network provided in the first aspect.
Drawings
Fig. 1 is a flowchart of a method for monitoring abnormal performance indexes of network elements of a communication network according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for monitoring abnormal performance indexes of network elements of a communication network according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating the re-election of a new best predictive model in accordance with an embodiment of the invention;
FIG. 4 is a schematic flow chart of training model update according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an apparatus for monitoring abnormal performance indexes of network elements of a communication network according to an embodiment of the present invention.
[ reference numerals description ]
1. A device for monitoring abnormal performance indexes of network elements of a communication network;
2. a processor;
3. a memory.
Detailed Description
In order that the above-described aspects may be better understood, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Example 1
Referring to fig. 1 to 4, the present invention provides a method for monitoring abnormal performance indexes of network elements of a communication network, comprising the steps of:
s1, acquiring historical data of each performance index acquired from a network element in a historical acquisition period, inputting the historical data into a training model, calling an algorithm matched with a current prediction scene from an algorithm library to train the historical data respectively, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored;
in this embodiment, the network element may be any device on the network, so that the data sources of each performance index collected from the network element include, but are not limited to, communication devices, operation and maintenance systems, monitoring systems, and the like, as shown in fig. 2, the historical data of each performance index in the network element are periodically collected, where the historical data is time-series data, including a time point and performance index information, the time point is based on the time point carried by the collected historical data, if the collected historical data does not include a specific time point, the collected time is used as the time point, the collected historical data includes performance index information, such as a performance index type, a performance index name, a performance index value, and the like, the historical data of each performance index collected in a historical collection period is used as an input parameter of a training model, and when the historical data is trained in the training model, an algorithm adapted to the current prediction scene is called from an algorithm library to train, and a plurality of to-be-scored prediction models are output, so that the best prediction model is selected from the plurality of to-be-scored prediction models.
The range of the collection period is generally set between 5 and 15 minutes, and the specific collection period can be set according to the actual conditions of the network element type and the performance index type, for example, the network element load of the data source is higher, frequent data collection cannot be processed, the importance degree of the performance index type and the performance index name contained in the collected data is lower, then the collection period can be set to be one collection every 15 minutes, and preferably, the collection period can be set to be 5 minutes to be one collection.
In the present embodiment, step S1 includes a data processing step S11, a model training step S12, and a model optimizing step S13.
Specifically, the data processing step S11 specifically includes:
s111, collecting the historical data of each performance index from the network element according to the historical collection period, if the historical data of each performance index is firstly collected from the network element in the historical collection period fails, carrying out supplementary collection on the historical data of each performance index again, and if the number of supplementary collection exceeds a supplementary collection threshold, marking the historical data as a null value.
In this embodiment, in the history acquisition period, when the first time of acquiring the history data of each performance index from the network element fails, the history data of each performance index may be subjected to the supplementary acquisition again, the number of supplementary acquisitions does not exceed a preset supplementary threshold, if the number of supplementary acquisitions exceeds the supplementary threshold, the history data is still marked as a null value, the setting of the supplementary threshold may be set according to the acquisition period, the supplementary acquisition frequency is not higher than the acquisition period, if the acquisition period is 5 minutes once, the first time of supplementary acquisitions may be set for one minute and the supplementary acquisition threshold is 4 times, that is, when the number of supplementary acquisitions exceeds 4 times, the history data is marked as a null value.
S112, acquiring historical data of each performance index acquired from the network element in a historical acquisition period.
S113, judging whether the historical data are structured data conforming to the parameter characteristics of the training model, if not, analyzing the historical data of unstructured data according to the parameter characteristics of the training model, and carrying out standardization processing on the analyzed historical data according to a preset rule to obtain the standardized historical data.
In this embodiment, as shown in fig. 2, the history data input into the training model is structured data to conform to the parameter characteristics of the training model, and the history data of each performance index collected from the network element has various formats, such as xml format, json format, log and other semi-structured and unstructured data, so that the history data needs to be parsed according to the parameter characteristics of the training model, the key fields of the history data are obtained, and the parsed history data is normalized according to preset rules, such as preset rules according to the time point, the type of the performance index, the name of the performance index and the style of the performance index value, so as to obtain normalized history data, and the preset rules of the normalization processing can be set and adjusted according to actual situations.
S114, judging whether null values exist in the historical data, if so, filling the null values through a preset filling rule to obtain filled historical data;
in this embodiment, as shown in fig. 2, for the history data after the specification in step S113, it is determined whether there is a null value, if there is a null value, the null value is filled by a preset filling rule, for example, the null value is filled by taking the average value, the median, the last time value, the lagrangian difference value, and the like of the history data, where the specific filling rule can be set and adjusted according to the actual situation.
S115, judging whether the filled historical data has abnormal values, and if so, replacing the abnormal values through a preset replacement rule to obtain the historical data without the abnormality.
In this embodiment, as shown in fig. 2, whether an abnormal value exists in the filled historical data is determined, the determination of the abnormal value may be performed by a 3 sigma principle, a box graph, or the like, if the abnormal value exists, the abnormal value may be replaced by a preset replacement rule, for example, a normal value adjacent to the abnormal value is replaced, and the specific abnormal value determination method and the replacement rule may be set according to the actual situation.
S116, performing first label processing on the historical data according to a first label rule to generate historical data with a first label;
in this embodiment, as shown in fig. 2, the history data input into the training model is subjected to a first label processing according to a first label rule, where the first label rule is to perform the first label processing on the history data according to an index type and an index name of a performance index, so as to generate the history data with the first label, where the first label rule is preset, and a specific first label rule may be adjusted according to an actual situation.
S117, clustering the historical data with the first labels according to a clustering rule to generate performance index sets, and performing second label processing on each performance index set according to region aggregations to generate the performance index sets with the first labels and the second labels;
in this embodiment, as shown in fig. 2, the history data after the first label processing in step S116 may be clustered according to a clustering rule, where the clustering rule is set to divide the performance indexes with high relevance into the same class through the performance index values, so as to generate a performance index set, a specific clustering rule may be set and adjusted according to actual situations, and each performance index set may be subjected to a second label processing according to regional aggregation property, for example, each performance index set is subjected to a digital label, so as to form a performance index set 1, a performance index set 2, a performance index set 3, and the like, and a specific second label processing manner may be set and adjusted according to actual situations, so that the final performance index set is provided with the first label and the second label.
Specifically, the model training step S12 specifically includes:
and for each performance index set, respectively training the performance index sets by calling an algorithm matched with the current prediction scene from an algorithm library, and outputting a plurality of prediction models to be scored.
In this embodiment, as shown in fig. 2, when an algorithm adapted to a current predicted scene is called from an algorithm library to perform training, instead of training for a single performance index, the performance index is trained in a performance index set manner, and a plurality of prediction models to be scored corresponding to the performance index set are output.
At this time, the invoking the algorithm adapted to the current predicted scene from the algorithm library in step S12 to train the historical data includes:
s121, establishing a parameter matrix based on parameters related to each algorithm in the algorithm library and preset parameter ranges;
in this embodiment, the algorithms in the algorithm library include, but are not limited to: the XGBoost algorithm, the S-H-ESD algorithm, the ARIMA algorithm, the RNN-LSTM algorithm and the Prophet algorithm are used for establishing parameter matrixes of parameters related to each algorithm and parameter ranges set for each parameter, for example, the parameters related to the RNN-LSTM algorithm comprise a hidden layer number Hidden Layer Size and a Learning rate, the larger the hidden layer number Hidden Layer Size is, the more complicated the neural network is, the better the prediction performance is, but the longer the training time is, so that the hidden layer number Hidden Layer Size of the RNN-LSTM algorithm is set between 10 and 30, the Learning rate is set between 0.001 and 0.1, and the parameter ranges of the specific algorithm can be adjusted and set according to actual conditions.
S122, invoking an algorithm matched with the current prediction scene from an algorithm library, and invoking corresponding parameters from the parameter matrix according to the algorithm to train the historical data.
In this embodiment, as shown in fig. 2, when the algorithm is called from the algorithm library to train the historical data, the algorithm is called according to the current prediction scene, if the current prediction scene is a short-term prediction scene, the algorithm adapted to the short-term prediction scene is called, if the current prediction scene is a long-term prediction scene, the algorithm adapted to the long-term prediction scene is called, after the algorithm is selected, the parameter matrix established in step S121 is called to train the historical data, and there may be a plurality of algorithms adapted to the same prediction scene, for example, the algorithm adapted to the short-term prediction scene includes XGBoost algorithm, S-H-ESD algorithm, ARIMA algorithm, and the algorithm adapted to the long-term prediction scene includes propset algorithm, RNN-LSTM algorithm, and the like.
Specifically, the model optimization step S13 specifically includes:
s131, taking the mean square error, the average absolute error and the average absolute value percentage error as evaluation indexes;
s132, calculating index scores of each predictive model to be scored based on the evaluation indexes;
in this embodiment, after the mean square error, the average absolute error, and the average absolute percentage error are used as the evaluation indexes to calculate the error value of each prediction model to be scored, the error values calculated by different evaluation indexes are converted into corresponding index scores by the following formula:
wherein score i Index score representing the i-th predictive model to be scored, e i Representing the error value of the i-th prediction model to be scored, min (e) represents the minimum error value of the prediction model to be scored, max (e) represents the maximum error value of the prediction model to be scored, s represents the total score, and s=100 if s is a percentile.
In a specific embodiment, the index score of the 1 st prediction model to be scored is calculated by mean square error, the error value of the first prediction model to be scored is 0.48, the minimum error value is 0.06, the maximum error value is 0.5, s is a percentage, the index score of the first prediction model to be scored= (1- (0.48-0.06)/(0.5-0.06)) =5, i.e. under mean square error, the index score of the first prediction model to be scored is 5, the index score of the 1 st prediction model to be scored is calculated by mean square error, the error value of the first prediction model to be scored is 0.4, the minimum error value is 0.03, the maximum error value is 0.48, s is a percentage, the index score of the first prediction model to be scored= (1- (0.4-0.03)/(0.03)) =18, i.e. under mean absolute error, the index score of the first prediction model to be scored is 18, the average absolute error value is calculated as 1 st error score of 0.04, the first prediction model to be scored is 0.04, the error value of the first prediction model to be scored is 0.45, and the error value of the first prediction model to be scored is 0.04, the maximum error value of 0.45 = 0.45.
S133, calculating the average score of the index scores, and calculating the comprehensive score of each prediction model to be scored according to the average score and the weight of each prediction model to be scored;
and S134, taking the prediction model with the highest comprehensive score to be scored as an optimal prediction model.
In this embodiment, the average score is calculated according to the index score of the corresponding prediction model to be evaluated calculated in step S132, and each prediction model to be scored is provided with a default weight, the comprehensive score of the prediction model to be scored is calculated according to the weight corresponding to the prediction model to be scored and the average score according to the following formula, and the prediction model to be scored with the highest comprehensive score is used as the optimal prediction model:
wherein the integration_score i And (3) representing the comprehensive score of the ith predictive model to be scored, n represents n evaluation indexes, and w represents the weight of the predictive model to be scored.
Wherein the initial weight of each prediction model to be scored is set to 1.
S2, predicting an index threshold value of a corresponding performance index through the optimal prediction model;
in this embodiment, as shown in fig. 2, the index threshold of the corresponding performance index is predicted by the optimal prediction model.
S3, acquiring real-time data of a first performance index acquired in a current acquisition period, performing anomaly detection on the real-time data according to a first index threshold of the first performance index, if the real-time data is within the first index threshold, no anomaly exists in the network element performance index corresponding to the real-time data, and otherwise, the anomaly exists in the network element performance index corresponding to the real-time data.
In this embodiment, as shown in fig. 2, real-time data of a first performance index acquired in a current acquisition period is acquired, anomaly detection is performed on the real-time data according to a first index threshold of the first performance index, if the real-time data is within the first index threshold, no anomaly exists in the network element performance index corresponding to the real-time data, and if the real-time data is not within the first index threshold, no anomaly exists in the network element performance index corresponding to the real-time data.
At this time, the step S3 further includes, after acquiring the real-time data of the first performance index acquired in the current acquisition period:
s31, judging whether an optimal prediction model corresponding to the first label and the second label of the first performance index exists, if so, acquiring a first index threshold value predicted by the corresponding optimal prediction model, otherwise, acquiring historical data of the first performance index acquired in a historical acquisition period, training, electing and predicting, and acquiring the first index threshold value of the first performance index.
In this embodiment, as shown in fig. 2, when acquiring the real-time data of the first performance index acquired in the current acquisition period, whether the optimal prediction model exists or not is first determined according to the first tag and the second tag of the first performance index, if so, the first index threshold predicted by the corresponding optimal prediction model is acquired, otherwise, the historical data of the first performance index acquired in the historical acquisition period is acquired for training, electing and predicting, so as to obtain the first index threshold of the first performance index.
S4, updating the model.
In this embodiment, the triggering principle of the model update includes:
(1) And acquiring error feedback of the performance index set in real time, performing weight reduction on the optimal prediction model subjected to the error feedback, and re-selecting a new optimal prediction model from a plurality of prediction models to be scored.
In this embodiment, as shown in fig. 3, error feedback of the performance index set is obtained in real time, that is, when the performance index set has error feedback, the best prediction model with error feedback is subjected to weight reduction, for example, the current weight of the best prediction model is 1, and when error feedback is obtained once, the weight reduction of the best prediction model with error feedback is reduced by 10%, that is, the weight of the best prediction model after weight reduction is 0.9, and at this time, a new best prediction model is newly selected from a plurality of prediction models to be scored.
(2) And updating the training model according to a preset updating period to obtain a new optimal prediction model.
In this embodiment, as shown in fig. 4, the training model is updated according to a preset updating period, and the historical data of each performance index in the network element of the historical acquisition period is re-acquired, and is input into the updated training model to be re-trained, so as to output a plurality of new prediction models to be scored, and then a new optimal prediction model is selected from the new prediction models, if the preset updating period is set to 7 days, the training model is updated once every 7 days from the day when the training model is formed, and the updating period can be set according to practical situations, such as 15 days, 1 month, 3 months, and the like.
(3) And acquiring the error feedback times of each performance index set in preset time in real time, and updating the training model of the corresponding performance index set to obtain a new optimal prediction model if the error feedback times exceed a time threshold.
In this embodiment, as shown in fig. 4, the error feedback times of each performance index set in the preset time are obtained in real time, when the error feedback times exceed the time threshold, the training model of the corresponding performance index set is updated, if the time threshold is set to be 10, that is, the error feedback times are greater than 10, the training model of the corresponding performance index set is updated, the historical data of each performance index in the historical acquisition period network element is re-obtained, the historical data are input into the updated training model for re-training, a new plurality of prediction models to be scored are output, a new optimal prediction model is re-selected from the new optimal prediction models, and the preset time and the time threshold can be set according to actual conditions.
(4) And manually triggering.
In this embodiment, the update of the training model may also be triggered manually.
It should be noted that the model update in this embodiment includes the four triggering principles described above, and in other embodiments, the triggering principles of the model update may include one, two, or three of them.
Example two
Referring to fig. 3, an apparatus 1 for monitoring abnormal performance indexes of network elements of a communication network includes a memory 3, a processor 2, and a computer program stored in the memory 3 and capable of running on the processor 2, wherein the processor 2 implements the steps in the first embodiment when executing the computer program.
Since the system/device described in the foregoing embodiments of the present invention is a system/device used for implementing the method of the foregoing embodiments of the present invention, those skilled in the art will be able to understand the specific structure and modification of the system/device based on the method of the foregoing embodiments of the present invention, and thus will not be described in detail herein. All systems/devices used in the methods of the above embodiments of the present invention are within the scope of the present invention.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the terms first, second, third, etc. are for convenience of description only and do not denote any order. These terms may be understood as part of the component name.
Furthermore, it should be noted that in the description of the present specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with the embodiment or example being included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art upon learning the basic inventive concepts. Therefore, the appended claims should be construed to include preferred embodiments and all such variations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, the present invention should also include such modifications and variations provided that they come within the scope of the following claims and their equivalents.
Claims (10)
1. A method for monitoring performance indicators of network elements of a communication network, comprising:
acquiring historical data of each performance index acquired from a network element in a historical acquisition period, inputting the historical data into a training model, respectively training the historical data by calling an algorithm matched with a current prediction scene from an algorithm library, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored;
predicting an index threshold value of a corresponding performance index through the optimal prediction model;
acquiring real-time data of a first performance index acquired in a current acquisition period, performing anomaly detection on the real-time data according to a first index threshold of the first performance index, if the real-time data is within the first index threshold, no anomaly exists in the performance index corresponding to the real-time data, otherwise, the anomaly exists in the performance index corresponding to the real-time data.
2. The method for monitoring abnormal performance indexes of network elements of a communication network according to claim 1, wherein the step of inputting the historical data into a training model, calling an algorithm adapted to a current prediction scene from an algorithm library to train the historical data respectively, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored comprises:
performing first label processing on the historical data according to a first label rule to generate historical data with a first label;
clustering the historical data with the first labels according to a clustering rule to generate performance index sets, and carrying out second label processing on each performance index set according to region aggregations to generate the performance index sets with the first labels and the second labels;
and for each performance index set, respectively training the performance index sets by calling an algorithm matched with the current prediction scene from an algorithm library, outputting a plurality of prediction models to be scored, and selecting an optimal prediction model from the plurality of prediction models to be scored.
3. The method for monitoring abnormal performance indexes of network elements of a communication network according to claim 2, wherein the step of acquiring the real-time data of the first performance index acquired in the current acquisition period further comprises:
judging whether an optimal prediction model corresponding to the first label and the second label of the first performance index exists, if so, acquiring a first index threshold value predicted by the corresponding optimal prediction model, otherwise, acquiring historical data of the first performance index acquired in a historical acquisition period, training, electing and predicting, and acquiring the first index threshold value of the first performance index.
4. The method for monitoring performance index anomalies of a network element of a communication network as recited in claim 1, wherein invoking an algorithm adapted to a current predicted scenario from the algorithm library to train the historical data, respectively, comprises:
establishing a parameter matrix based on parameters related to each algorithm in the algorithm library and preset parameter ranges;
and calling an algorithm matched with the current prediction scene from an algorithm library, and calling corresponding parameters from the parameter matrix according to the algorithm to train the historical data.
5. A method for monitoring performance anomalies in a network element of a communication network, as recited in claim 2, wherein prior to first labeling the historical data according to a first labeling rule, comprises:
judging whether null values exist in the historical data, if so, filling the null values through a preset filling rule to obtain filled historical data;
judging whether the filled historical data has abnormal values or not, and if so, replacing the abnormal values through a preset replacement rule to obtain the historical data without the abnormality.
6. A method of monitoring anomalies in performance indicators of a communication network element, as set forth in claim 1, wherein prior to inputting the historical data into a training model, comprises:
judging whether the historical data are structured data conforming to the parameter characteristics of the training model, if not, analyzing the historical data of unstructured data according to the parameter characteristics of the training model, and carrying out standardization processing on the analyzed historical data according to a preset rule to obtain the standardized historical data.
7. The method for monitoring abnormal performance indexes of network elements of a communication network according to claim 1, wherein the selecting the best prediction model from a plurality of prediction models to be scored comprises:
taking the mean square error, the average absolute error and the average absolute value percentage error as evaluation indexes;
calculating an index score of each predictive model to be scored based on the evaluation index;
calculating the average score of the index scores, and calculating the comprehensive score of each prediction model to be scored according to the average score and the weight of each prediction model to be scored;
and taking the prediction model with the highest comprehensive score to be scored as an optimal prediction model.
8. A method of monitoring performance indicators of a communication network element as recited in claim 2, further comprising:
acquiring error feedback of the performance index set in real time, reducing weights of optimal prediction models with the error feedback, and selecting a new optimal prediction model from a plurality of prediction models to be scored again;
and/or
Updating the training model according to a preset updating period to obtain a new optimal prediction model;
and/or
And acquiring the error feedback times of each performance index set in preset time in real time, and updating the training model of the corresponding performance index set to obtain a new optimal prediction model if the error feedback times exceed a time threshold.
9. A method of monitoring performance indicators of a communication network element as claimed in claim 1, wherein said acquiring historical data of each performance indicator collected in the network element preceded by a historical acquisition period comprises:
and if the historical data of each performance index fails to be acquired from the network element for the first time in the historical acquisition period, the historical data of each performance index is subjected to supplementary acquisition again, and if the number of supplementary acquisition exceeds a supplementary acquisition threshold value, the historical data is marked as a null value.
10. An apparatus for monitoring a communication network element for anomalies in performance indicators, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 9 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310306155.7A CN116545867A (en) | 2023-03-27 | 2023-03-27 | Method and device for monitoring abnormal performance index of network element of communication network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310306155.7A CN116545867A (en) | 2023-03-27 | 2023-03-27 | Method and device for monitoring abnormal performance index of network element of communication network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116545867A true CN116545867A (en) | 2023-08-04 |
Family
ID=87456744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310306155.7A Pending CN116545867A (en) | 2023-03-27 | 2023-03-27 | Method and device for monitoring abnormal performance index of network element of communication network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116545867A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056584A (en) * | 2023-10-08 | 2023-11-14 | 杭州海康威视数字技术股份有限公司 | Information system abnormal change monitoring method and equipment based on dynamic similarity threshold |
-
2023
- 2023-03-27 CN CN202310306155.7A patent/CN116545867A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056584A (en) * | 2023-10-08 | 2023-11-14 | 杭州海康威视数字技术股份有限公司 | Information system abnormal change monitoring method and equipment based on dynamic similarity threshold |
| CN117056584B (en) * | 2023-10-08 | 2024-01-16 | 杭州海康威视数字技术股份有限公司 | Information system abnormal change monitoring method and equipment based on dynamic similarity threshold |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111475804B (en) | Alarm prediction method and system | |
| CN111860897B (en) | Abnormality detection method, abnormality detection device, abnormality detection equipment and computer-readable storage medium | |
| CN110110870B (en) | Intelligent equipment fault monitoring method based on event map technology | |
| CN108199795B (en) | A kind of monitoring method and device of equipment state | |
| CN113556258B (en) | Anomaly detection method and device | |
| CN111506478A (en) | Method for realizing alarm management control based on artificial intelligence | |
| CN113361663A (en) | Artificial intelligence-based power grid event diagnosis autonomous learning method and system | |
| CN108470022A (en) | A kind of intelligent work order quality detecting method based on operation management | |
| CN112199805B (en) | Power transmission line hidden danger identification model evaluation method and device | |
| CN116545867A (en) | Method and device for monitoring abnormal performance index of network element of communication network | |
| CN115766068A (en) | Network security event grade classification method, device, equipment and medium | |
| CN114090393B (en) | Method, device and equipment for determining alarm level | |
| CN116821141A (en) | Data updating method, fault diagnosis method, electronic device, and storage medium | |
| CN111813644A (en) | System performance evaluation method and device, electronic equipment and computer readable medium | |
| CN115858606A (en) | Method, device and equipment for detecting abnormity of time series data and storage medium | |
| CN117216713A (en) | Fault delimiting method, device, electronic equipment and storage medium | |
| CN112699048A (en) | Program fault processing method, device and equipment based on artificial intelligence and storage medium | |
| CN109635008B (en) | Equipment fault detection method based on machine learning | |
| CN116756659A (en) | Intelligent operation and maintenance management method, device, equipment and storage medium | |
| CN111638989B (en) | Fault diagnosis method, device, storage medium and equipment | |
| CN116522213A (en) | Service state level classification and classification model training method and electronic equipment | |
| CN118228130B (en) | Monitoring method, system and storage medium based on equipment health state | |
| CN114064441B (en) | Method, device and equipment for detecting log abnormity | |
| CN117076230B (en) | Early warning method and system for computer log | |
| CN117992878A (en) | Abnormality detection method and system applied to power grid dispatching technical support system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |