CN117424640A - Operation and maintenance overhaul and risk prevention system and method based on associated data mining - Google Patents

Abstract

The invention relates to the technical field of operation, maintenance and repair and risk prevention of an OTN (optical transport network), in particular to an operation, maintenance and risk prevention system and method based on associated data mining. The invention adopts a deep learning method to predict faults, the troubleshooting process is automatic, alarm importance values of different alarm transactions are given through a method of combining K-means and a counter-propagation network, false alarm caused by OTN equipment parameter jitter is avoided, a chain alarm set is searched by utilizing a weighted Apriori algorithm, and warning is given to all potential risks of related equipment, so that advanced sensing, advanced processing and risk avoidance are realized.

Description

Operation and maintenance overhaul and risk prevention system and method based on associated data mining

Technical Field

The invention relates to the technical field of operation and maintenance overhaul and risk prevention of an OTN (optical transport network), in particular to an operation and maintenance overhaul and risk prevention system and method based on associated data mining.

Background

OTN networks are extremely complex in practical applications, and contain numerous devices and components, thus requiring operation and maintenance to ensure stability and reliability of the network. Meanwhile, as the scale and complexity of the network are continuously increased, the network risk is also increased. If a failure, attack or other problem occurs, it can have a serious impact on network performance and quality of service. Therefore, the risk of the OTN network can be previewed and precautionary measures are given by proper means, so that the service quality of the whole OTN network can be improved.

Aiming at the operation and maintenance overhaul and risk prevention of the OTN network, the prior art is divided into two types of traditional fault location and remote fault location. The traditional fault positioning means that faults or risks of the OTN are found and repaired in modes of manual inspection, field maintenance, equipment replacement and the like, but the method is time-consuming and labor-consuming, and large errors are caused by manual participation. The remote fault location refers to the remote monitoring and remote troubleshooting of faults and risks to a network through network monitoring, remote access and other modes, and the mode improves the efficiency of fault repair and risk discovery. However, remote fault location requires high network bandwidth and stability, and may have some delay or data loss. The method can only diagnose the equipment body with abnormal parameters, and can not check whether the related equipment has fault risks.

Disclosure of Invention

According to the invention, machine learning and deep learning are utilized to predict faults of OTN optical layer devices, the step of manual inspection and investigation in traditional fault positioning is replaced, the investigation cost is greatly reduced, an alarm transaction is extracted by a method combining time sequence segmentation and a time sliding window, an alarm importance value is given by a method combining K-means and a counter propagation network, an alarm notification rule can be set by a user according to an actual scene according to the alarm importance value, unnecessary overhaul is reduced, a chained alarm set is searched by utilizing a weighted Apriori algorithm, and a warning is given to nodes with potential risks of related equipment.

In order to solve the above-mentioned prior art problems, the present invention provides an operation and maintenance overhaul and risk prevention system based on associated data mining, comprising:

a data acquisition module;

an alarm transaction extraction module;

an OTN optical layer device fault prediction module;

the data acquisition module is used for data acquisition and preprocessing;

the alarm transaction extraction module is used for receiving the data acquired by the data acquisition module and dividing the data to obtain a plurality of alarm transactions;

and the OTN optical layer device fault prediction module receives the alarm transaction and predicts the alarm transaction to obtain the fault type.

Further, the data acquisition module comprises a data set making mode and a data pure acquisition mode;

the data set making mode has the following steps:

firstly, collecting all historical data in an OTN network; then the missing data are complemented according to the rules of the physical parameter data; then data compression is carried out, specifically, the data quantity is reduced by deleting redundant data; finally searching the input and output related indexes from the statistical angle;

the data pure acquisition mode is used for acquiring and preprocessing data.

Further, the data collected by the data collection module is time series data.

Further, the alarm transaction extraction module comprises the following steps:

firstly, dividing time sequence data into a plurality of subsequences with fixed length, wherein each subsequence represents data in a time period;

then, for each sub-sequence, a time sliding window method is used to divide the sub-sequence into a plurality of fixed-length windows, wherein the windows contain related physical parameters;

and finally, taking a window as an alarm transaction.

Further, the fault prediction module of the OTN optical layer device predicts faults by adopting an LSTM neural network.

Further, the system also comprises an alarm importance value calculation module, wherein the alarm importance value calculation module comprises the following steps:

firstly, carrying out clustering classification on the result of an OTN optical layer device fault prediction module by using K-means, wherein the obtained classification result is a pseudo tag, and training a counter-propagation network to obtain a pre-training model;

then extracting the clustering center and combining the knowledge graph or expert rules to obtain an alarm level;

and finally, fine tuning the pre-training model by taking the alarm level as a real label, thereby obtaining a back propagation network capable of calculating the alarm importance value.

Further, when the back propagation network calculates the importance value, the alarm importance value can be obtained by inputting an alarm transaction, and the alarm importance value is 0-100.

Further, the risk prompting system further comprises a risk prompting module, wherein the risk prompting module utilizes a weighted Apriori algorithm to find a chained alarm set, combines the chained alarm set with fault prediction, and gives a warning for potential risks of related equipment.

Further, the support and confidence in the Apriori algorithm are expressed as:

wherein, sigma represents the statistical calculation of the number of elements, sigma (xU y) represents the number of elements meeting the condition of (xU y);

wherein, extracting the element i meeting (xU y) can form:

wherein w is _i At t _n C in (a) is an alarm importance value, t _n Is a triplet (A, B, C), wherein A represents a device, B represents an alarm type, C represents an alarm importance value,representing the summation of the alarm importance values in the T element pointed to by the I element in I, which is the set of devices (I ₁ ,i ₂ ,…,i _j ) T is the alarm type and the alarm importance set (T ₁ ,t ₂ ,…,t _n ) N is the total number of elements in the set.

Further, an operation and maintenance overhaul and risk prevention method based on associated data mining comprises the following steps:

step1: the data acquisition module is enabled to work in a data acquisition mode, and data are acquired;

step2: training the LSTM neural network using the acquired data;

step3: the data acquisition module is enabled to work in a data pure acquisition mode;

step4: the alarm transaction extraction module works to obtain an alarm transaction;

step5: predicting the alarm transaction by using LSTM;

step6: clustering and classifying the prediction results of the LSTM to obtain pseudo labels and a pre-training model;

step7: and obtaining an alarm grade label through a predefined knowledge graph or expert rules, and fine-tuning the pre-training model to obtain a back propagation network capable of calculating an alarm importance value.

Step8: inputting the prediction result obtained in Step5 into a back propagation network to obtain a specific importance value;

step9: starting a risk prompting module to prompt the risk of equipment related to the fault;

step10: and when the node alarms, the alarm information is uploaded to a management center.

The beneficial effects of the present invention are embodied in that,

(1) The invention solves the problems of long maintenance time, high maintenance cost, large fault and risk investigation errors and the like caused by manual participation in the traditional fault positioning, predicts the faults of the OTN optical layer device according to the parameters such as the input optical power, the laser bias current and the like related to the optical transmission layer by adopting a machine learning and deep learning method, replaces the step of manual inspection and investigation in the traditional fault positioning, does not need manual participation in the whole fault investigation process, and greatly reduces the investigation cost.

(2) The invention extracts the alarm transaction by using a method combining time sequence segmentation and a time sliding window, gives an alarm importance value by using a method combining K-means and a counter-propagation network, and can set an alarm notification rule according to the alarm importance value by a user according to an actual scene so as to reduce unnecessary maintenance, avoid the phenomenon that the OTN network predicts faults by only using a machine learning or deep learning-based method in actual operation due to parameter jitter, and reduce the influence of false alarms on operation and maintenance management work.

(3) And searching a chained alarm set by using a weighted Apriori algorithm, and giving a warning to nodes with potential risks of the associated equipment by combining the chained alarm set with fault prediction, so that advanced sensing, advanced processing and risk avoidance are realized.

(4) The invention adopts the edge calculation mode to analyze and process the data, only uploads the processed characteristic data to the management center, and reduces the high dependence and high requirement on network bandwidth or network stability.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely in connection with the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

Example 1:

the invention discloses an operation and maintenance overhaul and risk prevention system and method based on associated data mining. The data acquisition module is responsible for data acquisition and preprocessing; the alarm transaction extraction module is responsible for extracting different types of alarm transactions by using a method of combining time sequence segmentation and a time sliding window; the OTN optical layer device fault prediction module is responsible for predicting the OTN optical layer device fault by using a machine learning or deep learning method, replaces the step of manual inspection and investigation in the traditional fault positioning, and greatly reduces the investigation cost.

In order to solve the problems of long maintenance time, high maintenance cost, large fault and risk investigation errors and the like caused by manual participation in the traditional fault positioning, because the optical layer fault is often closely related to the performance of a device, the physical parameters collected in daily operation can be quantified, and the parameters mainly comprise Input Optical Power (IOP), laser Bias Current (LBC), laser temperature bias (LTO), output Optical Power (OOP), environmental Temperature (ET), module internal temperature, optical signal to noise ratio and the like, in the embodiment, the physical parameters are collected through a data collection module, the data collection module is responsible for data collection and preprocessing, and the module is divided into two working modes: a data set making mode and a data pure acquisition mode. When the data set acquisition module works in a data set making mode, missing data is supplemented through data expansion, redundant data is removed, and the correlation between faults and parameters is obtained through a single index method. Specifically, first, all historical data in the OTN network is collected as much as possible; then, the missing data are complemented according to the rules of the physical parameter data; then data compression is carried out, in particular, the data quantity is reduced by deleting redundant data; and finally, searching the input and output related indexes from the statistical perspective, and reducing the pressure of a computer in the aspects of data processing and model training. In the case of using a single index, the relevant index of the fault state of the equipment, that is, the characteristic of a plurality of physical parameters, represents a fault corresponding to one type of equipment can be obtained through classification precision. Through the above steps, a useful data set can be established, wherein the input is performance data and the output is a future fault condition of the device. The data set can be used for training of a deep learning or machine learning model used by the OTN optical layer device failure prediction module. When the data set acquisition module works in a data pure acquisition mode, the module only acquires and performs necessary preprocessing on data.

The data acquired by the data acquisition module is transmitted to the alarm transaction extraction module, wherein the data acquired by the data acquisition module is a time sequence. Therefore, the alarm transaction extraction module firstly divides the time series data into a plurality of subsequences with fixed length, and each subsequence represents the data in one time period; then, for each sub-sequence, a time sliding window method can be used to divide the sub-sequence into a plurality of fixed-length windows, wherein the windows contain related physical parameters, and one window is used as an alarm transaction. The purpose of using a window as an alarm is to meet the computational performance of the edge computing device and facilitate feature fusion at a later stage to form an overall analysis of the time sub-sequence.

And transmitting the alarm transaction obtained from the alarm transaction extraction module to an OTN optical layer device fault prediction module, and predicting the alarm transaction by adopting the OTN optical layer device fault prediction module to obtain a fault type. The present embodiment predicts faults using LSTM, which is an improved network based on a recurrent neural network, with cells, input gates, forget gates, and output gates. The memory unit obtains input from the output of the last iteration of the LSTM; the input gate obtains a new input point from the outside and processes the new input data; the forget gate decides when to forget to output the result, it selects the best lag time for the input sequence; the output gate receives all the calculation results and generates an output for the LSTM neural network unit. By taking the alarm transaction as input, the fault type is available via the LSTM. Wherein the LSTM is trained using data acquired by the data acquisition module. Thus, LSTM has two distinct advantages over other deep learning algorithms: (1) the LSTM may capture temporal characteristics of the multi-source data; (2) During the information transfer process, the subsequent information retains the previous information.

In order to avoid the situation that parameter jitter of the OTN device causes false alarm, in this embodiment, the device further includes an alarm importance value calculation module, and the fault type and the device to which the alarm transaction belongs can be obtained through the prediction of the OTN optical layer device fault prediction module, but the same faults of the same device have various combinations in physical parameter expression. Therefore, the alarm importance value calculation module firstly uses K-means to carry out clustering classification on the results of the OTN optical layer device fault prediction module, the obtained classification results are pseudo tags, and the pseudo tags are utilized to train the counter-propagation network, so that a pre-training model is obtained. And then extracting the clustering center and combining a knowledge graph or expert rules (the knowledge graph or expert rules are common knowledge), so as to finally obtain an alarm level, wherein the alarm level is used for describing the importance of the alarm. The pre-training model can be finely tuned by taking the alarm level as a real label, so that a counter-propagation network capable of calculating the alarm importance value is obtained. When the importance value is calculated by using the back propagation network, K-means cluster classification is not needed any more, and the alarm transaction is directly input. The module may obtain alarm importance values of 0-100 and the user may set rules or thresholds to determine whether to notify the alarm.

In order to avoid potential risks of the related devices, in this embodiment, the system further includes a risk prompting module, and the risk prompting module is used to find a chain alarm set to give a warning to nodes with potential risks of the related devices. In particular, a linked alarm set is actually a set of consecutive risks, which in most cases are related to the same fault. Therefore, the cascading alarm set helps to group highly relevant risks, so that the risk of occurrence of faults of non-faulty body equipment is avoided conveniently in combination with fault prediction. The embodiment uses the weighted Apriori algorithm to find a chain alarm set, and the specific algorithm is as follows:

define a device set as i= (I) ₁ ,i ₂ ,…,i _j ) I.e. there are j devices. Alarm type and alarm importance set t= (T) ₁ ,t ₂ ,…,t _n ) N is the total number of elements in the set, t _n Is a triplet (A, B, C), wherein A represents a device, B represents an alarm type, and C represents an alarm importance value. If an element in set I points to K elements in T, then the point is defined as a K-point set. While there is a coincidence between the different sets of orientations, the support and confidence in the Apriori algorithm for the two sets of orientations x and y can be expressed as:

wherein, sigma represents the statistical calculation of the number of elements, for example, sigma (xU y) represents the number of elements meeting the condition of (xU y). At this time, element i satisfying (x u y) is extracted, and the constitution is:

wherein w is _i At t _n The value of the alarm importance is C in the formula (I),representing the summation of the alarm importance values in the T element pointed to by the I element in I. Subsequently, given a minimum threshold of support and confidence, solutions less than the threshold are removed, leaving a set of directives that satisfy the condition. Other equipment related to the fault can be obtained through the pointing set, so that staff is prompted to check.

The invention also discloses a method for operating, maintaining and repairing and risk preventing systems based on the associated data mining, which comprises the following steps:

step1: the data acquisition module is enabled to work in a data acquisition mode, and data are acquired;

step2: training the LSTM neural network using the acquired data;

step3: the data acquisition module is enabled to work in a data pure acquisition mode;

step4: the alarm transaction extraction module works to obtain an alarm transaction;

step5: predicting the alarm transaction by using LSTM;

step6: clustering and classifying the prediction results of the LSTM to obtain pseudo labels and a pre-training model;

step7: and obtaining an alarm grade label through a predefined knowledge graph or expert rules, and fine-tuning the pre-training model to obtain a back propagation network capable of calculating an alarm importance value.

Step8: inputting the prediction result obtained in Step5 into a back propagation network to obtain a specific importance value;

step9: starting a risk prompting module to prompt the risk of equipment related to the fault;

step10: and when the node alarms, the alarm information is uploaded to a management center.

It should be noted that the OTN network is composed of a plurality of nodes, and the present invention loads edge computing devices at each node, and is mainly responsible for processing Step1, step3, step4, step5, step8, step9, and Step10. Since model training requires a lot of computational effort, step2, step6, and Step7 are implemented in high performance servers. After the model is trained, the whole workflow can be completed on an edge computing platform. The data analysis of each node is carried out on the local edge computing equipment, and only the result is uploaded, so that the defect that the requirement on network bandwidth and the like is high due to the fact that a large amount of data are uploaded in a centralized manner in the middle-long-range fault positioning of the traditional method is avoided.

In the description of embodiments of the invention, a particular feature, structure, material, or characteristic may be combined in any suitable manner in one or more embodiments or examples.

In the description of the embodiments of the present invention, it is to be understood that "-" and "-" denote the same ranges of the two values, and the ranges include the endpoints. For example, "A-B" means a range greater than or equal to A and less than or equal to B. "A-B" means a range of greater than or equal to A and less than or equal to B.

In the description of embodiments of the present invention, the term "and/or" is merely an association relationship describing an association object, meaning that three relationships may exist, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. An operation and maintenance and risk prevention system based on associated data mining, comprising:

a data acquisition module;

an alarm transaction extraction module;

an OTN optical layer device fault prediction module;

the data acquisition module is used for data acquisition and preprocessing;

the alarm transaction extraction module is used for receiving the data acquired by the data acquisition module and dividing the data to obtain a plurality of alarm transactions;

and the OTN optical layer device fault prediction module receives the alarm transaction and predicts the alarm transaction to obtain the fault type.

2. The associated data mining-based operation and maintenance and risk prevention system of claim 1, wherein: the data acquisition module comprises a data set making mode and a data pure acquisition mode;

the data set making mode has the following steps:

firstly, collecting all historical data in an OTN network; then the missing data are complemented according to the rules of the physical parameter data; then data compression is carried out, specifically, the data quantity is reduced by deleting redundant data; finally searching the input and output related indexes from the statistical angle;

the data pure acquisition mode is used for acquiring and preprocessing data.

3. The associated data mining-based operation and maintenance and risk prevention system of claim 2, wherein: the data acquired by the data acquisition module are time sequence data.

4. The associated data mining-based operation and maintenance and risk prevention system of claim 3, wherein: the alarm transaction extraction module comprises the following steps:

firstly, dividing time sequence data into a plurality of subsequences with fixed length, wherein each subsequence represents data in a time period;

then, for each sub-sequence, a time sliding window method is used to divide the sub-sequence into a plurality of fixed-length windows, wherein the windows contain related physical parameters;

and finally, taking a window as an alarm transaction.

5. The associated data mining-based operation and maintenance and risk prevention system of claim 1, wherein: the OTN optical layer device fault prediction module predicts faults by adopting an LSTM neural network.

6. The associated data mining-based operation and maintenance and risk prevention system of claim 1, wherein: the alarm importance value calculation module comprises the following steps:

firstly, carrying out clustering classification on the result of an OTN optical layer device fault prediction module by using K-means, wherein the obtained classification result is a pseudo tag, and training a counter-propagation network to obtain a pre-training model;

then extracting the clustering center and combining the knowledge graph or expert rules to obtain an alarm level;

and finally, fine tuning the pre-training model by taking the alarm level as a real label, thereby obtaining a back propagation network capable of calculating the alarm importance value.

7. The associated data mining-based operation and maintenance and risk prevention system of claim 6, wherein: when the back propagation network calculates the importance value, the alarm importance value can be obtained by inputting an alarm transaction, and the alarm importance value is 0-100.

8. The associated data mining-based operation and maintenance and risk prevention system of claim 1, wherein: the risk prompting system further comprises a risk prompting module, wherein the risk prompting module searches a chained alarm set by utilizing a weighted Apriori algorithm, combines the chained alarm set with fault prediction, and gives a warning for potential risks of related equipment.

9. The associated data mining-based operation and maintenance and risk prevention system of claim 8, wherein: the support and confidence in the Apriori algorithm are expressed as:

wherein, sigma represents the statistical calculation of the number of elements, sigma (xU y) represents the number of elements meeting the condition of (xU y);

wherein, extracting the element i meeting (xU y) can form:

wherein w is _i At t _n C in (a) is an alarm importance value, t _n Is a triplet (A, B, C), wherein A represents a device, B represents an alarm type, C represents an alarm importance value,representing the summation of the alarm importance values in the T element pointed to by the I element in I, which is the set of devices (I ₁ ,i ₂ ,…,i _j ) T is the alarm type and the alarm importance set (T ₁ ,t ₂ ,…,t _n ) N is the total number of elements in the set.

10. An operation and maintenance overhaul and risk prevention method based on associated data mining is characterized by comprising the following steps of: the method comprises the following steps:

step1: the data acquisition module is enabled to work in a data acquisition mode, and data are acquired;

step2: training the LSTM neural network using the acquired data;

step3: the data acquisition module is enabled to work in a data pure acquisition mode;

step4: the alarm transaction extraction module works to obtain an alarm transaction;

step5: predicting the alarm transaction by using LSTM;

step6: clustering and classifying the prediction results of the LSTM to obtain pseudo labels and a pre-training model;

step7: and obtaining an alarm grade label through a predefined knowledge graph or expert rules, and fine-tuning the pre-training model to obtain a back propagation network capable of calculating an alarm importance value.

Step8: inputting the prediction result obtained in Step5 into a back propagation network to obtain a specific importance value;

step9: starting a risk prompting module to prompt the risk of equipment related to the fault;

step10: and when the node alarms, the alarm information is uploaded to a management center.