CN117041009A - Method and device for generating rule database for alarm analysis - Google Patents
Method and device for generating rule database for alarm analysis Download PDFInfo
- Publication number
- CN117041009A CN117041009A CN202310935966.3A CN202310935966A CN117041009A CN 117041009 A CN117041009 A CN 117041009A CN 202310935966 A CN202310935966 A CN 202310935966A CN 117041009 A CN117041009 A CN 117041009A
- Authority
- CN
- China
- Prior art keywords
- alarm
- alarm information
- information pair
- data
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 37
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 11
- 238000007621 cluster analysis Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 abstract description 18
- 238000012216 screening Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005065 mining Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000012098 association analyses Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/024—Standardisation; Integration using relational databases for representation of network management data, e.g. managing via structured query language [SQL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0604—Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application provides a method and a device for generating a rule database for alarm analysis, and relates to the technical field of network management. The method comprises the following steps: acquiring a plurality of pieces of alarm information; carrying out multidimensional clustering analysis on a plurality of pieces of alarm information to obtain clustering results; counting the support degree and the confidence coefficient of any first alarm information pair according to the clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence coefficient, wherein the first alarm information pair comprises two ordered alarm information in the clustering result; and acquiring the derived intensity of the alarm information pair according to the confidence coefficient, acquiring alarm rules based on the derived intensity and a plurality of alarm incidence matrixes, and generating a rule database which is used for alarm analysis. The application can improve the fault processing efficiency and flexibility, and can realize effective rule screening and improve the real-time performance and accuracy of alarm analysis processing when a large number of fault alarms occur.
Description
Technical Field
The application relates to the technical field of network management, in particular to a method and a device for generating a rule database for alarm analysis.
Background
In the related art, with the operation of the system, the basic platform and the service system generate a large amount of alarm information, and the information has huge data volume and a large number of sudden faults, when equipment or service breaks down and gives rise to an alarm, the equipment associated with the equipment or service can also give rise to a corresponding fault, and a large amount of alarm information is generated in a short time. Because a fault often causes a plurality of alarm events, equipment related to the fault and a business process can send out related alarm information, and meanwhile, the alarm information caused by a plurality of faults can be overlapped, so that effective alarm information is confused, and the difficulty in identifying the fault during fault processing is increased.
Therefore, how to improve the fault processing efficiency and flexibility, when a large number of fault alarms occur, the screening of effective rules is realized, and the instantaneity and the accuracy of alarm analysis processing are improved.
Disclosure of Invention
The present application aims to solve at least one of the technical problems in the related art to some extent. To this end, an object of the application is to propose a method for generating a rule database for alarm analysis.
A second object of the present application is to propose a rule database generation device for alarm analysis.
A third object of the present application is to propose an electronic device.
A fourth object of the present application is to propose a non-transitory computer readable storage medium.
A fifth object of the application is to propose a computer programme product.
To achieve the above object, an embodiment of a first aspect of the present application provides a method for generating a rule database for alarm analysis, including:
acquiring a plurality of pieces of alarm information;
carrying out multidimensional clustering analysis on a plurality of pieces of alarm information to obtain clustering results;
counting the support degree and the confidence coefficient of any first alarm information pair according to the clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence coefficient, wherein the first alarm information pair comprises two ordered alarm information in the clustering result;
and acquiring the derived intensity of the alarm information pair according to the confidence coefficient, acquiring alarm rules based on the derived intensity and a plurality of alarm incidence matrixes, and generating a rule database which is used for alarm analysis.
The application can improve the fault processing efficiency and flexibility, and can realize effective rule screening and improve the real-time performance and accuracy of alarm analysis processing when a large number of fault alarms occur.
To achieve the above object, an embodiment of a second aspect of the present application provides a rule database generating device for alarm analysis, including:
the first acquisition module is used for acquiring a plurality of pieces of alarm information;
the second acquisition module is used for carrying out multidimensional clustering analysis on the plurality of pieces of alarm information to acquire clustering results;
the first generation module is used for counting the support degree and the confidence coefficient of any first alarm information pair according to any clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence coefficient, wherein the first alarm information pair comprises two ordered alarm information in the clustering result;
the second generation module is used for acquiring the derived intensity of the alarm information pair according to the confidence coefficient, acquiring alarm rules based on the derived intensity and the alarm correlation matrixes, and generating a rule database which is used for alarm analysis.
To achieve the above object, an embodiment of a third aspect of the present application provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating a rules database for alert analysis provided in the embodiments of the first aspect of the present application.
To achieve the above object, an embodiment of a fourth aspect of the present application provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to execute a method for generating a rule database for alarm analysis provided in the embodiment according to the first aspect of the present application.
To achieve the above object, an embodiment of a fifth aspect of the present application proposes a computer program product comprising a computer program which, when executed by a processor, implements a method for generating a rule database for alarm analysis provided in an embodiment of the first aspect of the present application.
Drawings
FIG. 1 is a flow chart of a method of generating a rules database for alert analysis according to one embodiment of the present application;
FIG. 2 is a flow chart of a method of generating a rules database for alert analysis according to one embodiment of the application;
FIG. 3 is a flow chart of a method of generating a rules database for alert analysis according to one embodiment of the present application;
FIG. 4 is a schematic diagram of a method of generating a rules database for alert analysis according to one embodiment of the present application;
FIG. 5 is a schematic diagram of a method of generating a rules database for alert analysis according to one embodiment of the present application;
FIG. 6 is a block diagram of a rule database generation apparatus for alert analysis according to one embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The method and apparatus for generating a rule database for alarm analysis according to the embodiments of the present application are described below with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method of generating a rule database for alert analysis according to one embodiment of the present application, as shown in FIG. 1, the method comprising the steps of:
s101, acquiring a plurality of pieces of alarm information.
In some implementations, the plurality of pieces of alert information are obtained based on historical alert data for the network management system, traffic information for the traffic system, network topology data, and alert level information.
In some implementations, in order to improve accuracy of alarm analysis, after historical alarm data is acquired, the historical alarm data is filtered to remove data missing from a key field of first data, and multiple pieces of alarm information are generated by combining service information, network topology data and alarm level information, wherein the key field of the first data is a key field of the alarm data.
For example, the first data key field may include important information such as an alarm ID, an alarm time, and an alarm content, and in the embodiment of the present application, the alarm information is extracted from the historical alarm data, and if the first data key field is missing in the alarm information, the alarm information is described as abnormal, and the alarm information is removed.
Further, data such as network topology, service information of a service system, alarm level information and the like are obtained. Alternatively, the network topology information may include network device information (name, model, configuration, etc.), link information (start-stop device, bandwidth, protocol, etc.), user access points, network partition area information, important network node and core device information, etc. The service information may include service type (CRM application, database, big data, etc.), service priority information, department information to which the service belongs, requirements of the service on network metrics (bandwidth, latency, etc.), network resource information on which the service depends, etc. The alarm hierarchy information may include alarm severity level (general, important, severe, etc.), alarm impact range (full network, local area, etc.), whether core service alarm identification is affected, alarm type (network failure, configuration error, performance alarm, etc.).
S102, carrying out multidimensional clustering analysis on the plurality of pieces of alarm information to obtain clustering results.
Extracting multidimensional second data key fields from the plurality of pieces of alarm information, wherein the second data key fields are key fields of cluster analysis; for example, the second data key field can be extracted from different dimensions of time, type, position, business association and the like so as to facilitate subsequent multidimensional clustering analysis and help to mine alarm association rules. Optionally, the second data keyword may be an alarm time, an alarm type, a service class or a service priority, and further perform cluster analysis on the plurality of pieces of alarm information based on the second key field to obtain each cluster result.
In the embodiment of the application, the time distribution rule of the alarms, the difference of the alarms in different time periods and whether the alarms are aggregated in time are analyzed, so that the establishment of the alarm rule containing time attributes can be facilitated. The network alarm distribution differences of different regions are analyzed, the influence of the regions on the alarms is judged, and the subsequent establishment of rules containing the region characteristics can be facilitated. Since the association analyzes alarms in the same time period or region, the potential association between different alarm information can be found.
S103, counting the support degree and the confidence degree of any first alarm information pair according to the clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence degree, wherein the first alarm information pair comprises two ordered alarm information in the clustering result.
In the embodiment of the application, the first alarm information pair comprises orderly first alarm information and second alarm information.
In some implementations, for any first alarm information pair in the clustering result, the frequency of occurrence of the first alarm information pair in the clustering result is counted as the support degree of the first alarm information pair.
In some implementations, under the condition that the confidence coefficient is one, the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result is counted as the confidence coefficient of the first alarm information pair, and two alarm association matrixes are generated according to the support coefficient and the confidence coefficient.
In some implementations, if the confidence level is two, the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result is counted as the pre-piece confidence level of the first alarm information pair. And counting the conditional probability of the continuous occurrence of the second alarm information and the first alarm information in the clustering result as the back-piece confidence coefficient of the first alarm information pair, and generating three alarm association matrixes according to the support degree, the front-piece confidence coefficient and the back-piece confidence coefficient.
S104, acquiring the derived intensity of the alarm information pair according to the confidence coefficient, and acquiring alarm rules based on the derived intensity and a plurality of alarm incidence matrixes to generate a rule database which is used for alarm analysis.
The second alarm information pair comprises orderly second alarm information and first alarm information.
In some implementations, under the condition that the confidence coefficient is one, the derived strength is one, for any first alarm information pair, the front piece derived strength of the first alarm information pair is obtained according to the ratio of the front piece confidence coefficient of the first alarm information pair to the front piece confidence coefficient of the second alarm information pair, the alarm correlation matrix is filtered based on the threshold value of the front piece derived strength, alarm rules are obtained, and a rule database is generated.
In some implementations, under the condition that the confidence degrees are two, the derived intensities are two, for any first alarm information pair, the front part derived intensity of the first alarm information pair is obtained according to the ratio of the front part confidence degree of the first alarm information pair to the front part confidence degree of the second alarm information pair, the rear part derived intensity of the first alarm information pair is obtained according to the ratio of the rear part confidence degree of the first alarm information pair to the rear part confidence degree of the second alarm information pair, the alarm correlation matrix is filtered based on the threshold values corresponding to the front part derived intensity and the rear part derived intensity, alarm rules are obtained, and a rule database is generated.
In some implementations, in order to improve accuracy of the alert rule, the alert association matrix may be filtered based on multiple thresholds of derived strength, confidence, support, etc., to obtain the alert rule, and generate a rule database.
In the embodiment of the application, multidimensional clustering analysis is carried out on a plurality of pieces of alarm information to obtain clustering results; counting the support degree and the confidence coefficient of any first alarm information pair according to the clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence coefficient, wherein the first alarm information pair comprises two ordered alarm information in the clustering result; and acquiring the derived intensity of the alarm information pair according to the confidence coefficient, acquiring alarm rules based on the derived intensity and a plurality of alarm incidence matrixes, and generating a rule database which is used for alarm analysis. The application can improve the fault processing efficiency and flexibility, and can realize effective rule screening and improve the real-time performance and accuracy of alarm analysis processing when a large number of fault alarms occur.
FIG. 2 is a flow chart of a method of generating a rule database for alert analysis according to one embodiment of the present application, as shown in FIG. 2, the method comprising the steps of:
s201, a plurality of pieces of alarm information are acquired.
S202, carrying out multidimensional clustering analysis on a plurality of pieces of alarm information to obtain clustering results.
The description of step S201 to step S202 may be referred to the relevant content in the above embodiment, and will not be repeated here.
S203, counting the occurrence frequency of the first alarm information pairs in the clustering result aiming at any first alarm information pair, and taking the occurrence frequency as the support degree of the first alarm information pair.
In the embodiment of the application, the first alarm information pair is assumed to comprise orderly first alarm information a and second alarm information b, the first alarm information pair can be expressed as a- & gt b, and in the embodiment of the application, the position where the first alarm information pair a- & gt b appears in the clustering result, namely the association frequency, is counted and used as the support degree of the first alarm information pair.
S204, counting the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result as the front piece confidence level of the first alarm information pair.
In the embodiment of the application, the pre-part confidence degree refers to the conditional probability that the second alarm information b appears next on the premise that the first alarm information a appears.
S205, counting the conditional probability that the second alarm information and the first alarm information continuously appear in the clustering result as the afterpart confidence of the first alarm information pair.
In the embodiment of the application, the provision of the back-piece confidence coefficient is to solve the problem that the confidence coefficient ignores the support degree of the item set in the back-piece of the rule, the front-piece confidence coefficient a- & gt b is 100%, the first alarm information a is defined as b, the back-piece confidence coefficient is 100%, the first alarm information a is defined in front of the second alarm information b, the front-piece confidence coefficient is very low, but when the back-piece confidence coefficient is very high, the rule a- & gt b (the rule of the first alarm information pair) is considered to be effective, so that the back-piece confidence coefficient is introduced, namely, the conditional probability of the second alarm information b caused by the first alarm information a is introduced on the premise that the second alarm information b is found. The support level may be used to measure whether the first alarm message pair (a- > b) occurs frequently, and the front-part confidence level and the back-part confidence level are used to illustrate the strength of the association between the first alarm message pair (a- > b). The higher the front-piece confidence and the back-piece confidence, the greater the strength of the association between the first alarm information pair (a- > b) is explained.
S206, generating a plurality of alarm association matrixes according to the support degree and the confidence degree, wherein the first alarm information pair comprises two pieces of ordered alarm information in the clustering result.
In the embodiment of the application, the alarm incidence matrix comprises three elements, wherein the elements of the alarm incidence matrix are respectively the support degree of the alarm information pair, the support degree of the front part and the confidence degree of the back part.
Taking the row code of the alarm correlation matrix as an example to represent the preceding alarm information, the column code of the element as the following alarm information, the elements in the table are illustrated as supporting degrees, and the alarm correlation matrix may be shown in table 1, where c, d, e, f, g represents different alarm information, for example, element '666' in the table represents the supporting degrees of the alarm information to c→g.
TABLE 1
| c | d | e | f | g | |
| c | 26 | 110 | 20 | 280 | 0 |
| d | 376 | 910 | 680 | 787 | 23 |
| e | 32 | 324 | 4322 | 2345 | 27 |
| f | 23 | 57 | 89 | 54 | 23 |
| g | 666 | 45 | 65 | 43 | 679 |
S207, acquiring the derived intensity of the alarm information pair according to the confidence coefficient, and acquiring alarm rules based on the derived intensity and a plurality of alarm incidence matrixes, so as to generate a rule database which is used for alarm analysis.
The description of step S207 may be referred to the relevant content in the above embodiment, and will not be repeated here.
In the embodiment of the application, aiming at any first alarm information pair, the occurrence frequency of the first alarm information pair in a clustering result is counted, the occurrence frequency is used as the support degree of the first alarm information pair, the conditional probability that the first alarm information and the second alarm information continuously occur in the clustering result is counted as the front piece confidence degree of the first alarm information pair, the conditional probability that the second alarm information and the first alarm information continuously occur in the clustering result is counted as the rear piece confidence degree of the first alarm information pair, and a plurality of alarm association matrixes are generated according to the support degree and the confidence degree. The application can improve the fault processing efficiency and flexibility, and can realize effective rule screening and improve the real-time performance and accuracy of alarm analysis processing when a large number of fault alarms occur.
FIG. 3 is a flow chart of a method of generating a rule database for alert analysis according to one embodiment of the present application, as shown in FIG. 3, the method comprising the steps of:
s301, acquiring a plurality of pieces of alarm information.
S302, carrying out multidimensional clustering analysis on the pieces of alarm information to obtain clustering results.
S303, counting the support degree and the confidence degree of any first alarm information pair according to the clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence degree, wherein the first alarm information pair comprises two ordered alarm information in the clustering result.
The description of step S301 to step S303 may refer to the relevant content in the above embodiment, and will not be repeated here.
The confidence level includes a front-part confidence level and a back-part confidence level.
S304, aiming at any first alarm information pair, acquiring the front piece derived strength of the first alarm information pair according to the ratio of the front piece confidence coefficient of the first alarm information pair and the front piece confidence coefficient of the second alarm information pair.
It should be noted that, in order to screen out potential rules, in the embodiment of the present application, the derived strength of the front part and the derived strength of the back part are defined to measure the derived strength of the first alarm information pair.
The second alarm information pair comprises orderly second alarm information and first alarm information.
In the embodiment of the application, the first alarm information pair is assumed to comprise orderly first alarm information a and second alarm information b, the first alarm information pair can be expressed as a-b, and the following formula can be adopted to obtain the derived strength of the front part:
wherein, derive (a- > b) represents the front piece derived intensity of the ordered alarm information a- > b, conf (a- > b) represents the front piece confidence of the ordered alarm information a- > b, and conf (b- > a) represents the front piece confidence of the ordered alarm information b- > a.
In the embodiment of the application, the front piece confidence is an important index for evaluating the association rule. Ranging between 0 and 1. The higher the confidence, the more likely b will appear if a appears, and the more reliable the rule a→b.
S305, acquiring the derived strength of the back part of the first alarm information pair according to the ratio of the back part confidence coefficient of the first alarm information pair and the back part confidence coefficient of the second alarm information pair.
In the embodiment of the application, the first alarm information pair is assumed to comprise orderly first alarm information a and second alarm information b, the first alarm information pair can be expressed as a-b, and the following formula can be adopted to obtain the derived strength of the back part:
wherein bderive (a- > b) represents the strength of the derived back-piece of the ordered alarm information a- > b, bconf (a- > b) represents the confidence of the back-piece of the ordered alarm information a- > b, and bconf (b- > a) represents the confidence of the back-piece of the ordered alarm information b- > a.
S306, filtering the alarm association matrix based on the threshold values corresponding to the derived strength, the confidence coefficient and the support degree respectively, and obtaining the filtered alarm rule.
In some implementations, the evaluation index value of the potential rule includes a front piece derived strength, a back piece derived strength, a front piece confidence, a back piece confidence, a support, and a threshold value of the rule evaluation index is set. Optionally, in the embodiment of the present application, the front piece derived strength corresponds to a front piece derived strength threshold, the back piece derived strength corresponds to a back piece derived strength threshold, the front piece confidence level corresponds to a front piece confidence level threshold, the back piece confidence level corresponds to a back piece confidence level threshold, the support level corresponds to a support level threshold, the evaluation index value of each potential rule is compared with a set threshold, if the evaluation index value of the rule is higher than the corresponding threshold, the rule is reserved, if the evaluation index value of the rule is lower than the corresponding threshold, the rule is filtered, and a plurality of filtered alarm correlation matrices are obtained after the threshold filtration.
In the embodiment of the present application, the intensity of the filtering can be controlled by adjusting the threshold value, and the higher the threshold value is, the more strict the filtering is, and the less rules remain in the following.
It should be noted that the assumption on which the front piece derived intensity and the back piece derived intensity are based is that alarms cannot be pushed each other, and if (a→b) is present, there is no (b→a). If the derived strength of the front piece or the derived strength of the rear piece is larger than 1, a.fwdarw.b is more reliable than b.fwdarw.a, and the statistical rule is more satisfied.
It can be seen from equations (1) and (2) that there is no self-push between alarms, and if both derive (a→b) and bderive (a→b) are equal to 1, they need to be filtered out. According to the principle, potential alarm association rules can be obtained. The confidence level includes a back-piece confidence level that describes the strength of the association of a- > b, and a larger value describes a larger strength of the association. However, it is not entirely possible to state that a can derive b, since in this case the confidence of b→a may also be high. To avoid filtering out a portion of valid rules, some thresholds in the subsequent association rule algorithm may be set lower when mining potential advanced rules from the alert association matrix, and the resulting potential rules will be correspondingly higher.
In the embodiment of the application, the rule database for alarm analysis can be applied to the fusion alarm analysis of the IT system of the Internet technology, and the fusion alarm analysis of the IT system can comprise an alarm rule mining stage and an alarm analysis processing stage.
As shown in fig. 4, in some implementations, in the alert rule mining stage, the network management system derives the historical alert data file offline, as the learned data of rule mining, after the historical alert data is read, the data preprocessing module performs data preprocessing, that is, detects the validity of all data, screens out invalid data, and imports the invalid data into the alert database, and simultaneously imports the network topology, service information, alert hierarchy information and the like at the historical moment in the alert database. The data clustering module extracts data key fields required by clustering from the alarm database, further realizes the clustering of alarm information, and divides the data in time domain and geographic position. The rule mining module acquires the clustering result, extracts the alarm data from the alarm database, and performs association analysis on the alarm data of each cluster to realize rule mining. And then the mined rules are screened by the effective rules to form a rule database.
As shown in fig. 5, in some implementations, in the alarm analysis processing stage, the network management system transmits the alarm data at the current moment to the data interface module on line through the background interface, and the data interface module reads the current alarm data and then transmits the current alarm data to the alarm database, and the network topology, service information, alarm level information and the like at the current moment are simultaneously transmitted to the alarm database. The data clustering module extracts key fields with analysis alarm data from the alarm database, further realizes the clustering of the current alarm data, and divides the data in time domain and geographic position. And the alarm processing analysis module is used for acquiring a clustering result and extracting alarm data from an alarm database. Traversing all alarm rules from a rule database, analyzing each cluster of alarm data, judging the sequence association relation between different alarms according to the information such as the time sequence, the space position and the like of the occurrence of the alarms, matching each alarm with the alarm association rules in the rule database, judging which other alarms are associated with each alarm, and determining the source alarms in each cluster of alarms according to the alarm association relation, wherein the source alarms are the original alarms which cause other associated alarms. The root alarms are reserved, repeated and associated secondary alarms are compressed, and the number of alarms is reduced. Through the analysis steps, the root alarms in each alarm cluster can be distinguished, so that the fault cause can be rapidly positioned, and the alarm compression is realized.
In the embodiment of the application, the alarm association analysis processing method adopts a clustering unsupervised machine learning mode. Clustering divides the data set into a plurality of different classes according to the data characteristics in the data set of the unknown label sample, so that the data samples in the same class are similar as much as possible, and the similarity between the data samples in different classes is as small as possible. Therefore, the alarm information is clustered, different root alarms and derivative alarms thereof are distinguished according to the data attribute of the alarm information, namely, each type represents one root alarm and derivative alarms thereof, and then the associated rule analysis is performed, so that the accuracy can be improved. Clustering based on location and time information may utilize accurate location information (e.g., network elements) to "hard-divide" the alert data, e.g., clustering in the time dimension may be performed using a density-based clustering algorithm (e.g., DBSCAN algorithm) using the start time and end time of the alert.
Fig. 6 is a block diagram of a generating apparatus of a rule database for alarm analysis according to an embodiment of the present disclosure, and as shown in fig. 6, a generating apparatus 600 of a rule database for alarm analysis includes:
a first obtaining module 610, configured to obtain a plurality of alarm information;
a second obtaining module 620, configured to perform multidimensional clustering analysis on the plurality of alarm messages, and obtain clustering results;
the first generating module 630 is configured to count, for any clustering result, a support degree and a confidence degree of any first alarm information pair according to the clustering result, and generate a plurality of alarm association matrices according to the support degree and the confidence degree, where the first alarm information pair includes two ordered alarm information in the clustering result;
the second generating module 640 is configured to obtain derived intensity of the alarm information pair according to the confidence coefficient, and obtain alarm rules based on the derived intensity and the multiple alarm correlation matrices, and generate a rule database, where the rule database is used for alarm analysis.
In some embodiments, the confidence level includes two, the first alert information pair includes ordered first alert information and second alert information, and the first generation module 630 is further configured to:
counting the occurrence frequency of the first alarm information pairs in the clustering result aiming at any first alarm information pair, and taking the occurrence frequency as the support degree of the first alarm information pair;
counting the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result as the front piece confidence level of the first alarm information pair;
and counting the conditional probability that the second alarm information and the first alarm information continuously appear in the clustering result as the afterpart confidence coefficient of the first alarm information pair.
In some embodiments, the alert correlation matrix includes three elements, where the elements of the alert correlation matrix are the support of alert information pairs, the front piece support, and the back piece confidence, respectively.
In some embodiments, the derived intensities include a front piece derived intensity and a back piece derived intensity, the second generation module 640 is further configured to:
aiming at any first alarm information pair, acquiring the front piece derived strength of the first alarm information pair according to the ratio of the front piece confidence coefficient of the first alarm information pair to the front piece confidence coefficient of the second alarm information pair;
acquiring the back part derived strength of the first alarm information pair according to the ratio of the back part confidence coefficient of the first alarm information pair to the back part confidence coefficient of the second alarm information pair;
the second alarm information pair comprises orderly second alarm information and first alarm information.
In some embodiments, the second generating module 640 is further configured to:
and filtering the alarm association matrix based on the threshold values corresponding to the derived strength, the confidence coefficient and the support degree respectively to obtain a filtered alarm rule.
In some embodiments, the first obtaining module 610 is further configured to:
acquiring historical alarm data of a network management system and service information of a service system, and acquiring network topology data and alarm hierarchy information;
filtering the historical alarm data to remove data missing from a first data key field in the historical alarm data, wherein the first data key field is a key field of the alarm data;
and generating a plurality of pieces of alarm information according to the historical alarm data, the service information, the network topology data and the alarm hierarchy information.
In some embodiments, the second acquisition module 620 is further configured to:
extracting multidimensional second data key fields from the plurality of pieces of alarm information, wherein the second data key fields are key fields of cluster analysis;
and carrying out cluster analysis on the plurality of pieces of alarm information based on the second key field to obtain clustering results.
The application can improve the fault processing efficiency and flexibility, and can realize effective rule screening and improve the real-time performance and accuracy of alarm analysis processing when a large number of fault alarms occur.
Fig. 7 is a block diagram of an electronic device 700, according to an example embodiment.
As shown in fig. 7, the electronic device 700 includes:
a memory 701 and a processor 702, a bus 703 connecting different components (including the memory 701 and the processor 702), the memory 701 storing a computer program, the processor 702 implementing the rule database generation method for alert analysis of the embodiments of the present disclosure when executing the program.
Bus 703 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 700 typically includes a variety of electronic device readable media. Such media can be any available media that is accessible by electronic device 700 and includes both volatile and nonvolatile media, removable and non-removable media.
Memory 701 may also include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 704 and/or cache memory 705. Electronic device 700 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 706 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, commonly referred to as a "hard drive"). Although not shown in fig. 7, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 703 through one or more data medium interfaces. Memory 701 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the disclosure.
A program/utility 708 having a set (at least one) of program modules 707 may be stored in, for example, memory 701, such program modules 707 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 707 generally perform the functions and/or methods in the embodiments described in this disclosure.
The electronic device 700 may also communicate with one or more external devices 709 (e.g., keyboard, pointing device, display 711, etc.), with one or more devices that enable a user to interact with the electronic device 700, and/or with any device (e.g., network card, modem, etc.) that enables the electronic device 700 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 712. Also, the electronic device 700 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 713. As shown in fig. 7, the network adapter 713 communicates with other modules of the electronic device 700 via the bus 703. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 700, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processor 702 executes various functional applications and data processing by running programs stored in the memory 701.
It should be noted that, the implementation process and the technical principle of the electronic device in this embodiment refer to the foregoing explanation of the method for generating the rule database for alarm analysis in the embodiment of the disclosure, and are not repeated herein.
To achieve the above embodiments, the present disclosure also proposes a computer-readable storage medium.
Wherein the instructions in the computer-readable storage medium, when executed by the processor of the electronic device, enable the electronic device to perform a method of generating a rules database for alert analysis as previously described. Alternatively, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (16)
1. A method of generating a rules database for alert analysis, comprising:
acquiring a plurality of pieces of alarm information;
performing multidimensional clustering analysis on the plurality of pieces of alarm information to obtain clustering results;
counting the support degree and the confidence degree of any first alarm information pair according to any clustering result aiming at any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence degree, wherein the first alarm information pair comprises two ordered alarm information in the clustering result;
and acquiring the derived intensity of the alarm information pair according to the confidence coefficient, and acquiring alarm rules based on the derived intensity and the alarm incidence matrixes to generate a rule database, wherein the rule database is used for alarm analysis.
2. The method of claim 1, wherein the confidence level includes two, the first alarm information pair includes ordered first alarm information and second alarm information, and the counting the support level and the confidence level of any first alarm information pair according to the clustering result includes:
counting the occurrence frequency of the first alarm information pairs in the clustering result aiming at any first alarm information pair, and taking the occurrence frequency as the support degree of the first alarm information pair;
counting the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result as the front piece confidence level of the first alarm information pair;
and counting the conditional probability that the second alarm information and the first alarm information continuously appear in the clustering result as the back-piece confidence coefficient of the first alarm information pair.
3. The method of claim 2, wherein the alert correlation matrix comprises three elements, wherein the elements of the alert correlation matrix are the support of the alert information pair, the front support, and the back confidence, respectively.
4. A method according to claim 2 or 3, wherein the derived intensities comprise a front-piece derived intensity and a back-piece derived intensity, the obtaining the derived intensity of the first alert information pair according to the confidence level comprising:
aiming at any first alarm information pair, acquiring the front piece derived strength of the first alarm information pair according to the ratio of the front piece confidence coefficient of the first alarm information pair to the front piece confidence coefficient of the second alarm information pair;
acquiring the derived strength of the back part of the first alarm information pair according to the ratio of the back part confidence coefficient of the first alarm information pair to the back part confidence coefficient of the second alarm information pair;
wherein the second alarm information pair includes the second alarm information and the first alarm information in order.
5. The method of claim 4, wherein the obtaining an alert rule based on the derived intensities and the plurality of alert correlation matrices comprises:
and filtering the alarm association matrix based on the threshold values corresponding to the derived intensity, the confidence coefficient and the support degree respectively to obtain the filtered alarm rule.
6. The method of claim 1, wherein the obtaining a plurality of alert messages comprises:
acquiring historical alarm data of a network management system and service information of a service system, and acquiring network topology data and alarm hierarchy information;
filtering the historical alarm data to remove data missing from a first data key field in the historical alarm data, wherein the first data key field is a key field of the alarm data;
and generating a plurality of pieces of alarm information according to the historical alarm data, the service information, the network topology data and the alarm hierarchy information.
7. The method according to claim 1 or 6, wherein performing multidimensional clustering analysis on the plurality of pieces of alarm information to obtain clustering results includes:
extracting multidimensional second data key fields from the plurality of pieces of alarm information, wherein the second data key fields are key fields of cluster analysis;
and carrying out cluster analysis on the plurality of pieces of alarm information based on the second key field to acquire the clustering results.
8. A rule database generation apparatus for alert analysis, comprising:
the first acquisition module is used for acquiring a plurality of pieces of alarm information;
the second acquisition module is used for carrying out multidimensional clustering analysis on the plurality of pieces of alarm information to acquire clustering results;
the first generation module is used for counting the support degree and the confidence coefficient of any first alarm information pair according to any clustering result, and generating a plurality of alarm association matrixes according to the support degree and the confidence coefficient, wherein the first alarm information pair comprises two ordered alarm information in the clustering result;
the second generation module is used for acquiring the derived intensity of the alarm information pair according to the confidence coefficient, acquiring alarm rules based on the derived intensity and the alarm correlation matrixes, and generating a rule database, wherein the rule database is used for alarm analysis.
9. The apparatus of claim 8, wherein the confidence level comprises two, the first alert information pair comprises ordered first alert information and second alert information, the first generation module further to:
counting the occurrence frequency of the first alarm information pairs in the clustering result aiming at any first alarm information pair, and taking the occurrence frequency as the support degree of the first alarm information pair;
counting the conditional probability that the first alarm information and the second alarm information continuously appear in the clustering result as the front piece confidence level of the first alarm information pair;
and counting the conditional probability that the second alarm information and the first alarm information continuously appear in the clustering result as the back-piece confidence coefficient of the first alarm information pair.
10. The apparatus of claim 9, wherein the alert correlation matrix comprises three elements, wherein the elements of the alert correlation matrix are the support of the alert information pair, the front support, and the back confidence, respectively.
11. The apparatus of claim 9 or 10, wherein the derived intensities comprise a front piece derived intensity and a back piece derived intensity, the second generation module further configured to:
aiming at any first alarm information pair, acquiring the front piece derived strength of the first alarm information pair according to the ratio of the front piece confidence coefficient of the first alarm information pair to the front piece confidence coefficient of the second alarm information pair;
acquiring the derived strength of the back part of the first alarm information pair according to the ratio of the back part confidence coefficient of the first alarm information pair to the back part confidence coefficient of the second alarm information pair;
wherein the second alarm information pair includes the second alarm information and the first alarm information in order.
12. The apparatus of claim 11, wherein the second generation module is further configured to:
and filtering the alarm association matrix based on the threshold values corresponding to the derived intensity, the confidence coefficient and the support degree respectively to obtain the filtered alarm rule.
13. The apparatus of claim 8, wherein the first acquisition module is further configured to:
acquiring historical alarm data of a network management system and service information of a service system, and acquiring network topology data and alarm hierarchy information;
filtering the historical alarm data to remove data missing from a first data key field in the historical alarm data, wherein the first data key field is a key field of the alarm data;
and generating a plurality of pieces of alarm information according to the historical alarm data, the service information, the network topology data and the alarm hierarchy information.
14. The apparatus of claim 8 or 13, wherein the second acquisition module is further configured to:
extracting multidimensional second data key fields from the plurality of pieces of alarm information, wherein the second data key fields are key fields of cluster analysis;
and carrying out cluster analysis on the plurality of pieces of alarm information based on the second key field to acquire the clustering results.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the steps of the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935966.3A CN117041009A (en) | 2023-07-27 | 2023-07-27 | Method and device for generating rule database for alarm analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935966.3A CN117041009A (en) | 2023-07-27 | 2023-07-27 | Method and device for generating rule database for alarm analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117041009A true CN117041009A (en) | 2023-11-10 |
Family
ID=88638282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310935966.3A Pending CN117041009A (en) | 2023-07-27 | 2023-07-27 | Method and device for generating rule database for alarm analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117041009A (en) |
-
2023
- 2023-07-27 CN CN202310935966.3A patent/CN117041009A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210042270A1 (en) | Alarm log compression method, apparatus, and system, and storage medium | |
| CN109587008B (en) | Method, device and storage medium for detecting abnormal flow data | |
| US8751417B2 (en) | Trouble pattern creating program and trouble pattern creating apparatus | |
| US20160055044A1 (en) | Fault analysis method, fault analysis system, and storage medium | |
| CN111309565B (en) | Alarm processing method and device, electronic equipment and computer readable storage medium | |
| CN109684052B (en) | Transaction analysis method, device, equipment and storage medium | |
| CN114185708A (en) | Data analysis method and device based on distributed link tracking and electronic equipment | |
| US11863439B2 (en) | Method, apparatus and storage medium for application identification | |
| CN111078513A (en) | Log processing method, device, equipment, storage medium and log alarm system | |
| CN111160021A (en) | Log template extraction method and device | |
| CN112800061B (en) | Data storage method, device, server and storage medium | |
| CN111274218A (en) | Multi-source log data processing method for power information system | |
| CN111258798A (en) | Fault positioning method and device for monitoring data, computer equipment and storage medium | |
| CN113254255A (en) | Cloud platform log analysis method, system, device and medium | |
| US20220103442A1 (en) | Internet of things operations monitoring system | |
| CN117130851A (en) | High-performance computing cluster operation efficiency evaluation method and system | |
| CN117041009A (en) | Method and device for generating rule database for alarm analysis | |
| CN115102848A (en) | Log data extraction method, system, device and medium | |
| CN115330140A (en) | Building risk prediction method based on data mining and prediction system thereof | |
| CN111737371B (en) | Data flow detection classification method and device capable of dynamically predicting | |
| CN114860543A (en) | Anomaly detection method, device, equipment and computer readable storage medium | |
| CN112612679A (en) | System running state monitoring method and device, computer equipment and storage medium | |
| CN112347102A (en) | Multi-table splicing method and multi-table splicing device | |
| CN113342861B (en) | Data management method and device in service scene | |
| Chen et al. | A Robust Log Parsing Algorithm—Practice of Logslaw in Heterogeneous Logs of Pacific Credit Card Center of Bank of Communications (PCCC) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |