CN115883323A - Alarm analysis method, device, equipment and computer storage medium - Google Patents

Alarm analysis method, device, equipment and computer storage medium Download PDF

Info

Publication number
CN115883323A
CN115883323A CN202111144543.7A CN202111144543A CN115883323A CN 115883323 A CN115883323 A CN 115883323A CN 202111144543 A CN202111144543 A CN 202111144543A CN 115883323 A CN115883323 A CN 115883323A
Authority
CN
China
Prior art keywords
alarm
item
frequent
candidate
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111144543.7A
Other languages
Chinese (zh)
Inventor
崔棋纹
龙翼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Suzhou Software Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Suzhou Software Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Suzhou Software Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202111144543.7A priority Critical patent/CN115883323A/en
Publication of CN115883323A publication Critical patent/CN115883323A/en
Pending legal-status Critical Current

Links

Images

Abstract

The embodiment of the invention relates to the technical field of computer data processing, and discloses an alarm analysis method, an alarm analysis device, alarm analysis equipment and a computer storage medium, wherein the method comprises the following steps: processing original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence; performing reverse indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed; performing correlation analysis on a data set to be analyzed to obtain a frequent item set; determining alarm sequence information of each alarm item in the frequent item set; and determining alarm item root relation corresponding to the frequent item set according to the alarm sequence information. Through the mode, the alarm analysis efficiency is improved.

Description

Alarm analysis method, device, equipment and computer storage medium
Technical Field
The embodiment of the invention relates to the technical field of computer data processing, in particular to an alarm analysis method, an alarm analysis device, alarm analysis equipment and a computer storage medium.
Background
When network maintenance is carried out, correlation analysis needs to be carried out on mass alarm data, and root cause alarm is determined, so that the problem of alarm caused by pertinence is solved, and the stability of the network is improved.
The inventor finds that the existing root cause alarm analysis generally depends on manual work or experts to make a root cause alarm analysis rule, has large workload and high maintenance time cost, and cannot meet the requirement of daily network maintenance.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide an alarm analysis method, which is used to solve the problem in the prior art that alarm analysis efficiency is low.
According to an aspect of an embodiment of the present invention, there is provided an alarm analysis method, including:
processing the original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
performing inverted indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed;
performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
determining alarm sequence information of each alarm item in the frequent item set;
and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
In an optional manner, the method further comprises:
determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed;
1 is taken from k, and the candidate frequent k item set is screened according to the minimum support degree to obtain the frequent k item set of the alarm items; wherein k is a positive integer; the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set;
performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain alarm transaction item sets corresponding to candidate frequent k +1 item sets;
and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
In an alternative approach, the method is based on a distributed stream data processing framework; the method further comprises the following steps:
determining the calculation parallelism;
and performing parallel correlation analysis on the data set to be analyzed according to the distributed stream data processing framework and the calculation parallelism to obtain the frequent item set.
In an optional manner, the method further comprises:
determining a maximum frequent item set in the frequent item sets;
determining a maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed;
respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information;
determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average order;
calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item;
and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
In an optional manner, the method further comprises:
determining the alarm items for which the average order is greater than an order threshold as the candidate root alarm items;
determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
In an optional manner, the method further comprises:
respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set;
determining the number of simultaneous occurrences of the candidate root alarm item and the candidate derivative alarm item according to the number of occurrences;
and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
In an optional manner, the method further comprises:
acquiring the original alarm data;
performing feature extraction on the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items;
and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
According to another aspect of the embodiments of the present invention, there is provided an alarm analyzing apparatus, including:
the first processing module is used for processing the original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
the second processing module is used for performing reverse indexing on the alarm transaction item according to the alarm item to obtain a data set to be analyzed;
the analysis module is used for performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
the first determining module is used for determining the alarm sequence information of each alarm item in the frequent item set;
and the second determining module is used for determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
According to another aspect of the embodiments of the present invention, there is provided an alarm analysis device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the alarm analysis method as described.
According to a further aspect of the embodiments of the present invention, there is provided a computer-readable storage medium having at least one executable instruction stored therein, the executable instruction causing an alarm analysis device to perform the operations of the alarm analysis method as described.
The embodiment of the invention firstly processes the original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence; performing reverse indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed; therefore, the determining efficiency of the frequent item set is improved, the data set to be analyzed is subjected to correlation analysis to obtain the frequent item set, and finally the alarm sequence information of each alarm item in the frequent item set is determined; and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
The method and the device are different from the problem that the alarm analysis efficiency is low due to the fact that root cause analysis is carried out according to the rule set by people in the prior art, and the method and the device can improve the alarm analysis efficiency by carrying out transaction information extraction and reverse index processing on original alarm data, then carrying out frequent item set mining, and then carrying out root cause analysis based on alarm sequence information of alarm items in the frequent item set.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of an alarm analysis method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a sliding window processing in the alarm analysis method according to the embodiment of the present invention;
FIG. 3 is a performance diagram of an alarm analysis method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an alarm analysis device provided in an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an alarm analysis device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein.
FIG. 1 is a flow chart illustrating an alarm analysis method, as provided by an embodiment of the present invention, as performed by a computer processing device. The computer processing device may include a cell phone, a notebook computer, etc. As shown in fig. 1, the method comprises the steps of:
step 10: and processing the original alarm data to obtain a plurality of alarm transaction items.
In an embodiment of the invention, the original alarm data is preprocessed, the feature extraction is performed on the preprocessed data, the alarm data information field is extracted, the alarm data feature field related to association rule analysis is selected from the alarm data information field, and the alarm data feature field and the corresponding field value are determined as an alarm item.
In one embodiment of the invention, each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence. Wherein, the alarm time sequence refers to the occurrence time sequence of each alarm item.
For example, the alarm transaction item T may be { a }, { a, b }, or { b, c }, where a, b, c are each an alarm item, and the occurrence time of a, b, c gradually approaches the current time.
In consideration of the fact that data in the original alarm data are relatively discrete, more information redundancy exists and transaction characteristics are lacked, transaction characteristic extraction needs to be performed on the original alarm data, and in order to improve the efficiency of transaction characteristic extraction, the original alarm data can be preprocessed first to extract alarm item information related to association analysis and included in the original alarm data.
Thus, in a further embodiment of the present invention, step 10 further comprises: step 101: and acquiring the original alarm data.
In one embodiment of the invention, the raw alarm data may be periodically acquired, and the raw alarm data may be in the form of a discrete data stream.
Step 102: extracting the characteristics of the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items.
In one embodiment of the invention, the original alarm data is preprocessed and feature extracted, the alarm data information field is extracted, the alarm data feature field related to association rule analysis is selected from the alarm data information field, and the alarm data feature field and the corresponding field value are determined as an alarm item.
The alarm data information field may include an alarm identifier, a resource object identifier, an alarm title, an alarm suggestion, an alarm location, an alarm level, an alarm type, an alarm state, an alarm time, and the like. The alarm data characteristic field can be an alarm identifier used for specifically identifying one-time alarm, and other alarm data information fields such as an associated alarm title, an alarm level, an alarm suggestion and the like can be correspondingly searched according to the alarm identifier.
Step 103: and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
In an embodiment of the present invention, referring to the sliding window processing diagram shown in fig. 2, a sliding time window with a preset length is slid on candidate alarm transaction data, and an alarm item in one sliding time window is determined as one alarm transaction item, so as to extract multiple alarm transaction items.
Wherein, the time starting point of the sliding time window starts from the alarm occurrence time of the first alarm item, and slides in sequence according to the sliding step length (s in fig. 2); if the interval time between the alarm items exceeds the window length (w in fig. 2) of the sliding window, the start point of the sliding window is reset, and the sliding of the sliding window is performed again with the alarm time of the next alarm item as the start point.
In still another embodiment of the present invention, the window length of the sliding time window affects the number of alarm items in each alarm transaction item, that is, the length of the alarm transaction item, and the sliding step length is related to the number of alarm transaction items in the alarm transaction data, specifically, the sliding time window length may be 40s, and the sliding step length may be 20s.
In another embodiment of the present invention, each alarm item in the same sliding time window is subjected to deduplication processing to form an alarm item set, and meanwhile, a null set and a single element set are ignored, and each processed alarm item set is an alarm transaction item T.
In an embodiment of the present invention, the data format of the obtained alarm transaction item may refer to table 1.
Figure BDA0003284918650000061
/>
Figure BDA0003284918650000071
TABLE 1
Step 20: and performing inverted indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed.
In consideration of the prior art, when mining association rules of alarm data, an Apriori algorithm and an FP-Growth algorithm are generally adopted to mine association rules of alarm data and determine a root cause relationship.
The Apriori algorithm needs to scan a data alarm transaction database for multiple times, and when massive alarm data are faced, the mode that the Apriori algorithm searches frequent item sets is too violent, so that the problems of frequent I/O operation, low connection efficiency, high calculation overhead and the like exist.
The FP-Growth algorithm excavates a frequent item set without generating a candidate set item set in a mode of generating the FP-Tree, the algorithm only needs to scan a data alarm transaction database twice, and has certain advantages in efficiency compared with an Apriori algorithm, but the construction of the FP-Tree usually consumes a large amount of memory space, and meanwhile, as the FP-Tree grows, a large amount of time is consumed when the FP-Tree is traversed.
Therefore, in one embodiment of the present invention, mining between association rules of alarm terms is performed according to the Eclat (Equivalence Class Transformation) algorithm. Therefore, firstly, the alarm transaction items are reversely indexed according to the alarm items to obtain a data set to be analyzed. In connection with the foregoing example, the data set to be analyzed converted by inverting the alarm transaction item in table 1 may refer to table 2.
Figure BDA0003284918650000072
/>
Figure BDA0003284918650000081
TABLE 2
And step 30: and performing correlation analysis on the data set to be analyzed to obtain a frequent item set.
The current root cause alarm rule mining algorithm mostly uses a centralized architecture and a serial computing mode, and cannot meet the processing requirement of massive alarm data, so that the use of the traditional root cause alarm mining algorithm in a big data scene is limited.
In order to solve the problem of low root cause alarm rule mining efficiency in a big data scene, the root cause alarm mining algorithm based on a Hadoop and other distributed computing frames is adopted in the prior art, and the mining efficiency of the root cause alarm rule is improved by constructing block indexes of a data set and utilizing the concept of division and treatment, but under the condition of large updating data volume, a large amount of I/O operations can still be generated by the methods, so that the operation efficiency is influenced.
Thus, in a further embodiment of the invention, the method is based on a distributed stream data processing framework; wherein, the distributed stream data processing framework can be an Apache Flink framework. The Apache Flink distributed computing framework is an open-source computing platform facing distributed data stream processing and batch processing, and provides application functions supporting stream processing and batch processing. On the Flink platform, data can be processed as a bounded flow and an unbounded flow, which respectively correspond to batch processing and flow processing. The Flink platform uses batch processing as a special case of stream processing, and solves the batch processing problem by using the concept of stream processing, thereby further improving the efficiency of data processing on the basis of distributed computing.
Step 30 also comprises at least: step 301: and determining the calculation parallelism.
In one embodiment of the invention, the computational parallelism may be determined based on the amount of alarm data. The number of the alarm items can be determined.
Step 302: and performing parallel correlation analysis on the data set to be analyzed according to the distributed stream data processing framework and the calculation parallelism to obtain the frequent item set.
In one embodiment of the invention, the computational parallelism may be set by calling setParalleism method in Flink.
After the calculation parallelism is set, parallel correlation analysis is carried out on the data set to be analyzed in the Flink according to the Eclat algorithm, and a frequent item set is obtained.
In one embodiment of the present invention, step 30 further comprises:
step 303: and determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed.
Step 304: starting from k to 1, screening the candidate frequent k item set according to the minimum support degree to obtain a frequent k item set of the alarm items; wherein k is a positive integer; and the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set.
In an embodiment of the present invention, a filter operator in Flink may be called, a candidate frequent k item set is screened, and the candidate frequent k item set whose occurrence frequency is less than the minimum support degree is deleted, so as to obtain the frequent k item set, where the minimum support degree may be 2. The length of the alarm transaction item set refers to the number of the corresponding alarm transaction items.
Step 305: and performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain the alarm transaction item sets corresponding to the candidate frequent k +1 item sets.
In an embodiment of the invention, considering that a large amount of intersection operation needs to be performed when frequent item sets are mined according to the Eclat algorithm, cross operators of a Flink platform can be called, and the parallelism of the algorithm is increased while the system load caused by the intersection operation is reduced, so that the efficiency of alarm analysis is further improved.
In an embodiment of the invention, a union set between every two is obtained according to the frequent 1-item set obtained in the previous step, a candidate frequent 2-item set is obtained, meanwhile, an intersection set of corresponding alarm transaction item identification sets between every two is obtained, and the collection operation of the frequent item set is completed through cross operators. The cross operator is used for constructing a Cartesian product of two inputs, and calculation-intensive cross product operation can be efficiently completed in a Flink cluster mode, so that the efficiency of association mining is further improved.
Step 306: and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
In an embodiment of the present invention, the Eclat algorithm adopted in steps 301 to 306 only needs to scan the alarm transaction item database once, and obtains the candidate frequent item set through set intersection operation, specifically, reference may be made to fig. 3 for comparison of mining performance of different algorithms in the prior art and the frequent item set in the embodiment of the present invention.
Fig. 3 is a performance comparison diagram of the alarm analysis method provided by the embodiment of the present invention and the prior art. As can be seen from FIG. 3, compared with the Apriori algorithm, FP-Growth and Eclat algorithm used separately in the conventional association rule mining, the efficiency of mining the association rules based on the Flink framework according to the Eclat algorithm adopted in the embodiment of the present invention is significantly improved, thereby improving the efficiency of alarm analysis.
Step 40: and determining the alarm sequence information of each alarm item in the frequent item set.
In one embodiment of the invention, the alarm order information is the occurrence order information of each alarm item in the alarm transaction item containing the alarm item. For example, for an alarm item c in the frequent item set { b, c, d }, as can be known from table 2, alarm transaction items including the alarm item c are 2 and 3, and as can be known from table 1, the alarm order information of the alarm item c is that the alarm item c appears 2 nd in the alarm transaction item 2 and 1 st in the alarm transaction item 2.
Step 50: and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
In one embodiment of the invention, the alarm item root cause relationship refers to the association and cause-and-effect relationship between alarm items, that is, some alarm items occur together, while the appearance of one or some alarm items causes other secondary alarm items. Because each alarm item in the alarm transaction items shown in table 1 retains a certain alarm order relationship, a basis is provided for subsequent mining of alarm root cause relationships. Therefore, the average sequence of each alarm item in each frequent item set in the alarm transaction items containing the alarm item is determined according to the alarm sequence information, the average sequence is used for representing the general rule of the occurrence positions of the alarm items, and the root cause relationship among the alarm items is analyzed according to the average sequence.
Thus, in a further embodiment of the present invention, step 50 further comprises:
step 501: and determining the maximum frequent item set in the frequent item sets.
In one embodiment of the present invention, the frequent item set satisfying the non-inclusive relationship in the frequent item set is determined as the maximum frequent item set, i.e., if all supersets of the frequent item set L are non-frequent item sets, then L is determined as the maximum frequent item set. For example, the maximum frequent item set can be obtained as { b, c, d, f, e }.
Step 502: and determining the maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed.
In an embodiment of the present invention, according to the data set to be analyzed shown in table 2, the alarm transaction items corresponding to each alarm item in the maximum frequent item set are determined, and all the alarm transaction items corresponding to each alarm item are subjected to union set to obtain the maximum alarm transaction item set. For example, the maximum set of alarm transaction items corresponding to b, c, d, f, e is 1,2,3,4, which can be determined from Table 2.
Step 503: and respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information.
In an embodiment of the present invention, according to the association relationship between the alarm transaction items and the alarm items as shown in table 1, the occurrence order of each alarm item in the maximum frequent item set in each alarm transaction item included in the maximum alarm transaction item set is determined. If the alarm transaction item associated with the alarm transaction item b in the maximum alarm transaction item set comprises 1,2, the average order corresponding to b is determined according to the order of b in the alarm transaction items 1 and 2 respectively.
Step 504: and determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average sequence.
In one embodiment of the invention, the average order is filtered according to an order threshold, and the alarm items larger than the order threshold are determined as candidate root alarm items and at least one candidate derivative alarm item.
Thus, in one embodiment of the present invention, step 504 further comprises:
step 5041: determining the alarm items with the average order greater than an order threshold as the candidate root alarm items.
Considering that an alarm is generally a root alarm causing some related alarms and related alarms appear after the root alarm, therefore, an alarm always appearing first in time is more likely to be a root alarm item, therefore, in an embodiment of the present invention, an order threshold is used to characterize a median of the appearance orders of alarm items, when the average order of a certain alarm item is greater than the order threshold, the alarm item is always appeared first compared with other alarm items, and is determined as a candidate root alarm item.
Step 5042: determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
Step 505: and calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item.
In one embodiment of the invention, the degree of improvement represents the ratio of the probability of containing the candidate derived alarm item under the condition of containing the candidate root alarm item and the probability of only seeing the candidate derived alarm. The promotion degree reflects the correlation between the candidate root alarm item and the candidate derived alarm item in the association rule, the promotion degree is greater than 1 and higher indicates that the positive correlation is higher, the promotion degree is less than 1 and lower indicates that the negative correlation is higher, and the promotion degree is equal to 1 indicates that no correlation exists, namely the promotion degrees are mutually independent.
Thus, in one embodiment of the present invention, step 505 further comprises:
step 5051: and respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set.
In yet another embodiment of the present invention, when implemented based on the Flink framework, the number of occurrences of the candidate root alarm item and the candidate derived alarm item can be calculated respectively by using the map operator in the Flink framework.
Step 5052: and determining the simultaneous occurrence times of the candidate root alarm item and the candidate derivative alarm item according to the occurrence times.
In one embodiment of the invention, the times of occurrence of the candidate root alarm item and the candidate derived alarm item in the same alarm transaction item are counted as the times of simultaneous occurrence.
Step 5053: and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
In an embodiment of the present invention, a ratio of the number of occurrences of the candidate root alarm item X and the candidate root alarm item Y at the same time to the number of occurrences of the candidate derived alarm item Y is determined as a promotion degree between X and Y.
Step 506: and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
In an embodiment of the present invention, candidate root alarm items and corresponding candidate derived alarm items whose degree of lift is greater than a preset degree of lift threshold are determined as the root alarm items and the corresponding derived alarm items. Wherein, the preset threshold value of the lifting degree may be 1. The root alarm item is the alarm item that caused the derived alarm item.
The alarm analysis method provided by the embodiment of the invention is characterized in that after the original alarm data is subjected to transaction information extraction and inverted index processing, frequent item set mining is performed, and then root cause analysis is performed based on alarm sequence information of alarm items in the frequent item set, so that the alarm analysis efficiency can be improved.
Fig. 4 shows a schematic structural diagram of an alarm analysis device according to an embodiment of the present invention. As shown in fig. 4, the apparatus 600 includes: a first processing module 601, a second processing module 602, an analyzing module 603, a first determining module 604, and a second determining module 605, wherein,
the first processing module 601 is configured to: processing original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
the second processing module 602 is configured to: performing inverted indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed;
the analysis module 603 is configured to: performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
the first determining module 604 is configured to: determining alarm sequence information of each alarm item in the frequent item set;
the second determination module 605 is configured to: and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
In an alternative manner, the analysis module 603 is further configured to:
determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed;
1 is taken from k, and the candidate frequent k item set is screened according to the minimum support degree to obtain the frequent k item set of the alarm items; wherein k is a positive integer; the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set;
performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain alarm transaction item sets corresponding to candidate frequent k +1 item sets;
and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
In an alternative approach, the apparatus is based on a distributed stream data processing framework; the analysis module 603 is further configured to:
determining the calculation parallelism;
and performing parallel correlation analysis on the data set to be analyzed according to the distributed stream data processing framework and the calculation parallelism to obtain the frequent item set.
In an optional manner, the second determining module 605 is further configured to:
determining a maximum frequent item set in the frequent item sets;
determining a maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed;
respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information;
determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average sequence;
calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item;
and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
In an optional manner, the second determining module 605 is further configured to:
determining the alarm items for which the average order is greater than an order threshold as the candidate root alarm items;
determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
In an optional manner, the second determining module 605 is further configured to:
respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set;
determining the number of simultaneous occurrences of the candidate root alarm item and the candidate derived alarm item according to the number of occurrences;
and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
In an optional manner, the first processing module 601 is further configured to:
acquiring the original alarm data;
extracting the characteristics of the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items;
and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
The alarm analysis device provided by the embodiment of the invention extracts the transaction information and carries out inverted index processing on the original alarm data, then carries out frequent item set mining, and then carries out root cause analysis based on the alarm sequence information of the alarm items in the frequent item set, thereby being capable of improving the efficiency of alarm analysis.
Fig. 5 is a schematic structural diagram of an alarm analysis device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the alarm analysis device.
As shown in fig. 5, the alarm analysis device may include: a processor (processor) 702, a Communications Interface 704, a memory 706, and a communication bus 708.
Wherein: the processor 702, communication interface 704, and memory 706 communicate with each other via a communication bus 708. A communication interface 704 for communicating with network elements of other devices, such as clients or other servers. The processor 702 is configured to execute the program 710, and may specifically execute the relevant steps in the above embodiments of the alarm analysis method.
In particular, the program 710 may include program code comprising computer-executable instructions.
The processor 702 may be a central processing unit CPU, or an Application Specific Integrated Circuit ASIC (Application Specific Integrated Circuit), or one or more Integrated circuits configured to implement an embodiment of the present invention. The alarm analysis device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
The memory 706 stores a program 710. The memory 706 may comprise high-speed RAM memory, and may also include non-volatile memory (e.g., at least one disk memory).
The program 710 may be specifically invoked by the processor 702 to cause the alarm analysis device to perform the following operations:
processing original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
performing reverse indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed;
performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
determining alarm sequence information of each alarm item in the frequent item set;
and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed;
starting from k to 1, screening the candidate frequent k item set according to the minimum support degree to obtain a frequent k item set of the alarm items; wherein k is a positive integer; the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set;
performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain alarm transaction item sets corresponding to candidate frequent k +1 item sets;
and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
determining the calculation parallelism;
and performing parallel correlation analysis on the data set to be analyzed according to a distributed flow data processing framework and the calculation parallelism to obtain the frequent item set.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
determining a maximum frequent item set in the frequent item sets;
determining a maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed;
respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information;
determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average order;
calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item;
and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
determining the alarm items with the average order greater than an order threshold as the candidate root alarm items;
determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set;
determining the number of simultaneous occurrences of the candidate root alarm item and the candidate derived alarm item according to the number of occurrences;
and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
In an alternative manner, the program 710 is invoked by the processor 702 to cause the alert analysis device to:
acquiring the original alarm data;
extracting the characteristics of the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items;
and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
The alarm analysis equipment provided by the embodiment of the invention extracts the transaction information and carries out inverted index processing on the original alarm data, then carries out frequent item set mining, and then carries out root cause analysis based on the alarm sequence information of the alarm items in the frequent item set, thereby being capable of improving the efficiency of alarm analysis.
An embodiment of the present invention provides a computer-readable storage medium, where the storage medium stores at least one executable instruction, and when the executable instruction runs on an alarm analysis device, the alarm analysis device is enabled to execute an alarm analysis method in any method embodiment described above.
The executable instructions may be specifically configured to cause the alarm analysis device to perform the following operations:
processing original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
performing inverted indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed;
performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
determining alarm sequence information of each alarm item in the frequent item set;
and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
In an alternative, the executable instructions cause the alert analysis device to:
determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed;
1 is taken from k, and the candidate frequent k item set is screened according to the minimum support degree to obtain the frequent k item set of the alarm items; wherein k is a positive integer; the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set;
performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain alarm transaction item sets corresponding to candidate frequent k +1 item sets;
and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
In an alternative, the executable instructions cause the alert analysis device to:
determining the calculation parallelism;
and performing parallel correlation analysis on the data set to be analyzed according to the distributed stream data processing framework and the calculation parallelism to obtain the frequent item set.
In an alternative form, the executable instructions cause the alert analysis device to:
determining a maximum frequent item set in the frequent item sets;
determining a maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed;
respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information;
determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average order;
calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item;
and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
In an alternative form, the executable instructions cause the alert analysis device to:
determining the alarm items with the average order greater than an order threshold as the candidate root alarm items;
determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
In an alternative, the executable instructions cause the alert analysis device to:
respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set;
determining the number of simultaneous occurrences of the candidate root alarm item and the candidate derived alarm item according to the number of occurrences;
and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
In an alternative, the executable instructions cause the alert analysis device to:
acquiring the original alarm data;
performing feature extraction on the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items;
and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
The computer storage medium provided by the embodiment of the invention is characterized in that the problem of low alarm analysis efficiency caused by adopting root cause analysis according to a rule set by people in the prior art is solved, the original alarm data is subjected to transaction information extraction and inverted index processing, then frequent item set mining is carried out, and then the root cause analysis is carried out based on the alarm sequence information of the alarm items in the frequent item set, so that the alarm analysis efficiency can be improved.
The embodiment of the invention provides an alarm analysis device, which is used for executing the alarm analysis method.
Embodiments of the present invention provide a computer program that can be invoked by a processor to enable an alarm analysis device to execute an alarm analysis method in any of the above method embodiments.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer-readable storage medium, the computer program comprising program instructions that, when run on a computer, cause the computer to perform the alarm analysis method of any of the above-described method embodiments.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system is apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.

Claims (10)

1. An alarm analysis method, characterized in that the method comprises:
processing original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
performing reverse indexing on the alarm transaction items according to the alarm items to obtain a data set to be analyzed;
performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
determining alarm sequence information of each alarm item in the frequent item set;
and determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
2. The method of claim 1, wherein performing correlation analysis on the data set to be analyzed to obtain a frequent item set comprises:
determining each alarm item as each candidate frequent 1 item set of the data set to be analyzed;
1 is taken from k, and the candidate frequent k item set is screened according to the minimum support degree to obtain the frequent k item set of the alarm items; wherein k is a positive integer; the support degree of each candidate frequent k item set is determined according to the length of the alarm transaction item set corresponding to the candidate frequent k item set;
performing intersection operation on the alarm transaction item sets corresponding to the frequent k item sets to obtain alarm transaction item sets corresponding to candidate frequent k +1 item sets;
and performing iterative updating on k until a frequent k +1 item set or a candidate frequent k +1 item set does not exist, and determining the frequent k item set as the frequent item set.
3. The method of claim 1, wherein the method is based on a distributed stream data processing framework; performing correlation analysis on the data set to be analyzed to obtain a frequent item set, including:
determining the calculation parallelism;
and performing parallel correlation analysis on the data set to be analyzed according to the distributed stream data processing framework and the calculation parallelism to obtain the frequent item set.
4. The method according to claim 1, wherein the determining alarm item root relations corresponding to the frequent item sets according to the alarm order information comprises:
determining a maximum frequent item set in the frequent item sets;
determining a maximum alarm transaction item set corresponding to the maximum frequent item set according to the data set to be analyzed;
respectively determining the average sequence of each alarm item in the maximum frequent item set in the maximum alarm transaction item set according to the alarm sequence information;
determining at least one candidate root alarm item and at least one candidate derivative alarm item in the maximum frequent item set according to the average order;
calculating the promotion degree of each candidate root alarm item and each candidate derivative alarm item;
and respectively determining the root alarm item and the derivative alarm item in the maximum frequent item set from each candidate root alarm item and each candidate derivative alarm item according to the promotion degree.
5. The method of claim 4, wherein determining the candidate root alarm term and the candidate derived alarm term in the maximum frequent item set according to the average order comprises:
determining the alarm items with the average order greater than an order threshold as the candidate root alarm items;
and determining the alarm items remaining in the maximum frequent item set as the candidate derived alarm items.
6. The method according to claim 4, wherein the calculating the promotion degree of the candidate root alarm item and the candidate derived alarm item comprises:
respectively determining the occurrence times of the candidate root alarm item and the candidate derivative alarm item in each alarm transaction item of the maximum alarm transaction item set;
determining the number of simultaneous occurrences of the candidate root alarm item and the candidate derived alarm item according to the number of occurrences;
and determining the promotion degree according to the simultaneous occurrence times and the occurrence times of the candidate derived alarm items.
7. The method according to claim 1, wherein the processing of the original alarm data results in a plurality of alarm transaction items; the method comprises the following steps:
acquiring the original alarm data;
performing feature extraction on the original alarm data to obtain candidate alarm transaction data; the candidate alarm transaction data comprises a plurality of alarm items;
and processing the candidate alarm transaction data according to the sliding time window and the sliding step length to obtain the plurality of alarm transaction items.
8. An alarm analysis apparatus, characterized in that the apparatus comprises:
the first processing module is used for processing the original alarm data to obtain a plurality of alarm transaction items; each alarm transaction item comprises at least one alarm item arranged according to alarm time sequence;
the second processing module is used for performing reverse indexing on the alarm transaction item according to the alarm item to obtain a data set to be analyzed;
the analysis module is used for performing correlation analysis on the data set to be analyzed to obtain a frequent item set;
the first determining module is used for determining the alarm sequence information of each alarm item in the frequent item set;
and the second determining module is used for determining the alarm item root relation corresponding to the frequent item set according to the alarm sequence information.
9. An alarm analysis device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the alarm analysis method of any of claim 7.
10. A computer-readable storage medium having stored therein at least one executable instruction that, when executed on an alarm analysis device, causes the alarm analysis device to perform the operations of the alarm analysis method of any one of claims 7.
CN202111144543.7A 2021-09-28 2021-09-28 Alarm analysis method, device, equipment and computer storage medium Pending CN115883323A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111144543.7A CN115883323A (en) 2021-09-28 2021-09-28 Alarm analysis method, device, equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111144543.7A CN115883323A (en) 2021-09-28 2021-09-28 Alarm analysis method, device, equipment and computer storage medium

Publications (1)

Publication Number Publication Date
CN115883323A true CN115883323A (en) 2023-03-31

Family

ID=85763618

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111144543.7A Pending CN115883323A (en) 2021-09-28 2021-09-28 Alarm analysis method, device, equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN115883323A (en)

Similar Documents

Publication Publication Date Title
US20170206458A1 (en) Computer-readable recording medium, detection method, and detection apparatus
CN112364014B (en) Data query method, device, server and storage medium
CN114741544B (en) Image retrieval method, retrieval library construction method, device, electronic equipment and medium
CN110888981A (en) Title-based document clustering method and device, terminal equipment and medium
CN112328805A (en) Entity mapping method of vulnerability description information and database table based on NLP
CN111400448A (en) Method and device for analyzing incidence relation of objects
CN110083731B (en) Image retrieval method, device, computer equipment and storage medium
CN108875050B (en) Text-oriented digital evidence-obtaining analysis method and device and computer readable medium
CN113505278A (en) Graph matching method and device, electronic equipment and storage medium
CN113901037A (en) Data management method, device and storage medium
CN110874366A (en) Data processing and query method and device
US20160292258A1 (en) Method and apparatus for filtering out low-frequency click, computer program, and computer readable medium
CN111125226B (en) Configuration data acquisition method and device
CN116074183B (en) C3 timeout analysis method, device and equipment based on rule engine
CN116955856A (en) Information display method, device, electronic equipment and storage medium
CN115883323A (en) Alarm analysis method, device, equipment and computer storage medium
CN110543426A (en) software performance risk detection method and device
CN106682107B (en) Method and device for determining incidence relation of database table
CN115344627A (en) Data screening method and device, electronic equipment and storage medium
CN113946717A (en) Sub-map index feature obtaining method, device, equipment and storage medium
CN114528378A (en) Text classification method and device, electronic equipment and storage medium
CN112836747A (en) Eye movement data outlier processing method and device, computer equipment and storage medium
CN107247796B (en) Client head portrait loading method and device and terminal equipment
CN112671593B (en) Server management method and related equipment
CN110633430A (en) Event discovery method, device, equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination