RU2020137974A

RU2020137974A - SYSTEM, DEVICE AND METHOD FOR COMPRESSING ALARM REGISTRATION LOG AND DATA CARRIER

Info

Publication number: RU2020137974A
Application number: RU2020137974A
Authority: RU
Inventors: Цзиюань ШИ; Лян ЧЖАН; Шихао ЛИ; Дэвэй БАО; Цзянь Ли
Original assignee: Хуавей Текнолоджиз Ко., Лтд.
Priority date: 2018-04-23
Filing date: 2018-12-27
Publication date: 2022-05-19

Claims

1. A method for compressing an alarm log, comprising the steps of:

receiving a set of alarm log statistics generated by the first network device in the communication network, wherein the alarm log statistics set contains a plurality of alarm log statistics, and each alarm log statistics contains an alarm type and a time stamp generating;

separating the set of alarm log statistics into a plurality of subsets of alarm log statistics based on the generation timestamps of the alarm log statistics in the alarm log statistics set, all of the alarm log statistics in each subset of statistics the alarm logs are sequential in time sequence, and the combined set of the plurality of subsets of alarm statistics contains all of the alarm logs' statistics in the alarm log statistics set;

determining a correspondence between the type of alarms in the alarm log statistics set and the plurality of alarm log statistics subsets;

performing clustering processing on alarm types in the alarm log statistics set, based on a match, to generate at least one association rule, each association rule containing a major alarm type and at least one minor alarm type associated with each other. with friend; wherein said at least one association rule is used to

compressing the alarm logs to be processed to obtain an alarm log having an alarm type that is the root cause alarm type.

2. The method of claim 1, wherein the step of determining a correspondence between an alarm type in a set of alarm log statistics and a plurality of alarm log statistics subsets comprises sub-steps of:

getting all types of alarms in the alarm log statistics set to obtain a first set of alarm types; and

defining a time sequence vector of each alarm type in the first set of alarm types, each time sequence vector being used to reflect the correspondence between the corresponding alarm type and a plurality of alarm log statistics subsets, wherein

for the time sequence vector corresponding to each type of alarms, the values in the time sequence vector are in a one-to-one correspondence with a plurality of alarm log statistics subsets, the values in the time sequence vector contain at least one of a first value and a second value, where the first the value is used to indicate that the alarm type exists in the corresponding subset of the alarm statistics logs, and the second value is used to indicate that the alarm type does not exist in the corresponding subset of alarm logs and the first value is different from the second value.

3. The method of claim 2, wherein the step of determining the time sequence vector of each alarm type in the first set of alarm types comprises a sub-step of:

performing the procedure for determining the time sequence vector for each type of alarms in the first set of types of alarms, and

the procedure for determining the time sequence vector contains the steps in which:

successively determine the presence of the type of alarms in a plurality of subsets of statistical data of the alarm logs; and

determining an alarm type time sequence vector based on the detection result.

4. The method according to claim 2 or 3, wherein the step of performing clustering processing on the alarm types in the set of alarm log statistics based on the match to generate at least one association rule comprises a sub-step of:

performing clustering processing for all alarm types in the first alarm type set based on the time sequence vectors of all alarm types to generate at least one association rule.

5. The method of claim 4, wherein the step of performing clustering processing on all alarm types in the first set of alarm types, based on the time sequence vectors of all alarm types, to generate at least one association rule, comprises a sub-step of:

perform clustering operations for the first set of alarm types, wherein the clustering operations comprise:

setting a target alarm type set and a second alarm type set, both the target alarm type set and the second alarm type set being empty sets;

adding any alarm type in the first alarm type set to the target alarm type set and deleting from the first alarm type set the alarm type added to the target alarm type set;

repeatedly performing the determination procedure until the first set of alarm types is empty, and determining the target set of alarm types as an association rule; and

repeatedly executing, after repeatedly executing the determination procedure when the second set of alarm types is not an empty set, clustering operations using the second set of alarm types as a new first set of alarm types; or

terminating, after repeatedly executing the determination procedure when the second set of alarm types is an empty set, performing a clustering operation; wherein

The determination procedure contains the steps in which:

computing a correlation between the alarm type to be processed and the target alarm type set based on the alarm type time sequence vector to be processed and the time sequence vectors of all alarm types in the target alarm type set, the alarm type to be processed , is any alarm type in the first alarm type set other than the alarm types in the target alarm type set; and

when the correlation exceeds a predetermined correlation threshold, adding an alarm type to be processed to the target alarm type set to obtain an updated target alarm type set, and deleting the alarm type to be processed from the first alarm type set; or

when the correlation does not exceed the predetermined correlation threshold, adding an alarm type to be processed to the second alarm type set, and removing the alarm type to be processed from the second alarm type set.

6. The method of claim 4, wherein the step of performing clustering processing on all alarm types in the first set of alarm types, based on the time sequence vectors of all alarm types, to generate at least one association rule, comprises a sub-step of:

marking a target alarm type in the first alarm type set, wherein the target alarm type is any alarm type in the first alarm type set; and

perform clustering operations for the first e first set of alarm types, in which the clustering operation comprises:

executing the determination procedure repeatedly until all alarm types in the first set of alarm types have been viewed; and

after running the determination procedure repeatedly, if there is an alarm type that is unchecked in the first set of alarm types, defining any unchecked alarm type as the new target alarm type, marking the new target alarm type, and repeatedly performing the clustering operation, in which different target types of alarms have different marks; or

after repeatedly executing the determination procedure, in the absence of an alarm type that is not checked in the first set of alarm types, terminating the clustering operation and generating at least one association rule based on the first set of alarm types, in which one and the same checkbox is set for all alarm types in each association rule; a

The determination procedure contains the steps in which:

determining, as the target alarm type set, a set containing all alarm types in the first alarm type set having the same mark as the target alarm type;

calculating the correlation between the type of alarms to be processed and the target set of alarm types based on the vector of the time sequence of the type of alarms to be processed, and the time sequence of all types of alarms in the target set of alarm types, and the type of alarms to be processed, is any alarm type in the first alarm type set other than the alarm types in the target alarm type set; and

marking the type of alarms to be processed when the correlation exceeds the predetermined correlation threshold, wherein the flag of the alarm type to be processed is the same as the flag of the target alarm type.

7. The method according to claim 5 or 6, wherein the step of calculating the correlation between the type of alarms to be processed and the target set of alarm types based on the time sequence vector of the type of alarms to be processed and the time sequence vectors of all types of alarms in the target set alarm types contains sub-steps in which:

calculating the correlation between the alarm type to be processed and each alarm type in the set of target alarm types according to the Pearson correlation coefficient formula, the Pearson correlation coefficient formula being:

, where

represents the correlation between the type

alarm and type

alarm,

represents the covariance between

and

,

represents the standard deviation

,

represents the standard deviation

,

represents a time sequence vector of type

alarm and

represents a time sequence vector of type

emergency signal; and

calculating a correlation between the type of alarms to be processed and the target set of alarm types based on the correlation between the type of alarms to be processed and each type of alarms in the set of target alarm types according to the average correlation calculation formula, wherein the average correlation calculation formula correlations:

, where

indicates the correlation between the type

alarm and the target set R of alarm types, and

indicates the number of alarm types in the target set R of alarm types.

8. The method according to any one of claims 1 to 7, wherein the step of obtaining a set of alarm log statistics generated by the first network device in the communication network comprises a sub-step of:

preprocessing the alarm log statistics generated by the first network device in the first predetermined time period to remove redundant information from each alarm log statistics to obtain a set of alarm log statistics.

9. The method according to any one of claims 1 to 8, wherein each alarm log statistics is in a two-tuple format (M, t), where M is alarm log information and t is a generation timestamp. , wherein the alarm log information includes at least an alarm type field and a first network device identifier field.

10. The method of any one of claims 1 to 9, wherein the alarm log statistics in the alarm log set have a partial temporal order relationship, and the step of dividing the alarm log statistics set into a plurality of subsets of the alarm log statistics alarms based on the timestamps of generating the alarm log statistics in the alarm log statistics set, comprises sub-steps of:

obtaining a set of timestamps for generating alarm log statistics in the alarm log statistics set; and

classifying the statistics of the alarm logs into a plurality of subsets of the statistics of the alarm logs based on the set of generation timestamps, the specified time window duration, and the sliding window step using the sliding window technology, wherein the sliding window step does not exceed the time window duration.

11. The method according to any one of claims 1 to 10, further comprising the step of:

compressing the plurality of alarm logs to be processed based on at least one association rule to obtain an alarm log having an alarm type of the root cause alarm type.

12. The method of claim 11, further comprising, before the step of compressing the plurality of alarm logs, the steps of:

receiving, to be processed, a set of alarm logs generated by a second network device in the communication network in a second predetermined time period, wherein the set of alarm logs to be processed contains a plurality of alarm logs to be processed, and each alarm log, to be processed, contains the type of alarm and the timestamp of the generation; and

generating at least one alarm event based on the set of alarm logs to be processed, each alarm event being used to indicate the same type of alarm logs generated by the second network device; wherein

compressing the plurality of alarm logs comprises a sub-step of

compressing at least one alarm event based on at least one association rule to obtain at least one target alarm event, each target alarm event being used to specify an alarm log having an alarm type being an alarm type main reason.

13. The method of claim 12, wherein the step of generating at least one alarm event based on the set of alarm logs to be processed comprises sub-steps of:

obtaining, for each type of alarms in the set of alarm logs to be processed, a target alarm log in the set of alarm logs to be processed and related to the type of alarms;

performing a separate calculation of an occurrence time interval between each two target alarm logs in the set of alarm logs to be processed that are adjacent in time sequence; and

restoring, in one alarm event, based on the time interval of occurrence between every two target alarm logs, the target alarm logs in the set of alarm logs to be processed and related to one alarm event.

14. The method of claim 13, wherein the step of restoring, in one alarm event, based on the time interval of occurrence between every two target alarm logs, the target alarm logs in the set of alarm logs to be processed, and belonging to a single alarm event, contains sub-steps in which:

calculating an estimated time interval between two target alarm logs using an exponential moving average method, wherein the two target alarm logs are any two target alarm logs adjacent in time sequence, where the two target alarm logs comprising a first alarm log and a second alarm log, the first alarm log being generated prior to the second alarm log;

determining whether an occurrence time interval between the two target alarm logs and the estimated time interval satisfies a predetermined condition; and

determining when the occurrence time interval between the two target alarm logs and the estimated time interval satisfies the predetermined condition that the second alarm log belongs to the alarm event to which the first alarm log belongs, and aggregates the second alarm log into an event alarms, which owns the first alarm log; or

determining when an occurrence time interval between two target alarm logs and the estimated time interval does not satisfy a predetermined condition that the second alarm log does not belong to the alarm event to which the first alarm log belongs.

15. The method according to claim 14, in which the interval

time of occurrence, between two target alarm logs, is

, estimated time interval

is

, and the given condition is

, where

is the timestamp when the second alarm log was generated,

is the timestamp of the generation of the first alarm log,

, k is an integer greater than 1 and

is a positive number.

16. The method of claim 15, further comprising, prior to the step of determining whether an occurrence time interval between two target alarm logs and the estimated time interval satisfies a predetermined condition, comprises sub-steps of:

determining when an occurrence time interval between two target alarm logs is less than or equal to a predetermined minimum time interval threshold value that the second alarm log belongs to an alarm event to which the first alarm log belongs; or

determining when a time interval of occurrence between two target alarm logs exceeds a predetermined maximum time interval threshold value that the second alarm log does not belong to the alarm event to which the first alarm log belongs, wherein

,

is the minimum time interval threshold, and

is the maximum time interval threshold.

17. The method of any one of claims 14 to 16, further comprising, after determining that the second alarm log does not belong to the alarm event to which the first alarm log belongs, the steps of:

completing restoring the alarm event to which the first alarm log belongs and initializing a new alarm event; and

aggregating the second alarm log into a new alarm event.

18. The method of any one of claims 12-17, wherein the alarm event comprises an alarm type and at least one of an occurrence start time, an occurrence end time, an average time interval of occurrence, and a number of times the alarm log occurs in the alarm event. signals.

19. The method of any one of claims 12-18, further comprising, after compressing at least one alarm event to obtain at least one target alarm event, the step of:

outputting at least one alarm target event.

20. The method according to any one of claims 12-19, in which

the second network device and the first network device are one network device; or

the second network device and the first network device are different network devices of the same type.

21. An alarm log compression device, comprising:

a first obtaining module configured to obtain a set of alarm log statistics generated by a first network device in the communication network, wherein the set of alarm log statistics contains a plurality of alarm log statistics, and each alarm log statistics comprises alarm type and timestamp of generation;

a separating module configured to divide the set of alarm log statistics into a plurality of subsets of alarm log statistics based on the time stamps of generating the alarm log statistics in the set of alarm log statistics, wherein all of the alarm log statistics the alarms in each alarm log statistics subset are consecutive in time sequence, and the combined set of the plurality of alarm log statistics subsets contains all of the alarm log statistics in the alarm log statistics set;

a determining module, configured to determine a correspondence between an alarm type in the set of alarm log statistics and a plurality of subsets of alarm log statistics;

a clustering module configured to perform clustering processing on alarm types in a set of alarm log statistics based on a match to generate at least one association rule, each association rule comprising a major alarm type and at least one minor alarm type signals associated with each other; wherein said at least one association rule is used to compress the alarm logs to be processed to obtain an alarm log having an alarm type that is a root cause alarm type.

22. The apparatus of claim 21, wherein the determination module comprises:

a receiving sub-module configured to obtain all types of alarms in the alarm log statistics set to obtain a first set of alarm types; and

a determination sub-module configured to determine a time sequence vector of each alarm type in the first set of alarm types, each time sequence vector being used to reflect a correspondence between the corresponding alarm type and a plurality of subsets of alarm log statistics, wherein

for the time sequence vector corresponding to each alarm type, the values in the time sequence vector are in a one-to-one correspondence with the set of alarm log statistics subsets, the values in the time sequence vector contain at least one of the first value and the second value, the first value is used to indicate that the alarm type is in the corresponding alarm log statistics subset, and the second value is used to indicate that the alarm type is not in the corresponding alarm log statistics subset, with the first value different from the second value.

23. The apparatus of claim 22, wherein the determination submodule is configured to:

implementation of the procedure for determining the time sequence vector for each type of alarms in the first set of types of alarms, while

performing a sequential determination of the presence of the type of alarms in a plurality of subsets of statistical data logs of the alarms; and

determining an alarm type time sequence vector based on the detection result.

24. The device according to claim 22 or 23, in which the clustering module is configured to:

performing clustering processing for all alarm types of the first set of alarm types, based on the time sequence vectors of all alarm types in accordance with the clustering algorithm associated with the time sequence, to generate at least one association rule.

25. The apparatus of claim 24, wherein the clustering module is further configured to:

performing a clustering operation for a first set of alarm types, the clustering operation comprising:

adding any alarm type in the first alarm type set to the target alarm type set and deleting, from the first alarm type set, the alarm type added to the target alarm type set;

repeatedly performing the determination procedure until the first set of alarm types becomes an empty set, and determining the target set of alarm types as an association rule; and

terminating the execution of the clustering operation after repeatedly executing the determination procedure when the second set of alarm types is an empty set; a

the determination procedure contains:

calculating a correlation between the alarm type to be processed and the target alarm type set, based on the time sequence vector of the alarm type to be processed and the time sequence vectors of all alarm types in the target alarm type set, wherein the alarm type to be processed alarm type is any alarm type in the first alarm type set other than the alarm types in the target alarm type set; and

when the correlation does not exceed the predetermined correlation threshold, adding the type of alarms to be processed to the second set of alarm types, and deleting the type of alarms to be processed from the first set of alarm types.

26. The apparatus of claim 24, wherein the clustering module is further configured to:

performing a clustering operation for a first set of alarm types, wherein the clustering operation comprises:

after repeating the determination procedure, if there is an alarm type that is unchecked in the first set of alarm types, determining, as a new target alarm type, any unchecked alarm type, marking the new target alarm type signals and repeatedly performing the clustering operation, while different types of target alarms have different marks; or

after repeatedly executing the determination procedure, if there is no alarm type that is not flagged in the first set of alarm types, terminating the clustering operation and generating at least one association rule based on the first set of alarm types, with the same a flag is set for all types of alarms in each association rule; a

the determination procedure contains:

defining as the target alarm type set a set containing all alarm types in the first alarm type set having the same mark as the target alarm type;

calculating a correlation between the alarm type to be processed and the target alarm type set based on the alarm type time sequence vector to be processed and the time sequence vectors of all alarm types in the target alarm type set, the alarm type to be processed , is any alarm type in the first alarm type set other than the alarm types in the target alarm type set; and

flag of the type of alarms to be processed when the correlation exceeds the predetermined correlation threshold, wherein the flag of the type of alarms to be processed is the same as the flag of the target alarm type.

27. The device according to claim 25 or 26, wherein the clustering module is further configured to:

calculating the correlation between the alarm type to be processed and each alarm type in the target alarm type set according to the Pearson correlation coefficient formula, where the Pearson correlation coefficient formula is:

, where

represents the correlation between the type

alarm and type

alarm,

represents the covariance between

and

,

represents the standard deviation

,

represents the standard deviation

,

represents a time sequence vector of type

alarm and

represents a time sequence vector of type

emergency signal; and

calculating the correlation between the alarm type to be processed and the target alarm type set based on the correlation between the alarm type to be processed and each alarm type in the target alarm type set according to the average correlation calculation formula, wherein the average correlation calculation formula correlations:

, where

indicates the correlation between the type

alarm and the target set R of alarm types, and

indicates the number of alarm types in the target set R of alarm types.

28. An apparatus according to any one of claims 21-27, wherein the first acquisition module is configured to:

pre-processing the alarm log statistics generated by the first network device in the first predetermined time period to remove redundant information from each alarm log statistics to obtain a set of alarm log statistics.

29. The apparatus of any one of claims 21 to 28, wherein each alarm log statistics are in a two-tuple format (M, t), where M is alarm log information and t is a timestamp of generation and the alarm log information includes at least an alarm type field and an alarm network device ID field.

30. An apparatus according to any one of claims 21 to 29, wherein the alarm log statistics in the alarm log statistics set have a partial temporal order relationship, wherein the separating module is configured to:

obtaining a set of alarm log statistics generation timestamps in the alarm log statistics set; and

classifying the alarm log statistics into a plurality of subsets of the alarm log statistics based on a set of generation timestamps, a predetermined time window length, and a sliding window step, wherein the sliding window step does not exceed the time window length.

31. The apparatus of any one of claims 21 to 30, further comprising a compression module configured to compress a plurality of alarm logs to be processed based on at least one association rule to obtain an alarm log having an alarm type of root cause alarms.

32. The device according to claim 31, further comprising:

a second acquisition module configured to obtain a set of alarm logs to be processed generated by a second network device in the communication network in a second predetermined time period, wherein the set of alarm logs to be processed comprises a plurality of alarm logs to be processed, and each to be processed processing, the alarm log contains the type of alarms and the timestamp of the generation; and

a generating module configured to generate at least one alarm event based on the set of alarm logs to be processed, each alarm event being used to indicate the same type of alarm logs as those generated by the second network device; wherein

the compression module is configured to:

compressing at least one alarm event based on at least one association rule to obtain at least one target alarm event, each target alarm event being used to specify an alarm log having an alarm type that is an alarm type of primary causes.

33. The apparatus of claim 32, wherein the generating module comprises:

a recovery sub-module configured to perform an alarm event recovery procedure for each set of alarm logs to be processed, the alarm event recovery process comprising:

the receiving unit is configured to: for each type of alarms in the set of alarm logs to be processed, obtaining a target alarm log in the set of alarm logs to be processed related to the type of alarms;

a calculation unit configured to separately calculate an occurrence time interval between each two target alarm logs in the set of alarm logs to be processed that are adjacent in time sequence; and

a recovery unit configured to recover, into one alarm event, based on the time interval of occurrence between each two target alarm logs, the target alarm logs in the set of alarm logs to be processed and belonging to the same alarm event signals.

34. The apparatus of claim 33, wherein the recovery unit is configured to:

calculating an estimated time interval between two target alarm logs using an exponential moving average method, where the two target alarm logs are any two target alarm logs adjacent in time sequence, the two target alarm logs contain the first log alarm logs and a second alarm log, and the first alarm log is generated before the second alarm log;

determining whether a time interval of occurrence between the two target alarm logs and the estimated time interval satisfies a predetermined condition; and

determining when an occurrence time interval, between the two target alarm logs, and the estimated time interval satisfies a predetermined condition that the second alarm log belongs to the alarm event to which the first alarm log belongs, and aggregating the second alarm log into an event alarms, which owns the first alarm log; or

determining when the occurrence time interval between the two target alarm logs and the estimated time interval does not satisfy the predetermined condition that the second alarm log does not belong to the alarm event to which the first alarm log belongs.

35. The apparatus of claim 34, wherein the recovery unit is further configured to:

completing recovery of the alarm event to which the first alarm log belongs, and initiating a new alarm event; and

aggregating the second alarm log into a new alarm event.

36. The apparatus of any one of claims 32 to 35, wherein the alarm event comprises an alarm type and at least one of an occurrence start time, an occurrence end time, an average time interval of occurrence, and a number of times the alarm log occurs in the alarm event .

37. The device according to any one of claims 32-36, in which the device further comprises:

an output module configured to output at least one alarm target event.

38. The device according to any one of paragraphs.32-37, in which the second network device and the first network device are one network device; or

39. A computer-readable storage medium storing instructions causing, when executed by the processor, an implementation of the alarm log compression method according to any one of claims 1-20.