CN116910624A - Abnormality index detection method and device, storage medium and electronic equipment - Google Patents

Abnormality index detection method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN116910624A
CN116910624A CN202310920989.7A CN202310920989A CN116910624A CN 116910624 A CN116910624 A CN 116910624A CN 202310920989 A CN202310920989 A CN 202310920989A CN 116910624 A CN116910624 A CN 116910624A
Authority
CN
China
Prior art keywords
indexes
index
sample
determining
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310920989.7A
Other languages
Chinese (zh)
Inventor
曹诗苑
陆顺
刘汉生
赵龙刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Technology Innovation Center
China Telecom Corp Ltd
Original Assignee
China Telecom Technology Innovation Center
China Telecom Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Technology Innovation Center, China Telecom Corp Ltd filed Critical China Telecom Technology Innovation Center
Priority to CN202310920989.7A priority Critical patent/CN116910624A/en
Publication of CN116910624A publication Critical patent/CN116910624A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/27Regression, e.g. linear or logistic regression
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Abstract

The disclosure relates to the technical field of computers, and in particular relates to a method and a device for detecting abnormal indexes, a computer readable storage medium and electronic equipment, wherein the method comprises the following steps: acquiring a plurality of sample index sequences, and clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences; wherein the class cluster comprises a plurality of indexes; determining the stable relation among indexes in various clusters; monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with the stable relation among the indexes in the plurality of indexes according to the values of the plurality of indexes; an anomaly index is determined based on the plurality of candidate index combinations. By the technical scheme of the embodiment of the disclosure, the problem of low efficiency of checking abnormal indexes in the related technology can be solved.

Description

Abnormality index detection method and device, storage medium and electronic equipment
Technical Field
The present disclosure relates to the field of computer communication technology, and in particular, to an abnormality index detection method, an abnormality index detection device, a computer-readable storage medium, and an electronic apparatus.
Background
With the rapid development of software and hardware, services such as network high-definition monitoring, intelligent cloud network, video networking and the like are gradually applied. Currently, these services have weak operation and maintenance capabilities, and are usually performed in a conventional manner.
In the related art, an anomaly threshold is generally set based on an empirical value, and when the value of an index related to a service is greater than the anomaly threshold, an anomaly alarm is issued, and since a single failure generally causes a plurality of indexes to be anomalous, each anomaly index needs to be checked one by one to determine a failure index.
In the scheme in the related art, an abnormal threshold value needs to be set based on an experience value, manual experience is relied on, a plurality of abnormal indexes are checked, a large amount of time cost and labor cost are needed, and the fault processing efficiency is low.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to an abnormality index detection method, an abnormality index detection apparatus, a computer-readable storage medium, and an electronic device, which can solve the problem of low efficiency in checking an abnormality index in the related art.
Other features and advantages of the present disclosure will be apparent from the following detailed description, or may be learned in part by the practice of the disclosure.
According to a first aspect of the present disclosure, there is provided an abnormality index detection method including: acquiring a plurality of sample index sequences, and clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences; wherein the class cluster comprises a plurality of indexes; determining the stable relation among indexes in various clusters; monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with the stable relation among the indexes in the plurality of indexes according to the values of the plurality of indexes; an anomaly index is determined based on the plurality of candidate index combinations.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, obtaining a plurality of sample index sequences includes: acquiring sample data corresponding to a plurality of indexes in a preset time period, and acquiring abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index; and carrying out smoothing treatment on abnormal data of the sample data corresponding to each index, and carrying out filling treatment on missing data of the sample data corresponding to each index to obtain a plurality of sample index sequences.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, clustering, according to a plurality of sample index sequences, indexes corresponding to the plurality of sample index sequences into a plurality of class clusters includes: determining the mutual reachable distance between the indexes according to a plurality of sample index sequences; determining the association relation among the indexes according to the mutual reachable distance among the indexes; in the association relation among the indexes, the reachable distance among the indexes is shortest; and clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relation among the indexes.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, clustering, according to an association relationship between indexes, indexes corresponding to a plurality of sample index sequences into a plurality of class clusters includes: hierarchical clustering is carried out according to the association relation among the indexes to obtain a plurality of hierarchical initial class clusters; determining the stability of each initial cluster and the density of each initial cluster; and clustering indexes corresponding to the sample index sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, determining an association relationship between the indexes according to the mutually reachable distances between the indexes includes: constructing a minimum spanning tree according to the mutual reachable distance between the indexes; the minimum spanning tree is used for indicating the association relation among the indexes.
In one exemplary embodiment of the present disclosure, based on the foregoing scheme, determining a plurality of candidate index combinations that do not meet a stable relationship between the indices among the plurality of indices according to values of the plurality of indices includes: determining a stability threshold value between the indexes according to the stability relation between the indexes; when the residual between the values of the two indices is greater than or equal to the stability threshold between the indices, the two indices are determined to be candidate index combinations.
In one exemplary embodiment of the present disclosure, based on the foregoing scheme, determining the abnormality index from the plurality of candidate index combinations includes: connecting the same candidate indexes in the candidate index combinations to establish a directed graph; calculating the weight value of each node in the directed graph; the weight value of the node is used for indicating the probability that the candidate index corresponding to the node is an abnormal index; and determining an abnormal index from the candidate indexes according to the weight value of each node.
According to a second aspect of the present disclosure, there is provided an abnormality index detection apparatus including: the index sequence acquisition module is used for acquiring a plurality of sample index sequences and clustering indexes corresponding to the sample index sequences into a plurality of class clusters according to the sample index sequences; wherein the class cluster comprises a plurality of indexes; the stability relation determining module is used for determining stability relations among indexes in various clusters; a candidate combination determining module, configured to monitor a plurality of indexes, and determine a plurality of candidate index combinations that do not conform to a stable relationship between the indexes among the plurality of indexes according to values of the plurality of indexes; the abnormal index determining module is used for determining abnormal indexes according to the candidate index combinations.
According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the abnormality index detection method as in the first aspect of the above-described embodiments.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:
one or more processors; and
and a storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the abnormality index detection method as in the first aspect of the above embodiment.
The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:
in the abnormal index detection method provided by the embodiment of the disclosure, a plurality of sample index sequences can be obtained, indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences, stable relations among the indexes in the class clusters are determined, the plurality of indexes are monitored, a plurality of candidate index combinations which do not accord with the stable relations among the indexes are determined in the plurality of indexes according to the values of the plurality of indexes, and the abnormal index is determined according to the plurality of candidate index combinations. According to the scheme, the stable relation between similar indexes can be determined according to the existing data, and the abnormal indexes can be determined when the value of the detection index does not meet the stable relation, so that on one hand, an abnormal threshold value is not required to be set based on an empirical value, and manual experience is not required to be relied on; on the other hand, the method does not need to check a plurality of abnormal indexes, and avoids consuming a great deal of time cost and labor cost, so that the determination efficiency of the abnormal indexes is improved, and the failure processing efficiency is further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:
FIG. 1 schematically illustrates a schematic diagram of an exemplary system architecture to which an anomaly index detection method of embodiments of the present disclosure may be applied;
FIG. 2 schematically illustrates a flowchart of an anomaly index detection method in an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a flowchart of filling missing data of sample data corresponding to each index to obtain a plurality of sample index sequences in an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a flowchart of clustering indicators corresponding to a plurality of sample indicator sequences into a plurality of class clusters according to an association relationship between the indicators in an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates a flowchart of clustering metrics corresponding to a plurality of sample metric sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster in an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a flow chart of determining two metrics as candidate metric combinations when a residual between values of the two metrics is greater than or equal to a stability threshold between the metrics in an exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a flow chart for determining an anomaly index from among a plurality of candidate indexes according to weight values of nodes in an exemplary embodiment of the present disclosure;
FIG. 8 schematically illustrates a flowchart of another abnormality index detection method in an exemplary embodiment of the present disclosure;
FIG. 9 schematically illustrates a flow chart for determining stable relationships between indicators in various clusters in an exemplary embodiment of the present disclosure;
FIG. 10 schematically illustrates a flow chart of an abnormal indicator alert dispatch in an exemplary embodiment of the present disclosure;
fig. 11 schematically illustrates a composition diagram of an abnormality index detection device in an exemplary embodiment of the present disclosure;
fig. 12 schematically illustrates a structural schematic diagram of a computer system suitable for use in implementing the electronic device of the exemplary embodiments of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the disclosed aspects may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
FIG. 1 illustrates a schematic diagram of an exemplary system architecture to which the anomaly index detection method of embodiments of the present disclosure may be applied.
As shown in fig. 1, system architecture 1000 may include one or more of terminal devices 1001, 1002, 1003, a network 1004, and a server 1005. The network 1004 serves as a medium for providing a communication link between the terminal apparatuses 1001, 1002, 1003 and the server 1005. The network 1004 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, the server 1005 may be a server cluster formed by a plurality of servers.
A user can interact with a server 1005 via a network 1004 using terminal apparatuses 1001, 1002, 1003 to receive or transmit messages or the like. The terminal devices 1001, 1002, 1003 may be various electronic devices having a display screen including, but not limited to, smartphones, tablet computers, portable computers, desktop computers, and the like. In addition, the server 1005 may be a server providing various services.
In one embodiment, the execution subject of the abnormality index detection method of the present disclosure may be a server 1005, and the server 1005 may acquire user inputs transmitted by the terminal devices 1001, 1002, 1003, and acquire a plurality of sample index sequences according to the abnormality index detection method of the present disclosure, cluster indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences, determine a stable relationship between the indexes in the various class clusters, monitor the plurality of indexes, determine a plurality of candidate index combinations that do not meet the stable relationship between the indexes among the plurality of indexes according to the values of the plurality of indexes, determine an abnormality index according to the plurality of candidate index combinations, and then return the abnormality index obtained after the processing to the terminal devices 1001, 1002, 1003. In addition, the abnormality index detection method of the present disclosure may also be executed by the terminal device 1001, 1002, 1003, etc. to implement a process of obtaining a plurality of sample index sequences by the abnormality index detection method, clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences, determining a stable relationship between the indexes in the respective class clusters, monitoring the plurality of indexes, determining a plurality of candidate index combinations that do not conform to the stable relationship between the indexes among the plurality of indexes according to the values of the plurality of indexes, and determining the abnormality index according to the plurality of candidate index combinations.
Further, the implementation procedure of the abnormality index detection method of the present disclosure may also be implemented by the terminal devices 1001, 1002, 1003 and the server 1005 in common. For example, the terminal device 1001, 1002, 1003 may acquire a plurality of sample index sequences, cluster the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences, determine a stable relationship between the indexes in the respective class clusters, and then send the obtained stable relationship between the indexes in the respective class clusters to the server 1005, so that the server 1005 may monitor the plurality of indexes, determine a plurality of candidate index combinations that do not meet the stable relationship between the indexes among the plurality of indexes according to the values of the plurality of indexes, and determine an abnormal index according to the plurality of candidate index combinations.
According to the abnormal index detection method provided in the present exemplary embodiment, a plurality of sample index sequences are acquired, indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences, stable relations among the indexes in the class clusters are determined, the plurality of indexes are monitored, a plurality of candidate index combinations which do not accord with the stable relations among the indexes are determined among the plurality of indexes according to the values of the plurality of indexes, and the abnormal index is determined according to the plurality of candidate index combinations.
As shown in fig. 2, the abnormality index detection method may include the steps of:
step S210, a plurality of sample index sequences are obtained, and indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences; wherein the class cluster comprises a plurality of indexes;
step S220, determining the stable relation among indexes in various clusters;
step S230, monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with stable relations among the indexes according to the values of the indexes;
step S240, determining an abnormality index according to the plurality of candidate index combinations.
In the abnormal index detection method provided by the embodiment of the disclosure, a plurality of sample index sequences can be obtained, indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences, stable relations among the indexes in the class clusters are determined, the plurality of indexes are monitored, a plurality of candidate index combinations which do not accord with the stable relations among the indexes are determined in the plurality of indexes according to the values of the plurality of indexes, and the abnormal index is determined according to the plurality of candidate index combinations. According to the scheme, the stable relation between similar indexes can be determined according to the existing data, and the abnormal indexes can be determined when the value of the detection index does not meet the stable relation, so that on one hand, an abnormal threshold value is not required to be set based on an empirical value, and manual experience is not required to be relied on; on the other hand, the method does not need to check a plurality of abnormal indexes, and avoids consuming a great deal of time cost and labor cost, so that the determination efficiency of the abnormal indexes is improved, and the failure processing efficiency is further improved.
Next, steps S210 to S240 of the abnormality index detection method in the present exemplary embodiment will be described in more detail with reference to the drawings and the embodiments.
Step S210, a plurality of sample index sequences are obtained, and indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences;
in one example embodiment of the present disclosure, a plurality of sample index sequences may be acquired. Specifically, a sample index sequence corresponds to an index that can be used to measure, evaluate, or measure a quantitative measure or parameter of a particular thing or phenomenon, and can be used to describe and measure the state, performance, effect, or quality of a system, process, event, performance, etc.
For example, the indexes may include a live success rate, a cloud back view power, a public/intranet dial-up success rate, a bandwidth usage rate, a concurrent usage rate, a traffic, a bandwidth, a PV (Page Views), a tp99 error rate (Top 99error rate,99 quantile error rate), and the like.
The specific type of the index is not particularly limited in this disclosure.
In particular, the sample index sequence may be a measurement or monitoring result of an index.
For example, the sample index sequence is a monitor value of the live success rate from 0 to 9.
It should be noted that, the specific manner of obtaining the plurality of sample index sequences is not particularly limited in this disclosure.
In an example embodiment of the present disclosure, after the plurality of sample index sequences are obtained through the above steps, indexes corresponding to the plurality of sample index sequences may be clustered into a plurality of class clusters according to the plurality of sample index sequences. Specifically, clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences means that the indexes corresponding to the sample index sequences are grouped into different class clusters according to the similarity or the correlation of the sample index sequences. Wherein the class cluster comprises a plurality of indexes;
specifically, indexes corresponding to a plurality of sample index sequences may be clustered into a plurality of class clusters by calculating distances or similarities between the sample index sequences.
For example, indexes corresponding to a plurality of sample index sequences may be clustered into a plurality of class clusters by a hierarchical clustering algorithm, a K-means clustering algorithm (K-means clustering) and a DBSCAN clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, density-based noisy spatial clustering application algorithm) or the like. Specifically, the hierarchical clustering algorithm may divide indexes corresponding to the plurality of sample index sequences into hierarchical cluster structures; the K-means clustering algorithm can distribute indexes corresponding to a plurality of sample index sequences into K clusters, so that the distance between the index corresponding to each sample index sequence and the center point (centroid) of the cluster to which the index corresponds is minimized; the DBSCAN clustering algorithm can judge the density of the indexes corresponding to the sample index sequence by defining the neighborhood of the indexes corresponding to the sample index sequence, and cluster the indexes corresponding to the sample index sequence according to the density connectivity.
Note that, the specific manner of clustering the indexes corresponding to the plurality of sample index sequences into the plurality of class clusters according to the plurality of sample index sequences is not particularly limited in the present disclosure.
Step S220, determining the stable relation among indexes in various clusters;
in one example embodiment of the present disclosure, after the plurality of indexes are clustered into a plurality of class clusters through the above steps, a stable relationship between the indexes in the class clusters may be determined. Specifically, the stable relationship between indexes refers to a state in which the relationship between two indexes remains relatively stable under certain conditions.
Specifically, the stable relationship between the indexes in the various clusters can be determined by correlation analysis or regression analysis. The correlation analysis can be used for measuring the linear correlation degree between two indexes so as to determine the stable relation between the indexes in various clusters; regression analysis can be used to build a predictive model of one index against another to determine the stable relationship between the indices in the various clusters.
In one example embodiment of the present disclosure, stable relationships between the various indicators in the various clusters may be determined through time series analysis. In particular, time series analysis may be used to indicate changes and trends in the values of the indicators in the time dimension, thereby analyzing the stable relationship between the indicators.
In one example embodiment of the present disclosure, the stable relationship between the indices in the various clusters may be determined by arimx algorithm (Autoregressive Integrated Moving Average with Exogenous Variables, autoregressive moving average model and exogenous variable). Specifically, the arimx algorithm is a time series analysis method for building and predicting an autoregressive moving average model with exogenous variables. In the arimx model, the influence of external variables on the time series is also considered in addition to the autocorrelation and moving average inside the time series.
Specifically, the stable relationship between the two indices can be determined by the following expression, where a (), B (), C () are coefficients, q is the number of moving average terms, n k Representing the input sample size occurring before the input affects the output, u (t) represents the series of inputs (the value of one index) of the previous and delay on which the output depends, y (t) represents the output value over time (the value of the other index), e (t) represents the white noise disturbance value, and when specifically calculated, the sample index sequences corresponding to the two indices can be substituted into the expression to determine the stable relationship between the two indices:
it should be noted that, the specific manner of determining the stable relationship between the indexes in the various clusters is not particularly limited in the present disclosure.
Step S230, monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with stable relations among the indexes according to the values of the indexes;
in an example embodiment of the present disclosure, after the stable relationship between the indexes in each cluster is obtained through the above steps, a plurality of indexes may be monitored, and a plurality of candidate index combinations that do not meet the stable relationship between the indexes may be determined among the plurality of indexes according to values of the plurality of indexes. Specifically, each index can be monitored in the running process of the system, whether two indexes accord with the stable relation between the two indexes is judged, and when the two indexes are determined not to accord with the stable relation between the two indexes, the two indexes are determined to be a candidate index combination.
In particular, a stable relationship between the metrics may be used to indicate a trend or pattern in which the metrics remain relatively unchanged over a period of time. The indexes do not accord with the stable relation between the two indexes, and the indexes can represent the trend or mode that the indexes break through relative invariance. The failure to meet the stable relationship between the two indexes may be caused by the following conditions: external factor interference, data acquisition or processing errors, event changes, and the like.
In an example embodiment of the present disclosure, the indicator not conforming to the stable relationship between the two indicators may include that an expression is satisfied between the two indicators, and when the two indicators are not satisfied, the indicator not conforming to the stable relationship between the two indicators may be determined as a candidate indicator combination.
The specific manner of determining the plurality of candidate index combinations that do not meet the stable relationship between the indices among the plurality of indices according to the values of the plurality of indices is not particularly limited in the present disclosure.
Step S240, determining an abnormality index according to the plurality of candidate index combinations.
In one example embodiment of the present disclosure, after the plurality of candidate index combinations are obtained through the above steps, the abnormality index may be determined from the plurality of candidate index combinations. Specifically, the plurality of candidate index combinations are all candidate indexes affected by the fault root, and two candidate indexes in each candidate index combination are indexes in the same class cluster, that is, two candidate indexes in a certain candidate index combination have correlation, so that the plurality of candidate index combinations can be analyzed, and an abnormal index is determined in the plurality of candidate indexes corresponding to the plurality of candidate index combinations.
In one example embodiment of the present disclosure, the same candidate index may be found in a plurality of candidate index combinations, and the same candidate index with the largest number is determined as the abnormality index.
The present disclosure is not limited to a specific manner of determining the abnormality index from the plurality of candidate index combinations.
In one example embodiment of the present disclosure, after the abnormality index is determined through the above steps, an alert dispatch may be sent to process for the abnormality index.
In an example embodiment of the present disclosure, sample data corresponding to a plurality of indexes in a preset time period is obtained, abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index are obtained, the abnormal data of the sample data corresponding to each index is smoothed, and the missing data of the sample data corresponding to each index is filled to obtain a plurality of sample index sequences. Referring to fig. 3, the filling process is performed on missing data of sample data corresponding to each index to obtain a plurality of sample index sequences, and the steps S310 to S320 may include:
step S310, acquiring sample data corresponding to a plurality of indexes in a preset time period, and acquiring abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index;
In an example embodiment of the present disclosure, sample data corresponding to a plurality of indicators in a preset time period may be preset. Specifically, the sample data corresponding to the index refers to the value of the index, and the value of the index in the preset time period refers to the value of the index in a specific time period.
For example, the bandwidth usage rate is between 0 and 6 for the sample data corresponding to the index in the preset time period.
Further, when the preset time period is selected, the cutover time of the network device may be selected.
It should be noted that, the specific period of the preset time period and the specific manner of obtaining the sample data corresponding to the multiple indexes in the preset time period are not particularly limited in the present disclosure.
In one example embodiment of the present disclosure, abnormal data of sample data corresponding to each index and missing data of sample data corresponding to each index are acquired. Specifically, after the sample data corresponding to the index is obtained, it may be determined whether abnormal data exists in the sample data corresponding to the index.
For example, a threshold range may be set for the index, and when data exceeding the threshold range exists in the sample data corresponding to the index, the data exceeding the threshold range may be determined as abnormal data, for example, when the threshold range is set, the mean value of the values of the index from 6 to 24 on a certain day +/-3 x the standard deviation of the values of the index from 6 to 24 on the certain day may be regarded as the threshold range corresponding to the index.
The specific manner of acquiring the abnormal data of the sample data corresponding to each index is not particularly limited in the present disclosure.
In an example embodiment of the present disclosure, missing data of sample data corresponding to each index may be obtained. In particular, missing data may appear in the sample data, which may be generated for the following reasons: difficulty in data acquisition, equipment failure or technical problems, environmental impact and the like. The cause of the missing data is not particularly limited in the present disclosure.
Step S320, performing smoothing processing on the abnormal data of the sample data corresponding to each index, and performing filling processing on the missing data of the sample data corresponding to each index, so as to obtain a plurality of sample index sequences.
In an example embodiment of the present disclosure, after the abnormal data of the sample data corresponding to each index is obtained through the above steps, the abnormal data of the sample data corresponding to each index may be smoothed. Specifically, smoothing the abnormal data of the sample data corresponding to each index means that the abnormal data is eliminated or the influence of the abnormal data is reduced by adopting a smoothing method.
For example, the abnormal data of the sample data corresponding to each index may be smoothed by a moving average method, an exponential smoothing method, a data smoothing model, or the like.
In an example embodiment of the present disclosure, abnormal data of sample data corresponding to each index may be replaced according to a data average value before and after the abnormal data, so as to implement smoothing processing on the abnormal data of the sample data corresponding to each index.
In an example embodiment of the present disclosure, after the missing data of the sample data corresponding to each index is obtained through the above steps, the missing data of the sample data corresponding to each index may be subjected to the padding process. Specifically, the filling processing can be performed on missing data of sample data corresponding to each index in the modes of synchronous data, linear interpolation method, retrograde filling, data filling, filling based on similarity, time sequence analysis and the like.
The specific manner of filling the missing data of the sample data corresponding to each index is not particularly limited in the present disclosure.
In an example embodiment of the present disclosure, after performing smoothing processing on abnormal data of sample data corresponding to each index and performing padding processing on missing data of sample data corresponding to each index, a sample index sequence corresponding to a plurality of indexes may be obtained.
Through the steps S310 to S320, sample data corresponding to a plurality of indexes in a preset time period can be obtained, abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index are obtained, the abnormal data of the sample data corresponding to each index is smoothed, and the missing data of the sample data corresponding to each index is filled, so that a plurality of sample index sequences are obtained.
In an example embodiment of the present disclosure, a mutual reachable distance between indexes is determined according to a plurality of sample index sequences, an association relationship between indexes is determined according to the mutual reachable distance between indexes, and indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the association relationship between indexes. Referring to fig. 4, the method for clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relationship between the indexes may include the following steps S410 to S430:
step S410, determining the mutual reachable distance between the indexes according to a plurality of sample index sequences;
in an example embodiment of the present disclosure, after obtaining the plurality of sample index sequences through the above steps, a mutual reachable distance between the indexes may be determined according to the plurality of sample index sequences. In particular, the inter-reachable distance between indexes is a measure for measuring the degree of association between different indexes, and may represent the distance from one index to another index or the length of a path in one index space.
In one example embodiment of the present disclosure, when determining the mutual reachable distance between two indexes, a core distance of one of the indexes may be calculated, a core distance of the other index may be calculated, and the reachable distances of the two indexes may be calculated, and a maximum distance may be determined among the three distances (the core distance of one of the indexes, the core distance of the other index, and the reachable distances of the two indexes), and the maximum distance may be determined as the mutual reachable distance of the two indexes.
Wherein the core distance of the index is a threshold distance used in the density clustering algorithm to determine whether the data object is a core point. For example, in a DBSCAN clustering algorithm, a core point refers to a point that contains at least the minimum number of samples (k) of an index within a given radius centered around a certain index, and a core distance refers to, for each core point, the distance from that point to the kth nearest index within its neighborhood. The reachable distance of the two indexes can be used for measuring the similarity or the proximity degree between the two indexes, and the reachable distance of the indexes can be calculated through Euclidean distance, manhattan distance, minkowski distance and other modes. When calculating the core distance and the reachable distance, the calculation is required to be performed through a sample index sequence corresponding to each index.
In one example embodiment of the present disclosure, determining the mutual reachable distance between two indicators may employ the following expression, where d mreach-k For the distance between two indexes, core k (a) Core distance, core, as an index k (b) Is the core distance of another index, d (a, b) is the reachable distance of the two indexes, N k (x) The kth sample point that is nearest neighbor to sample point x:
d mreach-k (a,b)=max{core k (a),core k (b),d(a,b)},core k (x)=d(x,N k (x))
it should be noted that, the specific manner of determining the reachable distance between the indexes according to the plurality of sample index sequences is not particularly limited in the present disclosure.
Step S420, determining the association relation among the indexes according to the mutual reachable distance among the indexes;
in an example embodiment of the present disclosure, after the mutual reachable distance between the indexes is obtained through the above steps, the association relationship between the indexes may be determined according to the mutual reachable distance between the indexes. Specifically, the association relationship between the indexes can be used for indicating the degree of mutual association between the indexes, and all the indexes can be connected in a manner of minimum sum of weights by taking the mutual reachable distance between the indexes as the weight, so as to determine the association relationship between the indexes, namely, the mutual reachable distance between the indexes is the shortest in the association relationship between the indexes.
In one example embodiment of the present disclosure, a minimum spanning tree may be constructed from the mutual reachable distances between the metrics. The minimum spanning tree is used for indicating the association relation among the indexes. Specifically, the minimum spanning tree may be constructed with the indices as vertices and the inter-reachable distances between the indices as weights.
Step S430, clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relation among the indexes.
In an example embodiment of the present disclosure, after the association relationship between the indexes is obtained through the above steps, indexes corresponding to the plurality of sample index sequences may be clustered into a plurality of class clusters according to the association relationship between the indexes. Specifically, the association relationship between the indexes can be used for indicating the degree of mutual association between the indexes, a clustering algorithm based on similarity or distance can be adopted, indexes corresponding to a plurality of sample index sequences are clustered into a plurality of class clusters in combination with the degree of mutual association between the indexes, and the similar or close indexes can be classified into the same class cluster according to the association relationship between the indexes.
It should be noted that, the specific manner of clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relationship between the indexes is not particularly limited in the present disclosure.
Through the above steps S410 to S430, the reachable distances between the indexes can be determined according to the plurality of sample index sequences, the association relationships between the indexes can be determined according to the reachable distances between the indexes, and the indexes corresponding to the plurality of sample index sequences can be clustered into a plurality of clusters according to the association relationships between the indexes.
In an example embodiment of the present disclosure, hierarchical clustering may be performed according to an association relationship between indexes to obtain a plurality of hierarchical initial class clusters, stability of each initial class cluster and density of each initial class cluster are determined, and indexes corresponding to a plurality of sample index sequences are clustered into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster. Referring to fig. 5, the clustering of indexes corresponding to a plurality of sample index sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster may include the following steps S510 to S530:
step S510, hierarchical clustering is carried out according to the association relation among the indexes to obtain a plurality of hierarchical initial class clusters;
in an example embodiment of the present disclosure, after the association relationship between the indexes is obtained through the above steps, hierarchical clustering may be performed according to the association relationship between the indexes to obtain initial class clusters of multiple levels. Specifically, the clustering process of hierarchical clustering gradually merges or partitions clusters from a single index with the smallest index to form a plurality of hierarchical structures.
Specifically, each index is regarded as an independent class cluster, similar class clusters are gradually combined from the bottom layer according to the association relation among indexes until a large class cluster is formed, in the process, class clusters of multiple layers can be generated, each layer represents clusters with different granularity, and the class clusters with different scales, namely, the initial class clusters of multiple layers, can be seen from the uppermost layer to the bottommost layer.
It should be noted that, the specific manner of performing hierarchical clustering according to the association relationship between the indexes to obtain the initial class clusters of multiple hierarchies in the present disclosure is not particularly limited.
Furthermore, the association relation between the indexes is a minimum spanning tree constructed by taking the indexes as vertexes and the inter-reachable distance between the indexes as weight, and hierarchical clustering is carried out on the minimum spanning tree to obtain initial class clusters of a plurality of layers.
Further, the minimum spanning tree constructed in the above steps may be pruned. Specifically, the smallest subtree of the smallest spanning tree can be limited by parameters, so that the formed class cluster is prevented from being too large or too small.
Step S520, determining the stability of each initial cluster and the density of each initial cluster;
In an example embodiment of the present disclosure, after a plurality of initial class clusters are obtained through the above steps, stability of each initial class cluster and density of each initial class cluster may be determined. In particular, the stability of the initial cluster may be used to indicate the consistency or stability of the clustering results across different subsets of data or running experiments. If the obtained cluster-like structures are similar or overlap degree is high when the same clustering algorithm is operated on different data subsets or multiple times, the clustering result can be said to have higher stability. The density of the initial cluster can be used for indicating the compactness or concentration degree of the indexes in the cluster, and the density reflects the distribution condition of the indexes in the cluster, namely the concentration degree of the indexes in the space.
In an example embodiment of the present disclosure, the stability of an initial cluster may be calculated by the following expression, where distance (x) refers to the shortest distance among distances between an index x in the initial cluster and all indexes in other initial clusters, and λ is the stability of the initial cluster:
in one example embodiment of the present disclosure, the density of an initial class cluster may be calculated by the following expression, where σ is the density of the initial class cluster, p is the data points in the initial class cluster, cluster is the initial class cluster, λ p Is the density value of p, lambda birth Birth density value for p:
σ=∑ p∈clusterpbirth )
the specific manner of determining the stability of each initial cluster and the specific manner of determining the density of each initial cluster are not particularly limited in the present disclosure.
Step S530, according to the stability of each initial cluster and the density of each initial cluster, the indexes corresponding to the index sequences of the plurality of samples are clustered into a plurality of clusters.
In an example embodiment of the present disclosure, after the stability of each initial class cluster and the density of each initial class cluster are obtained through the above steps, indexes corresponding to a plurality of sample index sequences may be clustered into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster. Specifically, the stability of each initial cluster and the density of each initial cluster can be analyzed, and a plurality of initial clusters which are suitable for demands are selected as the final clustering result (a plurality of clusters); or, a clustering parameter can be set, the stability of each initial cluster and the density of each initial cluster are compared with the clustering parameter, and the initial cluster conforming to the clustering parameter is used as the final clustering result.
It should be noted that, the specific manner of clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster is not particularly limited in the present disclosure.
Through the steps S510 to S530, hierarchical clustering may be performed according to the association relationship between the indexes to obtain a plurality of hierarchical initial clusters, stability of each initial cluster and density of each initial cluster are determined, and indexes corresponding to the plurality of sample index sequences are clustered into a plurality of clusters according to the stability of each initial cluster and the density of each initial cluster.
In one example embodiment of the present disclosure, a stability threshold between the indicators may be determined according to a stability relationship between the indicators, and when a residual between values of the two indicators is greater than or equal to the stability threshold between the indicators, the two indicators are determined as candidate indicator combinations. Referring to fig. 6, when the residual between the values of the two indices is greater than or equal to the stability threshold between the indices, determining the two indices as candidate index combinations may include the following steps S610 to S620:
step S610, determining a stability threshold value between indexes according to a stability relationship between the indexes;
in an example embodiment of the present disclosure, after the stability relationship between the indexes in the various clusters is obtained through the above steps, the stability threshold between the indexes may be determined according to the stability relationship between the indexes. Specifically, the stability threshold may be used to indicate an acceptable range of fluctuation between values of the indicators, and when fluctuation between values of the indicators is within the stability threshold, the two indicators may be indicated as normal indicators, and when fluctuation between values of the indicators exceeds the stability threshold, the two indicators may be indicated as abnormal indicators.
The specific manner of determining the stability threshold value between the respective indices according to the stability relationship between the respective indices is not particularly limited in the present disclosure.
Step S620, determining the two indexes as candidate index combinations when the residual error between the values of the two indexes is greater than or equal to the stability threshold value between the indexes.
In an example embodiment of the present disclosure, when a residual between values of two indicators is greater than or equal to a stability threshold between the indicators, a fluctuation between values representing the indicators exceeds the stability threshold, at which time the two indicators may be abnormal indicators, the two indicators may be determined to be candidate indicator combinations.
In one example embodiment of the present disclosure, when monitoring the metrics in real time, a residual between the values of the two metrics may be calculated, and when the residual between the values of the two metrics is greater than or equal to a stability threshold between the metrics, determining the two metrics as candidate metric combinations may be calculated by the following expression, where δ (a,b) Is the residual between the values of the two indices, f (x a ) Is the value of an index, f (x b ) The value of another index:
δ (a,b) =f(x a )-f(x b )
through the above steps S610 to S620, the stability threshold value between the indexes can be determined according to the stability relationship between the indexes, and when the residual error between the values of the two indexes is greater than or equal to the stability threshold value between the indexes, the two indexes are determined as candidate index combinations.
In an example embodiment of the present disclosure, the same candidate indexes in a plurality of candidate index combinations may be connected to establish a directed graph, weight values of nodes in the directed graph are calculated, and an abnormal index is determined in the plurality of candidate indexes according to the weight values of the nodes. Referring to fig. 7, determining an abnormality index from among a plurality of candidate indexes according to the weight value of each node may include the following steps S710 to S730:
step S710, connecting the same candidate indexes in the candidate index combinations to establish a directed graph;
in an example embodiment of the present disclosure, after a plurality of candidate index combinations are obtained through the above steps, the same candidate index in the plurality of candidate index combinations may be subjected to connection establishment directed graph. Specifically, a directed graph is a graph structure that is made up of a set of vertices and a set of directed edges, each edge having a start vertex and an end vertex in the directed graph.
Specifically, the same candidate indexes in the plurality of candidate index combinations can be obtained, and the same candidate indexes are respectively connected, so that a connection result is a directed graph.
The specific manner of connecting the same candidate index in the plurality of candidate index combinations to create the directed graph is not particularly limited in the present disclosure.
Step S720, calculating weight values of all nodes in the directed graph;
in an example embodiment of the present disclosure, after the directed graph is obtained through the above steps, the weight value of each node in the directed graph may be calculated. The weight value of the node is used for indicating the probability that the candidate index corresponding to the node is an abnormal index. Specifically, the weight value of each node is the weight value of each candidate index, and the weight value of each node in the directed graph can be calculated by adopting a statistical method, a propagation algorithm based on graph theory and an algorithm based on a correlation coefficient or similarity measure.
In an example embodiment of the present disclosure, the PR value (PageRank ) of each node, that is, the weight value of each node in the directed graph, may be calculated based on a PageRank algorithm (page rank algorithm), where PR (a) is the PR value of node a, i is the current time or iteration number, PR (Ti) is the PR value of each other node (capable of pointing to a), and L (Ti) is the outgoing number of each other node (capable of pointing to a):
note that, the specific manner of calculating the weight value of each node in the directed graph is not particularly limited in this disclosure.
In step S730, an anomaly index is determined from the plurality of candidate indexes according to the weight value of each node.
In an example embodiment of the present disclosure, after the weight value of each node is obtained through the above steps, an abnormality index may be determined among a plurality of candidate indexes according to the weight value of each node. Specifically, each node is each candidate index, the higher the weight value is, the stronger the correlation exists between the node and other nodes, the stronger the influence or importance of the node in the whole network is, and the higher the probability that the candidate index corresponding to the node is an abnormal index is.
In an example embodiment of the present disclosure, each node may be ordered according to a weight value of each node, and a candidate index corresponding to a node with a highest weight value is determined as an anomaly index.
Through the steps S710 to S730, the same candidate indexes in the plurality of candidate index combinations may be connected to create a directed graph, the weight value of each node in the directed graph may be calculated, and the abnormal index may be determined from the plurality of candidate indexes according to the weight value of each node.
In an example embodiment of the present disclosure, as shown in fig. 8, another abnormality index detection method may include the following steps S801 to S805:
step S801, acquiring sample data corresponding to a plurality of indexes (x 1, x 2..xn) in a preset time period;
Step S802, performing data cleaning (including abnormal data smoothing processing and missing data filling processing) on sample data corresponding to a plurality of indexes;
step S803, the indexes corresponding to the sample index sequences are clustered into a plurality of class clusters;
step S804, determining stable relations among indexes in various clusters;
in step S805, a plurality of indexes are monitored, a stability threshold value between the indexes is determined according to a stability relationship between the indexes, and when a residual error between values of the two indexes is greater than or equal to the stability threshold value between the indexes, the two indexes are determined as candidate index combinations.
In an exemplary embodiment of the present disclosure, as shown in fig. 9, a flowchart for determining a stable relationship between indexes in various clusters may include the following steps S901 to S903:
step S901, determining the mutual reachable distance between each index according to a plurality of sample index sequences;
step S902, hierarchical clustering is carried out according to the association relation among the indexes to obtain a plurality of class clusters. Comprising the following steps: determining the association relation among the indexes according to the mutual reachable distances among the indexes, performing hierarchical clustering according to the association relation among the indexes to obtain a plurality of hierarchical initial class clusters, determining the stability of each initial class cluster and the density of each initial class cluster, and clustering the indexes corresponding to the sample index sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster.
Step S903, determining the stable relation among the indexes in the various clusters based on ARIMAX algorithm.
In an example embodiment of the present disclosure, a flowchart for assigning an abnormal indicator alert may include the following steps S1001 to S1004:
step S1001, monitoring a residual error between values of two indexes;
step S1002, when the residual error between the values of the two indexes is greater than or equal to the stability threshold value between the indexes, determining the two indexes as candidate index combinations;
step S1003, connecting the same candidate indexes in the candidate index combinations to establish a directed graph, calculating PR values of all nodes in the directed graph, and determining abnormal indexes in the candidate indexes according to the PR values of all the nodes;
step S1004, generating an alarm list and distributing the alarm list.
In the abnormal index detection method provided by the embodiment of the disclosure, a plurality of sample index sequences can be obtained, indexes corresponding to the plurality of sample index sequences are clustered into a plurality of class clusters according to the plurality of sample index sequences, stable relations among the indexes in the class clusters are determined, the plurality of indexes are monitored, a plurality of candidate index combinations which do not accord with the stable relations among the indexes are determined in the plurality of indexes according to the values of the plurality of indexes, and the abnormal index is determined according to the plurality of candidate index combinations. According to the scheme, the stable relation between similar indexes can be determined according to the existing data, and the abnormal indexes can be determined when the value of the detection index does not meet the stable relation, so that on one hand, an abnormal threshold value is not required to be set based on an empirical value, and manual experience is not required to be relied on; on the other hand, the method does not need to check a plurality of abnormal indexes, and avoids consuming a great deal of time cost and labor cost, so that the determination efficiency of the abnormal indexes is improved, and the failure processing efficiency is further improved.
It is noted that the above-described figures are merely schematic illustrations of processes involved in a method according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
In addition, in an exemplary embodiment of the present disclosure, an abnormality index detection apparatus is also provided. Referring to fig. 11, an abnormality index detection apparatus 1100 includes: an index sequence acquisition module 1110, a stable relation determination module 1120, a candidate combination determination module 1130, and an abnormal index determination module 1140.
The index sequence acquisition module is used for acquiring a plurality of sample index sequences, and clustering indexes corresponding to the sample index sequences into a plurality of class clusters according to the sample index sequences; wherein the class cluster comprises a plurality of indexes; the stability relation determining module is used for determining stability relations among indexes in various clusters; a candidate combination determining module, configured to monitor a plurality of indexes, and determine a plurality of candidate index combinations that do not conform to a stable relationship between the indexes among the plurality of indexes according to values of the plurality of indexes; the abnormal index determining module is used for determining abnormal indexes according to the candidate index combinations.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the apparatus further includes: the sample data acquisition unit is used for acquiring sample data corresponding to a plurality of indexes in a preset time period, and acquiring abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index; the data processing unit is used for carrying out smoothing processing on abnormal data of the sample data corresponding to each index and carrying out filling processing on missing data of the sample data corresponding to each index to obtain a plurality of sample index sequences.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, the apparatus further includes: a mutual reachable distance determining unit for determining the mutual reachable distance between the indexes according to the plurality of sample index sequences; the association relation determining unit is used for determining the association relation among the indexes according to the mutual reachable distance among the indexes; in the association relation among the indexes, the reachable distance among the indexes is shortest; and the clustering unit is used for clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relation among the indexes.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, indexes corresponding to a plurality of sample index sequences are clustered into a plurality of class clusters according to an association relationship between indexes, and the apparatus further includes: the hierarchical clustering unit is used for performing hierarchical clustering according to the association relation among the indexes to obtain a plurality of hierarchical initial class clusters; the attribute calculation unit is used for determining the stability of each initial cluster and the density of each initial cluster; and the class cluster determining unit is used for clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the stability of each initial class cluster and the density of each initial class cluster.
In an exemplary embodiment of the present disclosure, based on the foregoing solution, the apparatus further includes: a minimum spanning tree construction unit for constructing a minimum spanning tree according to the mutual reachable distance between the indexes; the minimum spanning tree is used for indicating the association relation among the indexes.
In an exemplary embodiment of the present disclosure, based on the foregoing aspect, the apparatus further includes: a stability threshold determining unit for determining a stability threshold between the indexes according to a stability relationship between the indexes; and a candidate index combination determination unit configured to determine two indexes as candidate index combinations when a residual error between values of the two indexes is greater than or equal to a stability threshold value between the indexes.
In an exemplary embodiment of the present disclosure, based on the foregoing aspect, the apparatus further includes: the directed graph establishing unit is used for establishing a directed graph by connecting the same candidate indexes in the candidate index combinations; the weight value calculation unit is used for calculating the weight value of each node in the directed graph; the weight value of the node is used for indicating the probability that the candidate index corresponding to the node is an abnormal index; and the abnormal index determining unit is used for determining an abnormal index from the candidate indexes according to the weight value of each node.
Since each functional module of the abnormality index detection apparatus of the exemplary embodiment of the present disclosure corresponds to a step of the exemplary embodiment of the abnormality index detection method described above, for details not disclosed in the embodiment of the apparatus of the present disclosure, please refer to the embodiment of the abnormality index detection method described above in the present disclosure.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-described abnormality index detection method is also provided.
Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 12000 according to such an embodiment of the present disclosure is described below with reference to fig. 12. The electronic device 12000 shown in fig. 12 is merely an example, and should not be construed to limit the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 12, the electronic device 12000 is embodied in the form of a general purpose computing device. The components of the electronic device 12000 may include, but are not limited to: the at least one processing unit 1210, the at least one memory unit 1220, a bus 1230 connecting the different system components (including the memory unit 1220 and the processing unit 1210), and a display unit 1240.
Wherein the storage unit stores program code that is executable by the processing unit 1210 such that the processing unit 1210 performs steps according to various exemplary embodiments of the present disclosure described in the above-described "exemplary methods" section of the present specification. For example, the processing unit 1210 may perform step S210 shown in fig. 2, acquire a plurality of sample index sequences, and cluster indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences; wherein the class cluster comprises a plurality of indexes; step S220, determining the stable relation among indexes in various clusters; step S230, monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with stable relations among the indexes according to the values of the indexes; step S240, determining an abnormality index according to the plurality of candidate index combinations.
As another example, the electronic device may implement the various steps shown in fig. 2.
The storage unit 1220 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 1221 and/or cache memory unit 1222, and may further include Read Only Memory (ROM) 1223.
Storage unit 1220 may also include a program/utility 1224 having a set (at least one) of program modules 1225, such program modules 1225 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 1230 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
The electronic device 12000 can also communicate with one or more external devices 1270 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 12000, and/or any device (e.g., router, modem, etc.) that enables the electronic device 12000 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 1250. Also, electronic device 12000 can communicate with one or more networks (e.g., a Local Area Network (LAN), wide Area Network (WAN), and/or public network, such as the internet) through network adapter 1260. As shown, network adapter 1260 communicates with other modules of electronic device 12000 over bus 1230. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with the electronic device 12000, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible embodiments, the various aspects of the present disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
Furthermore, the above-described figures are only schematic illustrations of processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (10)

1. An abnormality index detection method, characterized by comprising:
acquiring a plurality of sample index sequences, and clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences; wherein the class cluster comprises a plurality of indexes;
Determining a stable relation among the indexes in each class cluster;
monitoring a plurality of indexes, and determining a plurality of candidate index combinations which do not accord with stable relations among the indexes according to the values of the indexes;
and determining an abnormality index according to a plurality of candidate index combinations.
2. The method of claim 1, wherein the obtaining a plurality of sample index sequences comprises:
acquiring sample data corresponding to a plurality of indexes in a preset time period, and acquiring abnormal data of the sample data corresponding to each index and missing data of the sample data corresponding to each index;
and carrying out smoothing treatment on abnormal data of the sample data corresponding to each index, and carrying out filling treatment on missing data of the sample data corresponding to each index to obtain a plurality of sample index sequences.
3. The method according to claim 1, wherein the clustering the indices corresponding to the plurality of sample index sequences into a plurality of class clusters according to the plurality of sample index sequences comprises:
determining the mutual reachable distance between the indexes according to the plurality of sample index sequences;
determining the association relation between the indexes according to the mutual reachable distance between the indexes; in the association relation between the indexes, the reachable distance between the indexes is shortest;
And clustering indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relation among the indexes.
4. The method according to claim 3, wherein the clustering the indexes corresponding to the plurality of sample index sequences into a plurality of class clusters according to the association relation between the indexes includes:
hierarchical clustering is carried out according to the association relation among the indexes to obtain a plurality of hierarchical initial class clusters;
determining the stability of each initial cluster and the density of each initial cluster;
and clustering indexes corresponding to the sample index sequences into a plurality of class clusters according to the stability of the initial class clusters and the density of the initial class clusters.
5. A method according to claim 3, wherein said determining the association between the respective indices based on the mutual reachable distance between the respective indices comprises:
constructing a minimum spanning tree according to the mutual reachable distance between the indexes; the minimum spanning tree is used for indicating the association relation between the indexes.
6. The method of claim 1, wherein said determining a plurality of candidate index combinations among a plurality of said indices that do not meet a stable relationship between said indices based on values of said plurality of indices comprises:
Determining a stability threshold value between the indexes according to the stability relation between the indexes;
and determining the two indexes as the candidate index combination when the residual error between the values of the two indexes is larger than or equal to the stability threshold value between the indexes.
7. The method of claim 1, wherein said determining an anomaly index from a plurality of said candidate index combinations comprises:
connecting the same candidate indexes in the candidate index combinations to establish a directed graph;
calculating the weight value of each node in the directed graph; the weight value of the node is used for indicating the probability that the candidate index corresponding to the node is an abnormal index;
and determining an abnormal index from the candidate indexes according to the weight value of each node.
8. An abnormality index detection device, characterized by comprising:
the index sequence acquisition module is used for acquiring a plurality of sample index sequences and clustering indexes corresponding to the sample index sequences into a plurality of class clusters according to the sample index sequences; wherein the class cluster comprises a plurality of indexes;
the stability relation determining module is used for determining stability relation among the indexes in the class clusters;
A candidate combination determining module, configured to monitor a plurality of the indexes, and determine a plurality of candidate index combinations that do not conform to a stable relationship between the indexes from among the plurality of indexes according to values of the plurality of indexes;
and the abnormal index determining module is used for determining abnormal indexes according to the candidate index combinations.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-7.
10. An electronic device, comprising:
one or more processors; and
a memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-7.
CN202310920989.7A 2023-07-25 2023-07-25 Abnormality index detection method and device, storage medium and electronic equipment Pending CN116910624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310920989.7A CN116910624A (en) 2023-07-25 2023-07-25 Abnormality index detection method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310920989.7A CN116910624A (en) 2023-07-25 2023-07-25 Abnormality index detection method and device, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN116910624A true CN116910624A (en) 2023-10-20

Family

ID=88352928

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310920989.7A Pending CN116910624A (en) 2023-07-25 2023-07-25 Abnormality index detection method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN116910624A (en)

Similar Documents

Publication Publication Date Title
CN111124840B (en) Method and device for predicting alarm in business operation and maintenance and electronic equipment
US11061393B2 (en) Consolidating anomaly root causes and alarms using graphical granger models
US10599506B2 (en) Methods and systems for identifying action for responding to anomaly in cloud computing system
CN107924360B (en) Diagnostic framework in a computing system
US20170372212A1 (en) Model based root cause analysis
CN109981333B (en) Operation and maintenance method and operation and maintenance equipment applied to data center
Lockhart et al. Scission: Performance-driven and context-aware cloud-edge distribution of deep neural networks
US20230133541A1 (en) Alert correlating using sequence model with topology reinforcement systems and methods
CN114969366A (en) Network fault analysis method, device and equipment
CN111027591A (en) Node fault prediction method for large-scale cluster system
JP2019036112A (en) Abnormal sound detector, abnormality detector, and program
CN111901156B (en) Method and device for monitoring faults
CN114172819A (en) Demand resource prediction method, system, electronic device and storage medium for NFV network element
CN117336228A (en) IGP simulation recommendation method, device and medium based on machine learning
Taherizadeh et al. Incremental learning from multi-level monitoring data and its application to component based software engineering
WO2023224764A1 (en) Multi-modality root cause localization for cloud computing systems
CN116225848A (en) Log monitoring method, device, equipment and medium
CN116910624A (en) Abnormality index detection method and device, storage medium and electronic equipment
CN114266352B (en) Model training result optimization method, device, storage medium and equipment
CN114866438A (en) Abnormal hidden danger prediction method and system under cloud architecture
CN113064812A (en) Project development process quality defect prediction method, device and medium
CN115408189A (en) Artificial intelligence and big data combined anomaly detection method and service system
US20230179501A1 (en) Health index of a service
Liu et al. Towards dynamic reconfiguration of composite services via failure estimation of general and domain quality of services
CN111625727B (en) Information processing method, device and storage medium for social relationship data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination