CN117891643A

CN117891643A - Abnormality index sorting method and device, electronic equipment and storage medium

Info

Publication number: CN117891643A
Application number: CN202410089369.8A
Authority: CN
Inventors: 张照胜; 谭新培; 黄超斌; 张悦; 朱杰
Original assignee: Chengfang Financial Technology Co ltd
Current assignee: Chengfang Financial Technology Co ltd
Priority date: 2024-01-22
Filing date: 2024-01-22
Publication date: 2024-04-16

Abstract

The invention discloses an abnormal index sorting method, an abnormal index sorting device, electronic equipment and a storage medium, and relates to the technical field of index processing. The abnormality index ranking method comprises the following steps: generating an index reconstruction sequence according to the acquired abnormal index data set; determining an abnormality detection result based on a preset abnormality detection threshold and an index reconstruction sequence; generating an abnormality index ranking within the abnormality detection time according to the abnormality detection result and the ranking score standardization mapping relation. The method can solve the problem that the existing abnormal index sequencing does not consider the continuity of index dimension and time dimension, can improve the relevance among indexes, enhance the accuracy of the abnormal index sequencing, can improve the processing efficiency of the abnormal indexes, and enhance the stability of a system.

Description

Abnormality index sorting method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of index processing technologies, and in particular, to a method and apparatus for ordering abnormal indexes, an electronic device, and a storage medium.

Background

With the rapid development of the internet and information technology, the operation environment of modern enterprises becomes more and more complex, and the status of an operation and maintenance monitoring system is also more and more prominent, which can help the enterprises to monitor the performance and health status of application systems, IT systems and network devices in real time. In the operation and maintenance monitoring system, a large number of indexes are monitored, such as CPU (Central processing Unit) utilization rate, memory utilization rate, disk space occupation rate and the like, and in the running process of the system, the condition that a plurality of indexes are abnormal at the same time can occur, and at the moment, the abnormal indexes are required to be sequenced to determine which indexes are most critical and urgent, priority treatment is required, so that operation and maintenance personnel can be helped to more efficiently locate and solve the problems.

Currently, there is a sorting method based on threshold setting for multi-index anomaly sorting, in which a situation that an index exceeds a threshold is identified as anomaly by setting a proper threshold, and sorting is performed according to the degree of anomaly. And learning the relevance among indexes and the characteristics under abnormal conditions from historical data by training a deep learning model based on a deep learning method, so as to sort multiple indexes.

However, the sorting method based on the threshold setting does not consider complex correlations between indexes and does not consider duration sorting, which may seriously affect the accuracy of sorting. The deep learning-based method, while considering the correlation between indexes, does not consider the continuity and randomness of the anomaly ordering in the time dimension.

Disclosure of Invention

The invention provides an abnormal index sorting method, an abnormal index sorting device, electronic equipment and a storage medium, which solve the problem that the existing abnormal index sorting does not consider the continuity of index dimension and time dimension, can improve the relevance among indexes, enhance the accuracy of abnormal index sorting, can improve the processing efficiency of abnormal indexes and enhance the stability of a system.

In a first aspect, an embodiment of the present invention provides an anomaly index sorting method, including:

Generating an index reconstruction sequence according to the acquired abnormal index data set;

determining an abnormality detection result based on a preset abnormality detection threshold and an index reconstruction sequence;

generating an abnormality index ranking within the abnormality detection time according to the abnormality detection result and the ranking score standardized mapping relation.

In a second aspect, an embodiment of the present invention further provides an anomaly index sorting device, including:

the sequence forming module is used for generating an index reconstruction sequence according to the acquired abnormal index data set;

the abnormality detection module is used for determining an abnormality detection result based on a preset abnormality detection threshold value and an index reconstruction sequence;

and the index sorting module is used for generating an abnormal index sorting in the abnormal detection time according to the abnormal detection result and the sorting score standardized mapping relation.

In a third aspect, an embodiment of the present invention further provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the anomaly index ordering method of any one of the embodiments of the present invention.

In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores computer instructions for implementing the method for ordering abnormal indicators according to any one of the embodiments of the present invention when executed by a processor.

According to the technical scheme provided by the embodiment of the invention, the obtained abnormal index is generated into the index reconstruction sequence, the abnormal detection result of the index reconstruction sequence is determined according to the preset abnormal detection threshold, the abnormal index sequence corresponding to the abnormal detection result is determined according to the sequence score standardized mapping relation in the abnormal detection time, the problem that the continuity of the index dimension and the time dimension is not considered in the existing abnormal index sequence can be solved, the relevance among indexes can be improved, the accuracy of the abnormal index sequence can be enhanced, the processing efficiency of the abnormal index can be improved, and the stability of a system can be enhanced.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of an abnormality index ranking method according to an embodiment of the present invention;

FIG. 2 is a flowchart of another method for sorting abnormal indicators according to an embodiment of the present invention;

FIG. 3 is an exemplary diagram of an anomaly index sorting method according to an embodiment of the present invention;

FIG. 4 is an exemplary diagram of a ranking score normalization transfer function provided by an embodiment of the present invention;

FIG. 5 is an exemplary diagram of an abnormality index ranking result provided by an embodiment of the present invention;

FIG. 6 is a schematic diagram of an implementation of a multi-index anomaly ordering algorithm based on an exponential moving weighted average according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of an anomaly index sorting algorithm implementation based on anomaly times according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of an implementation of an anomaly index ranking algorithm based on the number of anomalies in a recent period of time for each index according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an abnormality index ranking apparatus according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of an electronic device implementing the abnormality index ranking method according to the embodiment of the invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In an embodiment, fig. 1 is a flowchart of an abnormal indicator sorting method according to an embodiment of the present invention, where the method may be performed by an abnormal indicator sorting apparatus, and the abnormal indicator sorting apparatus may be implemented in hardware and/or software.

As shown in fig. 1, the method for sorting abnormal indexes provided in this embodiment may include:

s110, generating an index reconstruction sequence according to the acquired abnormal index data set.

In the embodiment of the invention, the abnormal index data set can be understood as a set of abnormal indexes which are already identified by the operation and maintenance system.

Specifically, a plurality of identified abnormal indexes can be obtained, an abnormal index data set is generated into an index reconstruction sequence through a multi-index time sequence data reconstruction method, the smaller the value in the generated index reconstruction sequence is, the more abnormal the corresponding index is represented, and the value in the reconstruction sequence can be used for analyzing the abnormal condition of the index.

S120, determining an abnormality detection result based on a preset abnormality detection threshold and an index reconstruction sequence.

In the embodiment of the present invention, the preset abnormality detection threshold may be understood as an abnormality index threshold set in advance according to the data condition in the index reconstruction sequence. The setting method of the preset abnormality detection threshold may include manual setting according to the index condition, and automatic setting of the threshold using an automatic setting model.

Specifically, the value in the index reconstruction sequence can be compared with a preset abnormality detection threshold value, and an index abnormality detection result is determined according to the comparison condition. For example, when the value in the reconstructed sequence is smaller than the preset abnormality detection threshold, the index corresponding to the value may be determined to be abnormal, and when the value in the reconstructed sequence is greater than or equal to the preset abnormality detection threshold, the index corresponding to the value may be determined to be normal.

S130, generating an abnormal index sequence in the abnormal detection time according to the abnormal detection result and the sequence score standardized mapping relation.

In the embodiment of the invention, the index reconstruction sequences can be arranged in an ascending order to obtain the ordering sequences, the ordering sequences have ranking numbers corresponding to the abnormal indexes, the ranking numbers can be defined as the ranking scores for measuring the degree of abnormality among multiple indexes, and the smaller the score is, the more abnormal the index is compared with other indexes. The ranking score standardized mapping relation can be understood as a mapping relation for carrying out standardized processing on ranking scores of the reconstruction elements, so that the ranking scores of the reconstruction elements with various dimensions are aligned, and the ranking accuracy of various abnormal indexes is ensured. The abnormality detection time refers to a time for detecting an index abnormality, and the abnormality detection time may be one time point or one time zone. The anomaly index ranking can be understood as a comprehensive ranking of a plurality of anomaly indexes based on historical index anomalies and after eliminating the influence of random factors.

Specifically, the abnormality index can be extracted from the abnormality detection result at the abnormality detection time, the ranking score of the extracted abnormality index can be standardized through the ranking score standardized mapping relation, and the abnormality indexes are ranked according to the standardized ranking score to generate the abnormality index ranking, wherein the more front the abnormality index ranking is, the more serious the abnormality index can be.

On the basis of the foregoing embodiment, the method for sorting abnormal indexes provided by the embodiment of the present invention further includes:

based on a preset abnormality detection threshold value, counting the total abnormality times of all the reconstruction elements in the index reconstruction sequence in an abnormality detection time range, and generating an abnormality index sequence according to the total abnormality times of all the reconstruction elements;

and reconstructing average anomaly scores of all reconstruction elements in the sequence in an anomaly detection time range based on a preset anomaly detection threshold statistical index, and generating anomaly index sequencing according to the average anomaly scores of the reconstruction elements.

Specifically, the abnormality detection threshold value obtained in the foregoing may be compared with each reconstruction element (i.e., an abnormality index) in the index reconstruction sequence, so as to obtain an abnormality detection result of each reconstruction element, count the total number of abnormality results corresponding to each reconstruction element in the abnormality detection time range, and generate an abnormality index ranking according to the total number of abnormality results of each reconstruction element. The average anomaly score of all the reconstruction elements in the index reconstruction sequence in the anomaly detection time range can be counted, the average anomaly score can be compared with a preset anomaly detection threshold, and if the average anomaly score is smaller than the preset anomaly detection threshold, the anomaly index ranking can be generated according to the average anomaly score of each reconstruction element.

In an embodiment, fig. 2 is a flowchart of another abnormality index ranking method according to an embodiment of the present invention, where the process of generating abnormality index rankings within an abnormality detection time based on an index reconstruction sequence and a preset abnormality detection threshold is further optimized and expanded based on the above embodiments.

As shown in fig. 2, another method for sorting abnormal indicators provided in this embodiment may include:

S210, calling a variation self-encoder to process the abnormal index in the abnormal index data set to generate a reconstruction element constituting an index reconstruction sequence.

In an embodiment of the present invention, a variable Auto-Encoder (VAE) is a structure consisting of an Encoder and a decoder, and is trained to minimize reconstruction errors between encoded and decoded data and initial data. All reconstruction elements in the processed index reconstruction sequence can be made to be as similar as possible to the abnormal index in the abnormal index dataset.

Specifically, the variable self-encoder may be invoked to process the abnormal indexes in the abnormal index data set, generate the reconstruction elements constituting the index reconstruction sequence, and the smaller the values of the reconstruction elements in the processed index reconstruction sequence are, the more abnormal the corresponding indexes are represented, and the values of the reconstruction elements in the reconstruction sequence are taken as the index abnormal scores to quantitatively analyze the index abnormal conditions.

S220, arranging the reconstruction elements as an index reconstruction sequence.

Specifically, the obtained reconstruction elements may be arranged as an index reconstruction sequence, and the arrangement manner may include random arrangement.

S230, acquiring preset abnormal detection thresholds of different reconstruction elements in the corresponding index reconstruction sequence set by the peak value exceeding threshold model based on the extremum theory.

In the embodiment of the invention, the peak value exceeding threshold value model (Peak Over Thres hold, POT) based on the extremum theory refers to a threshold value automatic setting model which can automatically set a threshold value by selecting parameters without assuming data distribution.

Specifically, preset abnormality detection thresholds corresponding to different reconstruction elements in the index reconstruction sequence can be automatically set through the POT model, and the set preset abnormality detection thresholds are obtained.

S240, comparing the reconstruction elements with corresponding preset abnormal detection thresholds respectively.

Specifically, each reconstruction element in the index reconstruction sequence may be respectively compared with a preset anomaly detection threshold set by the corresponding POT model.

S250, when the reconstruction element is smaller than a preset abnormality detection threshold, adding a first mark to a result sequence of an abnormality detection result.

In the embodiment of the present invention, the first flag may be understood as a flag indicating that an index corresponding to a reconstruction element in the index reconstruction sequence is an abnormal condition, where the flag may include a number, a symbol, and the like.

Specifically, when the reconstruction element in the index reconstruction sequence is smaller than the corresponding preset abnormality detection threshold, the first mark may be added to the result sequence of the abnormality detection result of the index, which indicates that the index is abnormal.

And S260, when the reconstruction element is larger than or equal to a preset abnormality detection threshold, adding a second mark to a result sequence of the abnormality detection result.

In the embodiment of the present invention, the second flag may be understood as a flag indicating that the index corresponding to the reconstruction element in the index reconstruction sequence is normal, where the flag may include a number, a symbol, and the like.

Specifically, when the reconstruction element in the index reconstruction sequence is greater than or equal to the corresponding preset abnormality detection threshold, the second flag may be added to the result sequence of the abnormality detection result of the index, which indicates that the index is normal.

S270, sequencing the index reconstruction sequence results according to the value, and extracting the sequencing sequence numbers of the reconstruction elements to form a sequencing sequence.

In the embodiment of the invention, one or more reconstruction elements in the index reconstruction sequence result can be ranked according to the corresponding value, for example, the higher the value of the reconstruction element is, the more front the ranking is, or the lower the value of the reconstruction element is, the more front the ranking is, the ranked sequence number of each reconstruction element can be extracted, the sequence number can represent the position of the reconstruction element in the sequence generated by the whole reconstruction sequence result ranking, and each ranking sequence number can be ranked as a ranking sequence according to the position of the corresponding reconstruction element in the index reconstruction sequence result.

S280, extracting a ranking score standardization conversion function of the ranking score standardization mapping relation.

The ranking score standardization conversion function may be a mapping relation for carrying out standardization adjustment on the ranking score of the reconstruction element, and the ranking score standardization conversion function may include a mapping function for excluding the dimension influence of the ranking score of the reconstruction element.

In the embodiment of the invention, the sorting score standard conversion function can be configured locally in advance, and when multi-index abnormality is required to be sorted, the sorting score standard conversion function can be extracted so as to adjust the sorting of the reconstruction elements based on the sorting table conversion function, thereby solving the problem of sorting abnormality caused by different dimensions of different reconstruction elements.

S290, generating index sorting coefficients of the reconstruction elements according to the sorting score standardized conversion function, the sorting sequence and the abnormality detection result.

The index ranking coefficient may be a coefficient for adjusting the ranking of the reconstruction elements, where the index ranking coefficient may be determined by a ranking score normalization conversion function, a ranking sequence, and an anomaly detection result, and it may be understood that the greater the difference processed by the ranking score normalization conversion function, the greater the index ranking coefficient corresponding to the reconstruction element may be.

In the embodiment of the invention, the reconstruction element in the abnormality can be determined according to the abnormality detection result, the sequence number of the reconstruction element can be found in the sequence, and the sequence number is substituted into the sequence score standardization conversion function so as to realize the standardization mapping of the sequence score of the reconstruction element, thereby generating the index sequence coefficient for adjusting the sequence number of the reconstruction element.

And S2100, adjusting reconstruction elements in the index reconstruction sequence according to the index sequencing coefficient, and sequencing the adjusted index reconstruction sequence according to the value and the size, and then using the sequence as an abnormal index sequence.

In the embodiment of the invention, the values of the reconstruction elements in the index reconstruction sequence can be adjusted according to the index sorting coefficient, after the reconstruction elements are adjusted, the reconstruction elements can be rearranged according to the values, and the sorting among the rearranged reconstruction elements can be used as abnormal index sorting.

According to the technical scheme of the embodiment of the invention, the reconstruction elements in the abnormal index data set are processed by using the variable self-encoder, the reconstruction elements are sequenced into the index reconstruction sequence, the preset abnormal detection threshold values of different reconstruction elements in the index reconstruction sequence are obtained, the reconstruction elements are respectively compared with the corresponding preset abnormal detection threshold values, when the reconstruction elements are smaller than the preset abnormal detection threshold values, the first mark is added to the result sequence of the abnormal detection result, otherwise, the second mark is added to the result sequence, the index reconstruction sequence is sequenced again according to the value of the reconstruction elements, the sequencing sequence of each reconstruction element is formed into the sequencing sequence, the sequencing score standardization conversion function is extracted, the index sequencing coefficient of each reconstruction element is generated according to the sequencing score standardization function, the sequencing sequence and the abnormal detection result, the reconstruction elements in the index reconstruction sequence are adjusted according to the index sequencing coefficient, the adjusted index reconstruction sequence is sequenced according to the value of the sequence, the sequence is sequenced again and then used as the abnormal index, the implicit relation between the abnormal indexes can be achieved, the time sequence characteristics of the abnormal indexes can be maintained, the abnormal indexes can be improved, and the abnormal index accuracy is facilitated.

On the basis of the above embodiment, the processing procedure of the variable self-encoder is realized by the following formula:

wherein R is _X Representing index reconstruction sequence, p _θ (X|Z) represents the conditional probability that X is regenerated after X information is encoded into Z through a neural network, and X is a multidimensional random variable X= { X ₁ ，x ₂ ，…，x _n }，x _i As an abnormality index i, a reconstruction element of the abnormality index iDefined as log (p) _θ (x _i |Z))。

In the embodiment of the invention, a variable automatic-encoding (VAE) may be a depth probability map model, may detect multi-dimensional time series data anomalies, may map random variables in a high latitude space to random variables in a low latitude space through a neural network, may decode the random variables in the low latitude space through the neural network, and may ensure that the decrypted values are similar to the original values in an iterative process, where the variable automatic Encoder may be specifically

On the basis of the above embodiment, generating the index ranking coefficient of each reconstruction element according to the ranking score normalized transfer function, the ranking sequence and the anomaly detection result includes:

Sequentially extracting the mark information of the corresponding reconstruction element in the abnormal detection result; substituting the sequence numbers of the corresponding reconstruction elements in the sequence into the sequence score standardized conversion function to determine index sequence coefficients when the marking information is the first mark; when the marking information is the second marking, setting the index ordering coefficient of the corresponding reconstruction element to be 0;

wherein the ranking score normalization transfer function comprises at least:

n represents the number of the abnormal detection results, alpha is the change rate, beta is the offset coefficient, and n is larger than the value of alpha.

In the embodiment of the invention, the marking information of each reconstruction element in the abnormal detection result can be extracted, the marking information can represent whether the corresponding reconstruction element is an abnormal index, when the marking information is a first mark, the corresponding reconstruction element can be an abnormal index, then the sequence number of the reconstruction element in the sequence can be substituted into the sequence number standardized conversion function, the sequence number is processed through the sequence number standardized conversion function to generate the index sequence coefficient of the reconstruction element, and correspondingly, when the marking information is a second mark, the corresponding reconstruction element can be a non-abnormal index, and the index sequence coefficient of the reconstruction element can be directly set to 0, so that the sequence of the reconstruction element in the sequence is reduced. Specifically, in the embodiment of the present invention, the determination of the index ranking coefficient of the reconstruction element of the first marker may be determined by a ranking score normalization conversion function:

Wherein n represents the number of the abnormal detection results, alpha is the change rate, beta is the offset coefficient, and n is greater than the value of alpha.

In an exemplary implementation manner, fig. 3 is an exemplary diagram of an anomaly index sorting method provided by the embodiment of the present invention, and an object of the embodiment of the present invention is to provide an index weighted moving average multi-index anomaly sorting method, which can quantify characteristic changes when each index is abnormal, and can obtain a more accurate comprehensive anomaly index sorting according to historical anomaly sorting conditions of each index. Referring to FIG. 3, to order multi-index anomalies, the method includes the following: (1) Generating a reconstruction sequence of each index, and identifying abnormal indexes by defining a threshold on the reconstruction sequence; (2) calculating index ranking scores and normalizing; (3) Comprehensively considering the abnormal sequencing condition of each index in the last period of time to obtain more accurate index comprehensive abnormal sequencing. The specific implementation steps comprise:

1) Multi-index reconstruction sequence generation and threshold anomaly detection

Let the multi-index time series data be expressed as multi-dimensional random variable x= { X ₁ ，x ₂ ，…，x _n And are independently co-distributed, variable x _i Indicating index i. Variable Auto-Encoder (VAE) is a generic term for a class of depth probability map models applied to multidimensional time series data anomaly detection through neural networks Encoding a random variable X in a high-dimensional space into a random variable Z in a low-dimensional space, decoding Z into X 'through a neural network theta, and ensuring that the decoded X' is similar to an original value X as much as possible in an iterative process, wherein the VAE can be finally output as follows:

wherein p is _θ (X|Z) represents the conditional probability that X is regenerated after X information is encoded into Z through a neural network, and X is a multidimensional random variable X= { X ₁ ，x ₂ ，…，x _n }，x _i For the abnormality index i, R is defined _X Reconstructing the sequence for the whole and indicatingi reconstruction sequenceDefined as log (p) _θ (x _i Z). Thus, the reconstructed sequence of the index can be obtained from the encoder by variation, the value range of the reconstructed sequence is +.>Smaller values in the sequence represent more abnormal corresponding indicators, so we quantitatively analyze the indicator anomalies using the values of the reconstructed sequence as indicator anomaly scores.

In order to identify abnormal indexes from a plurality of indexes, a threshold analysis method is generally adopted, but because the analysis indexes are numerous, a great amount of labor cost is brought by manually setting a threshold value to identify the abnormal indexes, a POT (Peak Over Threshold) model based on extremum theory is selected, and the model has the advantages that the data distribution is not needed to be assumed, and the threshold value is automatically set by selecting parameters. Reconstruction of sequence R for multiple indices by POT model _X The threshold value automatically set isThe final multi-index anomaly detection result is:

wherein 1 represents an abnormality, 0 represents a normal,

2) Index ranking score normalization

Since the range of Rx reconstruction sequence isThe degree of abnormality of each index is different, the reconstructed sequence values have orders of magnitude difference, and the accuracy of the comprehensive abnormal sequencing of the indexes is affected due to the difference of the dimensions. Therefore, in order to be dimensionless, the index is required to beThe ranking score is normalized to obtain an index ranking coefficient, which can be analogically related to the relationship between covariance and correlation coefficient. Let the reconstruction sequence of the multiple index at time t be +.>The abnormality detection result isn is the index number. The ranking score normalized transfer function is as follows:

where x= {1,2, …, n }, α is the rate of change control rate of change, β is the lower limit of the offset coefficient control, and n > α, the range is [ β, β+1], and the transfer function image is as in fig. 2. The function is a monotonically decreasing concave function, if the ranking order is defined as an abscissa x, the ranking score is defined as an ordinate, the higher the ranking order is, the higher the distinguishing degree of the ranking score is, and the lower the distinguishing degree of the ranking score is after the ranking order is. Fig. 4 is an exemplary diagram of a normalized conversion function of ranking score according to an embodiment of the present invention, in a multi-index abnormal ranking scenario, because there are numerous abnormal indexes, we usually only pay attention to top k abnormal indexes after ranking, and referring to fig. 4, the conversion function can well measure abnormal ranking conditions of indexes.

Reconstruction sequence for multiple index t timeSequencing according to ascending order to obtain sequencing sequenceWherein->Representing the ranking number of index i, defining the ranking number as a ranking score for measuring the degree of abnormality between multiple indexes, wherein the smaller the score is compared with other indexesMarked anomaly, value range of [0, n ]]. Bonding ofAnd Y pairs of ranking scores->And (3) standardized mapping, wherein the final standardized index sorting coefficient is as follows:

wherein the method comprises the steps ofThe score mapped to the normalization function according to the ranking number when abnormality is detected in the index i at time t.

3) Index comprehensive anomaly ordering

Since the index anomalies have time continuity and random jitter characteristics, the history anomalies and the influence of random factors are needed to be considered for the index anomaly sorting, so that the problems are solved by adopting an index moving weighted average method. The exponential moving weighted average method is to give different weights to (1) the historical observed value and the current observed value respectively, calculate a moving average value according to the different weights, (2) the weighted coefficient of each numerical value is exponentially decreased along with time, the numerical value weighted coefficient is larger when the numerical value weighted coefficient is closer to the current moment, and the trend can be described on a continuous time line, meanwhile, short-term fluctuation is smoothed, and random shaking noise is filtered.

Let the history time window be [ t ] _s ，t _c ]，t _c T is the abnormality detection time point _s For the starting time point of the historical anomaly detection, the historical normalized index ranking coefficient sequence isWherein the method comprises the steps ofIs index i at [ ts, tc ]]Criteria within a windowOrdering score of the transformation, beta is weight, index i is t _c The moment comprehensive anomaly score calculation formula:

index i is at t _s Time-of-day comprehensive anomaly ranking score:

similarly, index i is at t _c Time-of-day comprehensive anomaly ranking score:

FIG. 5 is an exemplary graph of an anomaly score sequence for a multi-index composite anomaly score according to an embodiment of the present inventionThe multi-index comprehensive anomaly score sequences are ordered, and a bar chart of 5 indexes with the largest comprehensive anomaly score is selected and displayed as shown in figure 5. Fig. 6 is a schematic diagram of an implementation of a multi-index anomaly ordering algorithm based on an exponential moving weighted average according to an embodiment of the present invention, and the steps may be implemented by using the algorithm shown in fig. 6 on the basis of the embodiment of the present invention. Fig. 7 is a schematic diagram of an implementation of an anomaly index sorting algorithm based on anomaly times according to an embodiment of the present invention, and in other embodiments of the present invention, the above steps may be implemented by using the algorithm shown in fig. 7. Fig. 8 is a schematic diagram of an implementation of an anomaly index sorting algorithm based on the number of anomalies in a recent period of time of each index according to an embodiment of the present invention, and in other embodiments of the present invention, the above steps may be implemented by using the algorithm shown in fig. 8.

In an embodiment, fig. 9 is a schematic structural diagram of an abnormality index sorting device according to an embodiment of the invention. The present embodiment can perform the above-described implementation. The embodiment is applicable to the situation that various abnormal indexes are sequenced, and the device can be realized in a hardware/software mode and can be configured in electronic equipment.

As shown in fig. 9, the abnormality index sorting apparatus provided in the present embodiment includes: a sequence construction module 401, an anomaly detection module 402, and an index ordering module 403, wherein:

the sequence constructing module 401 is configured to generate an index reconstruction sequence according to the obtained abnormal index data set.

An anomaly detection module 402, configured to determine an anomaly detection result based on a preset anomaly detection threshold and an index reconstruction sequence.

The index ranking module 403 is configured to generate an abnormal index ranking within an abnormal detection time according to the abnormal detection result and the ranking score standardized mapping relationship.

According to the technical scheme provided by the embodiment of the invention, the sequence construction module is used for generating the index reconstruction sequence from the acquired abnormal indexes, the abnormal detection module is used for determining the abnormal detection result of the index reconstruction sequence according to the preset abnormal detection threshold, and the index sequencing module is used for determining the abnormal index sequencing corresponding to the abnormal detection result according to the sequencing score standardized mapping relation in the abnormal detection time, so that the problem that the continuity of index dimension and time dimension is not considered in the conventional abnormal index sequencing can be solved, the relevance among indexes can be improved, the accuracy of the abnormal index sequencing can be enhanced, the processing efficiency of the abnormal indexes can be improved, and the stability of a system can be enhanced.

Based on the above embodiment, the sequence constructing module 401 includes:

and the reconstruction element generation unit is used for calling the variation self-encoder to process the abnormal index in the abnormal index data set so as to generate reconstruction elements forming an index reconstruction sequence.

And the index reconstruction sequence generation unit is used for arranging the reconstruction elements as an index reconstruction sequence.

Based on the above embodiment, the abnormality detection module 402 includes:

the threshold setting unit is used for acquiring preset abnormal detection thresholds of different reconstruction elements in the corresponding index reconstruction sequence set by the peak value exceeding threshold model based on the extremum theory.

And the threshold comparison unit is used for comparing the reconstruction elements with corresponding preset abnormal detection thresholds respectively.

And the first mark adding unit is used for adding the first mark to the result sequence of the abnormal detection result when the reconstruction element is smaller than the preset abnormal detection threshold value.

And the second mark adding unit is used for adding the second mark to the result sequence of the abnormal detection result when the reconstruction element is larger than or equal to the preset abnormal detection threshold value.

Based on the above embodiment, the index sorting module 403 includes:

the sequencing sequence forming unit is used for sequencing the reconstruction elements according to the value of the index reconstruction sequence result and extracting the sequencing sequence numbers of the reconstruction elements to form a sequencing sequence.

And the conversion function extraction unit is used for extracting the ranking score standardized conversion function of the ranking score standardized mapping relation.

And the sequencing coefficient generation unit is used for generating index sequencing coefficients of each reconstruction element according to the sequencing score standardized conversion function, the sequencing sequence and the abnormality detection result.

The abnormal index sorting unit is used for adjusting the reconstruction elements in the index reconstruction sequence according to the index sorting coefficient, and re-sorting the adjusted index reconstruction sequence according to the value and the size to be used as the abnormal index sorting.

On the basis of the above embodiment, the ranking coefficient generating unit includes:

and the mark information extraction subunit is used for sequentially extracting the mark information of the corresponding reconstruction element in the abnormal detection result.

And the index sorting coefficient determining subunit is used for substituting the sorting serial numbers of the corresponding reconstruction elements in the sorting sequence into the sorting score standardized conversion function to determine the index sorting coefficient when the marking information is the first mark.

And the index sorting coefficient setting subunit is used for setting the index sorting coefficient of the corresponding reconstruction element to 0 when the marking information is the second marking.

Wherein the ranking score normalization transfer function comprises at least:

On the basis of the above embodiment, the abnormality index ranking apparatus further includes:

the abnormality index ranking generation module is used for counting the total abnormality times of all the reconstruction elements in the index reconstruction sequence in the abnormality detection time range based on a preset abnormality detection threshold value, and generating the abnormality index ranking according to the total abnormality times of all the reconstruction elements.

The abnormality index ranking generation module is used for counting average abnormality scores of all reconstruction elements in the index reconstruction sequence based on a preset abnormality detection threshold value, and generating abnormality index ranking according to the average abnormality scores of all reconstruction elements.

The abnormal index sorting device provided by the embodiment of the invention can execute any abnormal index sorting method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of the executing method. Reference is made to the description of any method embodiment of the invention for details not described in this embodiment.

In an embodiment, fig. 10 is a schematic structural diagram of an electronic device implementing the abnormality index ranking method according to the embodiment of the invention. Electronic device 50, which may be used to implement embodiments of the present invention, is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 10, the electronic device 50 includes at least one processor 51, and a memory, such as a Read Only Memory (ROM) 52, a Random Access Memory (RAM) 53, etc., communicatively connected to the at least one processor 51, in which the memory stores a computer program executable by the at least one processor, and the processor 51 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 52 or the computer program loaded from the storage unit 58 into the Random Access Memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, RAM 52 and RAM 53 are connected to each other by a bus 54. An input/output (I/O) interface 55 is also connected to bus 54.

Various components in the electronic device 50 are connected to the I/O interface 55, including: an input unit 55 such as a keyboard, a mouse, etc.; an output unit 57 such as various types of displays, speakers, and the like; a storage unit 58 such as a magnetic disk, an optical disk, or the like; and a communication unit 59 such as a network card, modem, wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.

The processor 51 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 51 performs the various methods and processes described above, such as the anomaly index ranking method.

In some embodiments, the deployment method of the monitoring points may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 58. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 50 via the ROM 52 and/or the communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the anomaly index ranking method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the anomaly index ordering method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. An anomaly index ordering method, the method comprising:

determining an abnormality detection result based on a preset abnormality detection threshold and the index reconstruction sequence;

generating an abnormality index ranking within the abnormality detection time according to the abnormality detection result and the ranking score standardization mapping relation.

2. The method of claim 1, wherein generating the index reconstruction sequence from the acquired anomaly index dataset comprises:

Invoking a variation self-encoder to process the abnormal index in the abnormal index data set to generate a reconstruction element forming the index reconstruction sequence;

and arranging the reconstruction elements as the index reconstruction sequence.

3. The method of claim 2, wherein the process of the variation self-encoder is implemented by the following formula:

wherein R is _X Representing index reconstruction sequence, p _θ (X|Z) represents the conditional probability that X is regenerated after X information is encoded into Z through a neural network, and X is a multidimensional random variable X= { X ₁ ,x ₂ ,…,x _n }，x _i As an abnormality index i, a reconstruction element of the abnormality index iDefined as log (p) _θ (x _i |Z))。

4. The method of claim 1, wherein the determining an anomaly detection result based on a preset anomaly detection threshold and the index reconstruction sequence comprises:

acquiring the preset abnormal detection threshold value of different reconstruction elements in the index reconstruction sequence, which is set by the peak value exceeding threshold value model based on the extremum theory;

comparing the reconstruction elements with the corresponding preset abnormal detection thresholds respectively;

when the reconstruction element is smaller than the preset abnormality detection threshold, adding a first mark to a result sequence of the abnormality detection result;

And adding a second mark to a result sequence of the abnormality detection result when the reconstruction element is greater than or equal to the preset abnormality detection threshold.

5. The method of claim 1, wherein generating an anomaly index ranking within an anomaly detection time according to the anomaly detection result and ranking score normalized mapping relationship comprises:

sequencing the index reconstruction sequence results according to the value, and extracting sequencing numbers of the reconstruction elements to form a sequencing sequence;

extracting a ranking score standardization conversion function of the ranking score standardization mapping relation;

generating index sorting coefficients of the reconstruction elements according to the sorting score standardized conversion function, the sorting sequence and the abnormality detection result;

and adjusting the reconstruction elements in the index reconstruction sequence according to the index sequencing coefficient, and sequencing the adjusted index reconstruction sequence according to the value and the size, and then sequencing the index reconstruction sequence as the abnormal index.

6. The method of claim 5, wherein said generating an index ranking coefficient for each of said reconstruction elements in accordance with said ranking score normalized transfer function, said ranking sequence, and said anomaly detection result comprises:

Sequentially extracting the mark information corresponding to the reconstruction element in the abnormal detection result;

substituting the sequence number of the reconstruction element in the sequence into the sequence score standardization conversion function to determine the index sequence coefficient when the mark information is the first mark;

setting the index ranking coefficient corresponding to the reconstruction element to 0 when the marker information is a second marker;

wherein the ranking score normalization transfer function comprises at least:

and n represents the number of the abnormal detection results, alpha is the change rate, beta is the offset coefficient, and n is larger than the value of alpha.

7. The method as recited in claim 1, further comprising:

counting the total abnormal times of all the reconstruction elements in the index reconstruction sequence in an abnormal detection time range based on the preset abnormal detection threshold value, and generating the abnormal index ranking according to the total abnormal times of the reconstruction elements;

and counting the average anomaly scores of all the reconstruction elements in the index reconstruction sequence within the anomaly detection time range based on the preset anomaly detection threshold value, and generating the anomaly index ranking according to the average anomaly scores of the reconstruction elements.

8. An abnormality index ranking apparatus, comprising:

the abnormality detection module is used for determining an abnormality detection result based on a preset abnormality detection threshold value and the index reconstruction sequence;

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the anomaly index ranking method of any one of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to implement the anomaly index ranking method of any one of claims 1-7 when executed.