CN110071913A - A kind of time series method for detecting abnormality based on unsupervised learning - Google Patents

Abstract

The present invention relates to a kind of time series method for detecting abnormality based on unsupervised learning, comprising: time series data is subjected to cutting in the position of its significant changes, and setting length is padded to the data segment after each cutting；Multiple data segments training one using the time series cutting under normal condition and after filling up is used for the neural network of abnormality detection；Multiple data segments by time series cutting to be detected and after filling up are detected as the input of abnormality detection model, and output abnormality score；Judge whether abnormal score is more than threshold value, if it has, then judgement is abnormal, conversely, then judging no exceptions.Compared with prior art, the present invention has the advantages that not depend on that markd abnormal data, not lose data information, performance excellent etc..

Description

A Time Series Anomaly Detection Method Based on Unsupervised Learning

technical field

The invention relates to an anomaly detection method, in particular to a time series anomaly detection method based on unsupervised learning.

Background technique

Anomaly Detection is a means of detecting anomalies in data, where "abnormal" refers to patterns that do not conform to normal behaviors. For example, in the field of network traffic analysis, normal patterns refer to normal network access behaviors, and abnormal patterns are Refers to the behavior of network intruders. Anomaly detection is used in many fields, such as medical and health field, network security field, financial security field, system maintenance field and so on.

Time series refers to a series of data in the form of <timestamp, data>. Time series is often used to record data such as system operating status and human health data in real time. By analyzing time series data, it is possible to determine where the system is located. state, and analyze system behavior to assist humans in decision-making. In real life, many systems use time series data to record system operating status, such as website system traffic and server CPU operating status. In addition, in the medical and health field, electrocardiogram data, disease development and change data, etc. are also represented by time series.

Abnormalities in the time series can often reflect the abnormality of the system. For example, in the website system, database blockage or deadlock will be reflected in the monitoring data of the database. In the ECG data, the abnormality caused by heart disease will also be reflected in the ECG. in the data. Therefore, anomaly detection for time series data helps people to detect anomalies as early as possible and take appropriate measures to avoid them.

At present, anomaly detection is mainly divided into two types: supervised methods and unsupervised methods. The supervised method requires a large amount of data marked with anomalies for model training. However, anomalies are often accidental, so it is difficult to obtain in real life. A lot of abnormal data. Therefore, we consider using unsupervised methods to achieve anomaly detection.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a time series anomaly detection method based on unsupervised learning in order to overcome the above-mentioned defects of the prior art.

The object of the present invention can be realized through the following technical solutions:

An unsupervised learning-based time series anomaly detection method, including:

Divide the time series data at its significantly changed position, and fill each segmented data segment to a set length;

The anomaly detection model is trained using multiple data segments that are segmented and filled in the normal state of the time series as input;

The multiple data segments that are divided and filled by the time series to be detected are used as the input of the anomaly detection model to detect, and the anomaly score is output;

It is judged whether the abnormality score exceeds the threshold value, if yes, it is judged that an abnormality has occurred, otherwise, it is judged that no abnormality has occurred.

The process of segmenting the time series data at its significantly changed positions specifically includes:

Find all extreme points of time series data;

Then, the position of the extreme value point with the larger absolute value is used as the split point, and the time series is split into multiple data segments, wherein the split point is determined by the manually set absolute value threshold of the extreme value point of the data.

The anomaly detection model includes a data compressor and a Gaussian mixture model estimator. The data compressor adopts a many-to-many LSTM network structure, and the Gaussian mixture model estimator adopts a multi-layer perceptron structure.

The compression process of the data compressor includes:

Compress and reconstruct the data segment;

Calculate the relative distance and cosine distance before and after compression;

The relative distance, cosine distance, and the output of the hidden layer unit of the LSTM network are synthesized as the input of the Gaussian mixture model estimator.

The mathematical expression of the relative distance is:

Among them: r is the relative distance, L is the length of the time series contained in the data segment, x _i is the element in the time series contained in the data segment, and x' is the element in the time series obtained after recombination.

The mathematical expression of the cosine distance is:

Where: c is the cosine distance, ||·|| is the norm, x _i is the element in the time series contained in the data segment, and x′ is the element in the time series obtained after recombination.

The training process of the Gaussian mixture model estimator includes:

The output of the data compressor is received and mapped to a K-dimensional vector, where K is the number of Gaussian distributions in the model,

Based on each element of the K-dimensional vector, the mixture probability, mean and covariance of each Gaussian distribution are obtained;

The detection process of the Gaussian mixture model includes:

The output of the data compressor is received and an anomaly score is calculated.

The mathematical expression of the abnormal score is:

Among them: Score(z) is the abnormal score, is the mixture probability of the kth Gaussian distribution, is the covariance of the kth Gaussian distribution, z is the output of the data compressor, is the mean of the kth Gaussian distribution, for The inverse matrix of .

The mixture probability of the k-th Gaussian distribution is:

The mean of the k-th Gaussian distribution is:

The covariance of the kth Gaussian distribution is:

Where: N is the total number of training samples, is the k-th dimension data of the ith training sample, and _zi is the ith training sample.

The data compressor and the Gaussian mixture model estimator are trained in an end-to-end manner, and the training objective function is as follows:

Among them: J is the objective function, λ ₁ and λ ₂ are manually set parameters, x _i is the time series included in the i-th data segment, and x′ is the time series after the time series included in the i-th data segment is sufficient , for punishment.

Compared with the prior art, the present invention has the following beneficial effects:

1) Before model training and anomaly detection, the time series data is segmented at its significantly changed position, and the segmented sequence data is used for model training. Conventional anomaly detection methods use a fixed-length time window to slide and select time slices, resulting in a large amount of redundant information generated in the segmented sequence data, which is not conducive to the feature learning of neural networks. On the other hand, the use of fixed-length time series cannot be detrimental to The data in the characterization time window has a fixed meaning, and it is impossible to compare time series with similar physical meanings.

2) Using the method based on density estimation, the divided training samples are regarded as samples sampled from an unknown Gaussian mixture distribution, and a neural network is used to estimate the Gaussian mixture model of the unknown distribution. In the conventional method, only the probability of the entire data is considered. distribution without considering the different feature distributions of each segment of the data.

3) In the training phase, the segmented data is sent to a many-to-many recurrent neural network for reconstructing the training samples, the output of the last step of the hidden layer of the neural network, and the relationship between the reconstructed sequence and the original sequence. The relative distance and the cosine distance are simultaneously fed into a neural network for estimating the parameters of the Gaussian mixture model, and conventional methods only use the reconstruction error as the basis for estimating the Gaussian mixture model.

Description of drawings

1 is a schematic flow chart of the main steps of the present invention;

Fig. 2 is the model training flow chart;

Fig. 3 is the neural network model structure schematic diagram used in the present invention;

Fig. 4 is the abnormal prediction flow chart;

FIG. 5 is a schematic diagram showing the performance comparison between the method of the present invention and the performance of the existing method.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation manner and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

A time series anomaly detection method based on unsupervised learning mainly includes two steps: model training and anomaly detection, as shown in Figure 1, including:

The time series data is segmented at its significantly changed position, and each segmented data segment is filled to a set length; in order to achieve the above requirements, the flow chart of the model training steps of the present invention is shown in FIG. 2 . The data preprocessing includes two steps:

Data segmentation: First find all the extreme points of the sequence, and then use the extreme point position with a larger absolute value as the segmentation point, and segment the time series into multiple data segments, where the segmentation points are manually set data The absolute value threshold of the extreme point is determined.

Data padding: Use 0 to pad multiple split sequences to the input length of the anomaly detection model.

The divided and filled data segments are used as independent samples for training subsequent models.

Train a neural network for anomaly detection using multiple segments of time series sliced and filled in the normal state;

The multiple data segments that are divided and filled by the time series to be detected are used as the input of the anomaly detection model to detect, and the anomaly score is output;

It is judged whether the abnormality score exceeds the threshold value, if yes, it is judged that an abnormality has occurred, otherwise, it is judged that no abnormality has occurred.

As shown in Figure 3, the anomaly detection model includes a data compressor and a Gaussian mixture model estimator, and the data compressor adopts a many-to-many LSTM network structure.

The time step size used in the model structure in the data compressor is larger than all possible input sample lengths. The time series samples in the input LSTM model are denoted as x=[x ₁ ,x ₂ ,…,x _L ], and the time series reconstructed by the LSTM network are denoted as x′=[x′ ₁ ,x′ ₂ ,…,x′ _L ], where L is the length of the time series, then the loss function for the training of the LSTM network is as follows:

where x _i is the ith element in a time series sample, x′ _i is the ith element in the reconstructed time series sample, and L is the length of the time series.

Its compression process includes:

Compress and reconstruct the data segment;

Calculate the relative distance and cosine distance before and after compression;

The relative distance, cosine distance, and the output of the hidden layer unit of the LSTM network are synthesized as the input of the Gaussian mixture model estimator.

The mathematical expression for relative distance is:

Among them: r is the relative distance, L is the length of the time series contained in the data segment, x _i is the element in the time series contained in the data segment, and x' is the element in the time series obtained after recombination.

The mathematical expression for cosine distance is:

Where: c is the cosine distance, ||·|| is the norm, x _i is the element in the time series contained in the data segment, and x′ is the element in the time series obtained after recombination.

The Gaussian mixture model estimator adopts a multi-layer perceptron structure (Multilayer perceptions, MLP). Given the number K of Gaussian distributions used by the Gaussian mixture model, the Gaussian mixture model estimator is used to estimate the three parameters of the K Gaussian distributions, which are the mixture probability Φ, the mean μ, and the covariance Σ.

The parameter estimation process is as follows:

(1) First use a multilayer neural network to map input samples into K-dimensional vectors to determine the data used for estimating each Gaussian distribution. The mapping process is:

p=MLN(z; θ)

where z is the data input to the Gaussian mixture model estimator, MLN( ) is a multi-layer neural network whose parameters are θ, and softmax( ) is the softmax function, is the sample used to estimate the parameters of the Gaussian mixture model.

(2) The parameters of the Gaussian mixture model: the estimation formulas of the mixture probability Φ, the mean μ, and the covariance Σ are as follows:

in and are the mixture probability, mean, and covariance of the kth Gaussian distribution, respectively, is the k-th dimension data of the ith training sample, _zi is the ith training sample, and N is the total number of training samples.

The formula for the anomaly score output by the Gaussian mixture model estimator is as follows:

where z is the data input to the estimated Gaussian mixture model estimator, K is the given number of Gaussian distributions, and are the mixture probability, mean, and covariance of the kth Gaussian distribution, respectively.

The data compressor and the Gaussian mixture model estimator are trained in an end-to-end manner, and the training objective function is as follows:

Among them: J is the objective function, λ ₁ and λ ₂ are manually set parameters, x _i is the time series included in the i-th data segment, and x′ is the time series after the time series included in the i-th data segment is sufficient , is the penalty term, and its formula is as follows:

where: d is the dimension of the sample z input to the Gaussian mixture model estimator, and K is the given number of Gaussian distributions.

The method of determining the data segment for anomaly detection is as follows: Calculate the variance of the Gaussian distribution generated by training for each segment of the segmented data, and select the data segment that can generate the smallest variance as the data sent to the anomaly detection model in the anomaly detection stage.

The flowchart of anomaly detection steps is shown in Figure 4, where data preprocessing includes two steps:

(1) Data segmentation: First, find all the extreme points of the sequence, and then use the position of the extreme point with the largest absolute value as the segmentation point.

(2) Data padding: padding multiple sequences that have been segmented to the input length of the anomaly detection model.

(3) Pick out the data segment determined in the model training stage for anomaly detection.

The neural network model is the anomaly detection model trained in the model training step, and τ is the artificially given anomaly score classification threshold.

The performance of the above method is evaluated on the Two-lead ECG data set, and AUC and ROC are used as indicators to measure performance. The AUC of the method proposed by the present invention is 0.8396573. Figure 5 lists the performance of the method proposed by the present invention and other methods Performance comparison data on the same dataset, where Seq2Cluster is the proposed method. It can be seen that the method proposed in the present invention is superior to all existing unsupervised anomaly detection methods of the same kind, which shows that the anomaly detection method described in this patent is advanced.

Claims (10)

1. a time series anomaly detection method based on unsupervised learning, is characterized in that, comprises: Divide the time series data at its significantly changed position, and fill each segmented data segment to a set length; The anomaly detection model is trained using multiple data segments that are segmented and filled in the normal state of the time series as input; The multiple data segments that are divided and filled by the time series to be detected are used as the input of the anomaly detection model to detect, and the anomaly score is output; It is judged whether the abnormality score exceeds the threshold value, if yes, it is judged that an abnormality has occurred, otherwise, it is judged that no abnormality has occurred. 2. The method for detecting anomalies in time series based on unsupervised learning according to claim 1, wherein the method for segmenting the time series data at its significantly changed position specifically includes: Find all extreme points of time series data; Then, the position of the extreme value point where the absolute value is too expensive to set the threshold value is used as the segmentation point to be divided into multiple data segments. 3. The method for detecting anomalies in time series based on unsupervised learning according to claim 1, wherein the anomaly detection model comprises a data compressor and a Gaussian mixture model estimator, and the data compressor adopts multiple pairs of Multiple LSTM network structures, the Gaussian mixture model estimator adopts a multi-layer perceptron structure. 4. The method for detecting anomalies in time series based on unsupervised learning according to claim 3, wherein the compression process of the data compressor comprises: Compress and reconstruct the data segment; Calculate the relative distance and cosine distance before and after compression; The relative distance, cosine distance, and the output of the hidden layer unit of the LSTM network are synthesized as the input of the Gaussian mixture model estimator. 5. a kind of time series anomaly detection method based on unsupervised learning according to claim 4, is characterized in that, the mathematical expression of described relative distance is: Among them: r is the relative distance, L is the length of the time series contained in the data segment, x _i is the data segment The elements in the included time series, x' is the element in the time series obtained after recombination. 6. a kind of time series anomaly detection method based on unsupervised learning according to claim 4, is characterized in that, the mathematical expression of described cosine distance is: Where: c is the cosine distance, ||·|| is the norm, x _i is the element in the time series contained in the data segment, and x′ is the element in the time series obtained after recombination. 7. The method for detecting anomalies in time series based on unsupervised learning according to claim 4, wherein the training process of the Gaussian mixture model estimator comprises: The output of the data compressor is received and mapped into a K-dimensional vector using a multilayer neural network, where K is the number of Gaussian distributions in the model, Based on each element of the K-dimensional vector and using the multilayer perceptron model, the mixture probability, mean and covariance of each Gaussian distribution are obtained; The detection process of the Gaussian mixture model includes: The output of the data compressor is received and an anomaly score is calculated. 8. a kind of time series abnormal detection method based on unsupervised learning according to claim 7, is characterized in that, the mathematical expression of described abnormal score is: Among them: Score(z) is the abnormal score, is the mixture probability of the kth Gaussian distribution, is the covariance of the kth Gaussian distribution, z is the output of the data compressor, is the mean of the kth Gaussian distribution, for The inverse matrix of . 9. A kind of time series anomaly detection method based on unsupervised learning according to claim 8, is characterized in that, The mixture probability of the k-th Gaussian distribution is: The mean of the k-th Gaussian distribution is: The covariance of the kth Gaussian distribution is: Where: N is the total number of training samples, is the k-th dimension data of the ith training sample, and _zi is the ith training sample. 10. The method for detecting anomalies in time series based on unsupervised learning according to claim 3, wherein the data compressor and the Gaussian mixture model estimator are trained in an end-to-end manner, and the training objective function is as follows : Among them: J is the objective function, λ ₁ and λ ₂ are manually set parameters, x _i is the time series included in the i-th data segment, and x′ is the time series after the time series included in the i-th data segment is sufficient , for punishment.