CN113159163A - Lightweight unsupervised anomaly detection method based on multivariate time series data analysis - Google Patents

Lightweight unsupervised anomaly detection method based on multivariate time series data analysis Download PDF

Info

Publication number
CN113159163A
CN113159163A CN202110418526.1A CN202110418526A CN113159163A CN 113159163 A CN113159163 A CN 113159163A CN 202110418526 A CN202110418526 A CN 202110418526A CN 113159163 A CN113159163 A CN 113159163A
Authority
CN
China
Prior art keywords
abnormal
data
score
model
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110418526.1A
Other languages
Chinese (zh)
Inventor
刘振涛
樊谨
汪森
陈金华
冯龙超
匡振中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110418526.1A priority Critical patent/CN113159163A/en
Publication of CN113159163A publication Critical patent/CN113159163A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

The invention discloses a lightweight unsupervised anomaly detection method based on multivariate time series data analysis. The invention comprises two models: a detection model and an inference model; the detection model extracts time dependency characteristics of the captured multivariate time sequence data through a random convolutional neural network, and then encodes and decodes the multivariate time sequence data after characteristics are extracted through a deep Bayesian network, and the detection model can determine a detection accuracy range; the inference model is composed of a score attention unit, a threshold value automatic selection unit and a point adjustment unit, wherein the score attention unit adopts an attention mechanism to enlarge the characteristic difference between abnormal data and normal data and provides a theoretical basis for abnormal interpretation, the threshold value automatic selection unit can automatically calculate a threshold value, the point adjustment unit can simulate the real abnormal generation process, and the inference model can improve the accuracy, stability and interpretability of abnormal detection. The invention can deal with the rapidly-increased data scale and the complicated and changeable abnormal variety.

Description

Lightweight unsupervised anomaly detection method based on multivariate time series data analysis
Technical Field
The invention belongs to the field of machine learning anomaly detection, and particularly relates to a lightweight unsupervised anomaly detection method based on multivariate time series data analysis.
Background
Anomalies are individuals or subsets of observations that differ significantly from the ontology of observations, and these individuals or subsets deviate from the original generation pattern. Anomaly detection is the process of finding these individuals or subsets. The multivariate time sequence data is a numerical sequence set with precedence relationship, and particularly, each element in the numerical sequence is a multidimensional vector; the multivariate time sequence data can describe the state of the observation value body and imply the change rule of the indication phenomenon, so that the analysis of the multivariate time sequence data is very important in the field of anomaly detection.
The traditional supervised learning method needs labeled data for model training and can only identify the known abnormal type, so that the application is very limited. The unsupervised multivariate timing data anomaly detection can be roughly divided into three categories:
and (5) reducing the dimensionality. When multiple sensors monitor a single system, there is often a relationship between the values generated by each sensor, and dimensionality reduction is intended to identify and abstract the key relationships between these attributes. A common method of dealing with multivariate situations is Principal Component Analysis (PCA), which reduces the overall size of the system under study. Projection pursuit offers another way to reduce the dimensionality of multivariate systems, but like PCA, it can cost a significant amount of computation. The Auto Encoder (Auto Encoder) reduces the length of the hidden layer and can produce the same effect as the PCA method, but the computation cost is much lower. Convolutional variational autocoder (CNN-VAE) greatly reduces the size, complexity and training cost of the autocoder, making it more suitable for industrial internet of things, but it is deficient in timing dependency capture. The LSTM is used for replacing a feedforward network in the VAE, the LSTM and the VAE are combined together, and the problem of lack of timing dependence capture can be solved to a certain extent.
And (6) clustering. In previous research, an overall method for performing anomaly detection by using an isolation tree (called an isolation forest) has been proposed, and compared with a class of SVMs, random forests and the like, the method has a good effect. Multi-core anomaly detection (MKAD) uses kernel functions to learn similarity measures between variables in a data stream and uses One-class SVM to perform classification tasks.
Other methods. A series of methods for capturing time dependence in a multivariate time series using RNN have been proposed. The neural network based on LSTM and GRU is obviously superior to the clustering method in various abnormal detection. The CNN, in combination with a trainable wavelet transform layer, is able to identify progressive drift and input data distribution over conceptual tradeoffs over time.
The multivariate time series anomaly detection random recurrent neural network proposed by omnianomallly can learn multivariate time series with random variable chaining and flat normalized flow, but the model is too large and training costs are too high. A GAN-based autoencoder architecture can learn how to amplify reconstruction errors of input information containing anomalies. Compared with Omnianally, the method has great improvement on training time, but the practicability and stability of the method are greatly reduced due to the problems of mode collapse, difficulty in convergence, difficulty in training and the like inherent in the method, so that the method is not easy to use in practice. Compared with the method, the lightweight unsupervised anomaly detection method based on the multivariate time sequence data analysis can clearly model the time dependence and randomness of the time sequence, and the reasoning model can better improve the accuracy of the model and realize the anomaly interpretation. In addition, the scale of the device is greatly reduced, and the stability and the practicability of the device are better improved.
Disclosure of Invention
The invention aims to solve the technical problems of reducing the scale of model training as low as possible and improving the stability and the usability of an abnormality detection method on the premise of ensuring high accuracy of abnormality detection, so that the abnormality detection method can be deployed and used on equipment with limited resources and computing power.
In the anomaly detection based on the supervised learning method, the data collection difficulty with labels is large, the feasibility is low, the novel anomaly which occurs cannot be judged, and the practicability and the stability are greatly reduced. The traditional RNN-based anomaly detection method has the advantages of large model scale, high training difficulty, long time consumption and difficulty in being applied to resource and computing-capacity-limited equipment.
The technical scheme adopted by the invention for solving the technical problems is as follows: the random convolution neural network is used for replacing the traditional neural network based on RNN, and then the processed data is reconstructed according to the variational encoder technology, so that unsupervised learning is realized; in addition, a general multivariate time series data anomaly detection and inference model is designed in the scheme, and anomaly judgment can be carried out according to the generated anomaly score self-adaptively selecting a corresponding threshold value. The method is realized by the following steps:
step 1: data are preprocessed and divided, and the data are processed into sizes corresponding to the sizes of windows according to set window parameters so as to adapt to the requirements of a detection model on input data; the processed data is divided into training data and test data.
Step 2: and (5) training a model. Training the whole model according to the training data obtained in the step 1 to obtain the required parameter W*-s,b*S, φ -s and θ -s.
And step 3: starting an anomaly detection process, and carrying out N times of detection by using a random convolution neural network in a detection model by means of a group of window test data obtained in the step 1: expanding convolution, weight normalization, a rectification unit, residual error connection processing and capturing time dependence.
And 4, step 4: and (3) reconstructing the data processed in the step (3) by means of the deep Bayesian network parameters obtained by training in the step (2) to obtain a multivariate abnormal score.
And 5: and (4) calculating the abnormal degree of each observation example by adopting a score attention unit in the inference model according to the multiple abnormal scores obtained in the step (4) to obtain the abnormal score of each observation example.
Step 6: the auto-select threshold unit may use the generalized Pareto distribution with parameters to construct a threshold using the set of anomaly scores obtained in step 5.
And 7: the point adjusting means determines an abnormality based on the abnormality score obtained in step 5 and the threshold value obtained in step 6, using the characteristics of an actual abnormality.
And 8: and (5) repeating the steps 3 to 7 until all the test data are detected.
The invention reduces the scale of the model and the training cost by using the random convolution neural network, realizes unsupervised anomaly detection by using the variational encoder technology, and improves the usability and the stability of the model by adopting the score attention unit, the automatic threshold selection unit and the point adjustment unit.
The invention has the beneficial effects that: when the multi-element time series data abnormity is detected, the method based on supervised learning is always influenced by the difficulty in obtaining the label data and the changeability of the abnormity types, so that the abnormity detection accuracy is low in practical application, and the abnormity is difficult to identify; the traditional RNN-based abnormality detection method is large in scale and small in application range. The method reconstructs the abnormal detection method of the multivariate time sequence data, solves the defect of supervised learning on the premise of ensuring the accuracy of abnormal supervision, and greatly reduces the model scale, thereby effectively relieving the problem that the current abnormal detection method is difficult to be applied to resource-limited equipment, and greatly improving the practicability of abnormal detection.
Drawings
Fig. 1 is an overall system architecture of the present invention.
Fig. 2 is a structural diagram of a detection model.
Fig. 3 is an implementation of a stochastic convolutional neural network.
FIG. 4 is a block diagram of an inference model.
FIG. 5 is a graph comparing training time and model size with OmniAnomally method under three public data sets.
Fig. 6 is a graph comparing F1 metrics generated for three public data sets, four unsupervised anomaly detection techniques.
Detailed Description
The invention is further described with reference to the accompanying drawings and specific implementation steps:
a lightweight unsupervised anomaly detection method based on multivariate time series data analysis comprises the following steps:
step 1: data are preprocessed and divided, and the data are processed into sizes corresponding to the sizes of windows according to set window parameters so as to adapt to the requirements of a detection model on input data; the processed data is divided into training data and test data.
As shown in fig. 1, the overall structure of the present invention is shown. The data processing and dividing part is arranged at the entrance of the structure of the invention and is responsible for carrying out primary processing on the original data to form a data structure required by the detection model. Notably, we account for the window size as T +1, test data x at time TtDepends on xt-T:tSequences, rather than individual test values. Wherein xt-T:tRepresenting the sequence of test values from time T-T to time T.
Step 2: and (5) training a model. Training the whole model according to the training data obtained in the step 1 to obtain the required parameter W*-s,b*S, φ -s and θ -s.
Model training is an important part influencing an anomaly detection result, parameters are adjusted in an ELBO optimizing mode in detection model training, and a loss function of a single observation example can be described as follows:
Figure BDA0003026943390000041
the detection model comprises an encoder qφ(z | x) and decoder pθ(x | z) module, its parameter W*-s,b*S, φ -s and θ -s are adjusted synchronously by optimizing ELBO.
Figure BDA0003026943390000051
Represents xtAt the point of satisfying qφ(zt∣ht) Expectation under distributed conditions. Setting the window size to be T +1 according to the training data obtained in the step 1, and simulating and calculating the expectation by using a Monte Carlo methodSetting the sampling length to be L, wherein the first sample is recorded as
Figure BDA0003026943390000052
ht＝TCN(xt-T:t)，
Figure BDA0003026943390000053
TCN is a paradigm of a stochastic convolutional neural network, represented as
Figure BDA0003026943390000054
xtIs subject to a priori probability
Figure BDA0003026943390000055
First term of formula (1)
Figure BDA0003026943390000056
Is a standard multivariate Gaussian normal distribution, the second term represents the loss of Kullback-Leibler (KL), and the third term
Figure BDA0003026943390000057
Is a non-negative reconstruction error. The total loss under the entire window is recorded as:
Figure BDA0003026943390000058
where T +1 represents the window size,
Figure BDA0003026943390000059
representing the loss of a single instance of observation,
Figure BDA00030269433900000510
representing the total loss of observed data under the entire window.
And step 3: detecting each group of test data by using a random convolutional neural network in the detection model, wherein N groups of test data are total; detection includes dilation convolution, weight normalization, rectification unit, residual concatenation, and capturing time dependence.
The model structure for capturing the time dependence is shown in fig. 2, which represents the random convolutional neural network implementation process in our detection model. The dilation convolution operation is defined as:
Figure BDA00030269433900000511
wherein the input sequence x ∈ RnThe filter f: { 0., k-1} → R, d is a dilation factor, k is the convolution kernel size, and s-d · i represents the subscript to the history value. The expansion means that a fixed step is introduced between two adjacent filters, and when d is equal to 1, the expansion convolution is a regular convolution. The field of view of the convolutional network can be effectively enlarged by using a larger expansion factor.
In addition, the method of residual error connection can effectively prevent the gradient disappearance phenomenon from occurring, so that each layer can learn the change process more accurately instead of learning the whole conversion mapping. Is defined as:
Figure BDA0003026943390000061
the whole test data time series capture process can therefore be summarized as:
Figure BDA0003026943390000062
where h is the generated hidden layer variable.
And 4, step 4: by detecting model parameters W*-s,b*And (4) coding and decoding the test data processed by the random convolutional neural network to obtain each group of multivariate abnormal score.
Each set of test data comprises a plurality of test cases, the dimension of each abnormality score corresponds to the dimension of each test case, and the value of each abnormality score represents the abnormality degree of the dimension of the corresponding test case.
The dimensions of the multivariate abnormality score correspond to the dimensions of the test data, and the value of each dimension represents the degree of abnormality of each dimension of the corresponding observation instance.
The step is a core of the detection model, and a practical framework of the detection model is designed by using the ideas of the VAE and the random convolutional neural network, as shown in FIG. 3.
The left part of fig. 3 shows details of the encoder module, including the random convolutional neural network layer, the VAE layer, and the planar nf layer. In this module, the hidden layer sequence h is generated by means of step 3t-T:tInput into the VAE layer to generate a mean vector muzSum variance vector σzObtaining a sequence z of potential spaces using a resampling technique0. Finally, z is laminated using a PlanarNF layer0Conversion to potential space zk. The process is summarized as follows:
Figure BDA0003026943390000071
the first expression in formula (6) represents the process of capturing the time dependency of the stochastic convolutional neural network in step 3, the second and third expressions represent the inference probability generation process in VAE, fφ(h) Representing the hidden layer by using the ReLu activation function and the mean vector muzFrom the linear layer, variance vector σzGenerated by the Soft-Plus activation function and the minor perturbation ∈. The fourth, fifth and sixth formulas show the generation process of the potential space variable z, wherein z is zK。
Figure BDA0003026943390000072
Representing obedience mean value μzVariance is σzA gaussian distribution of (a).
Figure BDA0003026943390000073
uz,Wz,bzand phi represents the model parameters trained in step 2. By means of the encoder module, the potential spatial sequence z will be obtained finallyt-T:t。
The right half of fig. 3 represents details of the decoder module, including the random convolutional neural network layer and the VAE layer. The process is the same as the encoder module, but with minor changes, the process is disclosed as:
Figure BDA0003026943390000074
wherein, the expression (7) shows that the random convolution neural network module generates the hidden layer sequence
Figure BDA0003026943390000075
The process of (1). The two-three formula in the public display (7) is similar to the two-two formula in the public display (6), and the only difference is that
Figure BDA0003026943390000076
The process of (1). Reconstruction sequence
Figure BDA0003026943390000077
From probability distribution
Figure BDA0003026943390000078
Rather than from the Planar NF layer. Same parameters
Figure BDA0003026943390000079
and theta represents the model parameters after the training in the step 2.
The detection model uses the reconstruction probability as an anomaly score of each dimension of the test case, and is defined as:
Figure BDA0003026943390000081
wherein s istRepresents xtIs scored for abnormality, and its dimension is related to xtThe same is true.
And 5: and (4) calculating the abnormal degree of each observation example by adopting a score attention unit in the inference model according to the multiple abnormal scores obtained in the step (4) to obtain the abnormal score of each observation example.
The scoring attention unit can effectively process the abnormity close to the normal example, and plays a great role in improving the accuracy of the whole model. For an M-dimensional anomaly score, the unit can be described as:
Figure BDA0003026943390000082
AStan anomaly score representing an observation instance, aiIs stThe weight of the ith dimension. The abnormality score is an index for measuring the degree of abnormality of an observation instance, and a larger abnormality score means that the observation instance is less likely to be abnormal, and conversely, the observation instance is more likely to be abnormal.
Step 6: the auto-select threshold unit may use the generalized Pareto distribution with parameters to construct a threshold using the set of anomaly scores obtained in step 5.
And 7: the point adjusting means determines an abnormality based on the abnormality score obtained in step 5 and the threshold value obtained in step 6, using the characteristics of an actual abnormality.
The abnormality judgment is the last step of the abnormality detection method, and the judgment process is as follows: and comparing the abnormal score of each observation example with a threshold, if the abnormal score is higher than the threshold, judging the observation example to be abnormal, and if not, judging the observation example to be normal.
And 8: and (5) repeating the steps 3 to 7 until all the test data are detected.
FIG. 5 shows a comparison of training time and model size for different anomaly detection techniques under the same experimental environment and three public data sets. Due to the use of a lightweight stochastic convolutional neural network in the LUAD, it was found that the LUAD was reduced by a factor of 1.5 in training time and 24 in model scale compared to the current advanced method omnianomally.
Fig. 6 shows a comparison of the performance of five unsupervised anomaly detection techniques under the three public data set conditions, which is a measure of the F1 metric. It can be found that a single approach is: AE and LSTM-VAE, the performance expression is much lower than the other three methods. Omni Anomaly uses a traditional RNN-based anomaly detection method, and the large-scale model of the method has better performance on a small-batch data set, but has poorer performance expression on a large-scale data set. USAD is based on the non-supervision anomaly detection model of GAN realization, and this model has better performance in detecting "light weight" anomaly, and USAD needs a large amount of data training in addition, and the practicality is relatively poor. The LUAD can overcome the defects of training time and model scale brought by the traditional RNN-based method, and can also express excellent performance on data sets with various sizes.

Claims (4)

1. A lightweight unsupervised anomaly detection method based on multivariate time series data analysis is characterized in that the method is realized by two models: a detection model and an inference model; the detection model extracts time dependency characteristics of the captured multivariate time sequence data through a random convolutional neural network, and then encodes and decodes the multivariate time sequence data after characteristics are extracted through a deep Bayesian network, and the detection model can determine a detection accuracy range; the inference model is composed of a score attention unit, a threshold value automatic selection unit and a point adjustment unit, wherein the score attention unit adopts an attention mechanism to enlarge the characteristic difference between abnormal data and normal data and provides a theoretical basis for abnormal interpretation, the threshold value automatic selection unit automatically calculates a threshold value by using a peak-over-threshold method, the point adjustment unit simulates the generation process of real abnormality by using a point-adjust mechanism, and the inference model can improve the accuracy, stability and interpretability of abnormal detection.
2. The lightweight unsupervised anomaly detection method based on multivariate time series data analysis as claimed in claim 1, characterized in that the method comprises the following steps:
step 1: data are preprocessed and divided, and the data are processed into sizes corresponding to the sizes of windows according to set window parameters so as to adapt to the requirements of a detection model on input data; dividing the processed data into training data and testing data;
step 2: training a model; training the detection model according to the training data obtained in the step 1 to obtain the required model parameter W*-s，b*-s, φ -s and θ -s;
model training is an important part influencing an anomaly detection result, parameters are adjusted in an ELBO optimizing mode in detection model training, and a loss function of a single observation example can be described as follows:
Figure FDA0003026943380000011
the detection model comprises an encoder module qφ(zx) and decoder module pθ(x | z), parameter W thereof*-s，b*-s, φ -s and θ -s are adjusted synchronously by optimizing ELBO;
Figure FDA0003026943380000012
represents xtAt the point of satisfying qφ(zt|ht) (iii) a desire under distribution conditions; setting the window size to be T +1 according to the training data obtained in the step 1, simulating and calculating expectation by using a Monte Carlo method, and setting the sampling length to be L, wherein the first sampling is recorded as
Figure FDA0003026943380000021
1≤l≤L；ht＝TCN(xt-T：t)，
Figure FDA0003026943380000022
TCN is a paradigm of a stochastic convolutional neural network, represented as
Figure FDA0003026943380000023
xtIs subject to a priori probability
Figure FDA0003026943380000024
Formula (II)(1) Item I of (1)
Figure FDA0003026943380000025
Is a standard multivariate Gaussian normal distribution, the second term represents the loss of Kullback-Leibler (KL), and the third term
Figure FDA0003026943380000026
Is a non-negative reconstruction error and the total loss under the whole window is recorded as:
Figure FDA0003026943380000027
where T +1 represents the window size,
Figure FDA0003026943380000028
representing the loss of a single instance of observation,
Figure FDA0003026943380000029
represents the total loss of observed data under the whole window;
and step 3: detecting each group of test data by using a random convolutional neural network in a detection model, wherein N groups of test data are total; detecting comprises expanding convolution, weight normalization, a rectifying unit, residual error connection and capturing time dependence;
and 4, step 4: by detecting model parameters W*-s，b*Coding and decoding the test data processed by the random convolutional neural network to obtain each group of multivariate abnormal score;
each group of test data comprises a plurality of test cases, the dimension of each abnormal score corresponds to the dimension of each test case, and the value of each abnormal score represents the abnormal degree of the dimension of the corresponding test case;
and 5: taking the multivariate abnormal score as the input of a score attention unit in the inference model, and calculating the abnormal degree of each test case through the score attention unit so as to obtain the abnormal score of each test case;
the score attention unit can effectively process the abnormity close to the normal example, and plays a great role in improving the precision of the whole model; for an M-dimensional anomaly score, the score attention unit describes the anomaly score as:
Figure FDA00030269433800000210
AStrepresents the current t-th test case stAn abnormality score ofiIs the current test case stThe weight of the ith dimension of (1); siIs the current test case stThe abnormality score of the ith dimension of (1); the abnormality score is an index for measuring the abnormality degree of an observation example, and the larger the abnormality score is, the smaller the possibility that the observation example is abnormal is, otherwise, the higher the possibility that the observation example is abnormal is;
step 6: the automatic threshold selection unit constructs a threshold by using generalized Pareto distribution with parameters based on each group of abnormal scores obtained in the step 5;
and 7: the point adjusting unit judges the abnormality of each group of abnormal scores obtained in the step 5 and the corresponding threshold value obtained in the step 6 by using the characteristics of the abnormality in reality;
the abnormality judgment is the last step of the abnormality detection method, and the judgment process is as follows: comparing the abnormal score of each observation example with a threshold, if the abnormal score is higher than the threshold, judging the observation example to be abnormal, otherwise, judging the observation example to be normal;
and 8: and sorting the abnormal scores of each dimension of the heavy observed examples which are judged to be abnormal, wherein the larger the abnormal score is, the larger the influence on the observed examples is.
3. The lightweight unsupervised anomaly detection method based on multivariate time series data analysis as claimed in claim 2, characterized in that the encoder module comprises a random convolutional neural network layer, a VAE layer and a Planar NF layer; hidden layer sequence h generated by means of step 3t-T：tInputting it into VAE layer, and generatingMean vector μzSum variance vector σzObtaining a sequence z of potential spaces using a resampling technique0(ii) a Finally z is laminated using a Planar NF layer0Conversion to potential space zk(ii) a The process is summarized as follows:
Figure FDA0003026943380000031
the first expression in formula (6) represents the process of capturing the time dependency of the stochastic convolutional neural network in step 3, the second and third expressions represent the inference probability generation process in VAE, fφ(h) Representing the hidden layer by using the ReLu activation function and the mean vector muzFrom the linear layer, variance vector σzGenerating by a Soft-Plus activation function and a small perturbation epsilon; the fourth, fifth and sixth formulas show the generation process of the potential space variable z, wherein z is zK；
Figure FDA0003026943380000041
Representing obedience mean value μzVariance is σz(ii) a gaussian distribution of;
Figure FDA0003026943380000042
uz，Wz，bzand phi represents the model parameters trained in the step 2; by means of the encoder module, the potential spatial sequence z will be obtained finallyt-T：t。
4. The lightweight unsupervised anomaly detection method based on multivariate time series data analysis as claimed in claim 3, characterized in that the decoder module comprises a random convolutional neural network layer and a VAE layer, the process is similar to the encoder module, and the process is disclosed as:
Figure FDA0003026943380000043
wherein, the following is shown in the same type in the public display (7)Hidden layer sequence generated by machine convolution neural network module
Figure FDA0003026943380000044
The process of (2); the two-three formula in the public display (7) is similar to the two-two formula in the public display (6), and the only difference is that
Figure FDA0003026943380000045
The process of (2); reconstruction sequence
Figure FDA0003026943380000046
From probability distribution
Figure FDA0003026943380000047
Instead of from the Planar NF layer; same parameters
Figure FDA0003026943380000048
Figure FDA0003026943380000049
and theta represents the model parameters after the training in the step 2.
CN202110418526.1A 2021-04-19 2021-04-19 Lightweight unsupervised anomaly detection method based on multivariate time series data analysis Pending CN113159163A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110418526.1A CN113159163A (en) 2021-04-19 2021-04-19 Lightweight unsupervised anomaly detection method based on multivariate time series data analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110418526.1A CN113159163A (en) 2021-04-19 2021-04-19 Lightweight unsupervised anomaly detection method based on multivariate time series data analysis

Publications (1)

Publication Number Publication Date
CN113159163A true CN113159163A (en) 2021-07-23

Family

ID=76868803

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110418526.1A Pending CN113159163A (en) 2021-04-19 2021-04-19 Lightweight unsupervised anomaly detection method based on multivariate time series data analysis

Country Status (1)

Country Link
CN (1) CN113159163A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113568774A (en) * 2021-07-27 2021-10-29 东华大学 Real-time anomaly detection method for multi-dimensional time sequence data by using unsupervised deep neural network
CN113722176A (en) * 2021-07-30 2021-11-30 银清科技有限公司 Self-adaptive abnormal performance index determining method and device
CN113780238A (en) * 2021-09-27 2021-12-10 京东科技信息技术有限公司 Multi-index time sequence signal abnormity detection method and device and electronic equipment
CN113794689A (en) * 2021-08-20 2021-12-14 浙江网安信创电子技术有限公司 Malicious domain name detection method based on TCN
CN114217136A (en) * 2022-02-22 2022-03-22 山东卓朗检测股份有限公司 Lightning protection grounding resistance detection statistical method based on big data
CN114692767A (en) * 2022-03-31 2022-07-01 中国电信股份有限公司 Abnormality detection method and apparatus, computer-readable storage medium, and electronic device
CN116232772A (en) * 2023-05-08 2023-06-06 中国人民解放军国防科技大学 Unsupervised network data intrusion detection method based on ensemble learning
CN117131452A (en) * 2023-08-29 2023-11-28 国网山东省电力公司信息通信公司 Abnormality detection method and system based on normalized flow and Bayesian network
CN117893528A (en) * 2024-03-13 2024-04-16 云南迪安医学检验所有限公司 Method and device for constructing cardiovascular and cerebrovascular disease classification model

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113568774B (en) * 2021-07-27 2024-01-16 东华大学 Multi-dimensional time sequence data real-time abnormality detection method using unsupervised deep neural network
CN113568774A (en) * 2021-07-27 2021-10-29 东华大学 Real-time anomaly detection method for multi-dimensional time sequence data by using unsupervised deep neural network
CN113722176A (en) * 2021-07-30 2021-11-30 银清科技有限公司 Self-adaptive abnormal performance index determining method and device
CN113794689A (en) * 2021-08-20 2021-12-14 浙江网安信创电子技术有限公司 Malicious domain name detection method based on TCN
CN113780238A (en) * 2021-09-27 2021-12-10 京东科技信息技术有限公司 Multi-index time sequence signal abnormity detection method and device and electronic equipment
CN113780238B (en) * 2021-09-27 2024-04-05 京东科技信息技术有限公司 Abnormality detection method and device for multi-index time sequence signal and electronic equipment
CN114217136A (en) * 2022-02-22 2022-03-22 山东卓朗检测股份有限公司 Lightning protection grounding resistance detection statistical method based on big data
CN114217136B (en) * 2022-02-22 2022-05-06 山东卓朗检测股份有限公司 Lightning protection grounding resistance detection statistical method based on big data
CN114692767A (en) * 2022-03-31 2022-07-01 中国电信股份有限公司 Abnormality detection method and apparatus, computer-readable storage medium, and electronic device
CN114692767B (en) * 2022-03-31 2024-01-19 中国电信股份有限公司 Abnormality detection method and apparatus, computer-readable storage medium, and electronic device
CN116232772B (en) * 2023-05-08 2023-07-07 中国人民解放军国防科技大学 Unsupervised network data intrusion detection method based on ensemble learning
CN116232772A (en) * 2023-05-08 2023-06-06 中国人民解放军国防科技大学 Unsupervised network data intrusion detection method based on ensemble learning
CN117131452A (en) * 2023-08-29 2023-11-28 国网山东省电力公司信息通信公司 Abnormality detection method and system based on normalized flow and Bayesian network
CN117893528A (en) * 2024-03-13 2024-04-16 云南迪安医学检验所有限公司 Method and device for constructing cardiovascular and cerebrovascular disease classification model
CN117893528B (en) * 2024-03-13 2024-05-17 云南迪安医学检验所有限公司 Method and device for constructing cardiovascular and cerebrovascular disease classification model

Similar Documents

Publication Publication Date Title
CN113159163A (en) Lightweight unsupervised anomaly detection method based on multivariate time series data analysis
Shao et al. A deep learning approach for fault diagnosis of induction motors in manufacturing
Wang et al. A novel weighted sparse representation classification strategy based on dictionary learning for rotating machinery
Ko et al. Fault classification in high-dimensional complex processes using semi-supervised deep convolutional generative models
Harrou et al. Detecting abnormal ozone measurements with a deep learning-based strategy
US8630962B2 (en) Error detection method and its system for early detection of errors in a planar or facilities
US20230094389A1 (en) Quantum computing based deep learning for detection, diagnosis and other applications
Paynabar et al. Monitoring and diagnosis of multichannel nonlinear profile variations using uncorrelated multilinear principal component analysis
CN109902399A (en) Rolling bearing fault recognition methods under a kind of variable working condition based on ATT-CNN
Zhang et al. Fault detection and recognition of multivariate process based on feature learning of one-dimensional convolutional neural network and stacked denoised autoencoder
Yang et al. Combined wireless network intrusion detection model based on deep learning
CN114492826A (en) Unsupervised anomaly detection analysis solution method based on multivariate time sequence flow data
Zhang et al. Recognition of mixture control chart pattern using multiclass support vector machine and genetic algorithm based on statistical and shape features
CN110991471B (en) Fault diagnosis method for high-speed train traction system
CN115290326A (en) Rolling bearing fault intelligent diagnosis method
CN113177577A (en) Bearing fault diagnosis method based on improved convolutional neural network
CN116796272A (en) Method for detecting multivariate time sequence abnormality based on transducer
Li et al. A framework for diagnosing the out-of-control signals in multivariate process using optimized support vector machines
Che et al. Deep meta-learning and variational autoencoder for coupling fault diagnosis of rolling bearing under variable working conditions
CN114861778A (en) Method for rapidly classifying rolling bearing states under different loads by improving width transfer learning
Wu et al. Real-time monitoring and diagnosis scheme for IoT-enabled devices using multivariate SPC techniques
Yang et al. Label propagation algorithm based on non-negative sparse representation
Wu et al. A multi-sensor fusion-based prognostic model for systems with partially observable failure modes
CN111898565B (en) Forest smoke and fire real-time monitoring system and method based on robust multi-view
Howard et al. Identifying nonlinear variation patterns with deep autoencoders

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination