CN117786374B - Multivariate time sequence anomaly detection method and system based on graph neural network - Google Patents
Multivariate time sequence anomaly detection method and system based on graph neural network Download PDFInfo
- Publication number
- CN117786374B CN117786374B CN202410221484.6A CN202410221484A CN117786374B CN 117786374 B CN117786374 B CN 117786374B CN 202410221484 A CN202410221484 A CN 202410221484A CN 117786374 B CN117786374 B CN 117786374B
- Authority
- CN
- China
- Prior art keywords
- sequence
- time
- input
- time sequence
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 93
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 38
- 238000012549 training Methods 0.000 claims abstract description 72
- 230000005856 abnormality Effects 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 48
- 230000008569 process Effects 0.000 claims abstract description 34
- 230000004927 fusion Effects 0.000 claims abstract description 14
- 238000010276 construction Methods 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 63
- 230000007246 mechanism Effects 0.000 claims description 53
- 239000010410 layer Substances 0.000 claims description 42
- 238000010606 normalization Methods 0.000 claims description 32
- 238000013527 convolutional neural network Methods 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 4
- 230000010339 dilation Effects 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 239000002356 single layer Substances 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 4
- 238000010200 validation analysis Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
Landscapes
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention discloses a multivariable time sequence anomaly detection method and a multivariable time sequence anomaly detection system based on a graph neural network, wherein the method comprises the following steps: acquiring time sequence input data to be detected, and inputting the time sequence input data into a preset time sequence abnormality detection model to judge time sequence abnormality; the construction process of the time sequence abnormality detection model comprises the following steps: training a timing anomaly detection model by using the sample set after the sample set is acquired; converting the unit sample x into a fusion time sequence which fuses the time domain and the space domain relations; Will fuse time seriesInput to encoder output intermediate features; Will fuse time seriesAnd intermediate featuresInputting the training output sequence to a decoder, calculating a training loss value based on the training output sequence, optimizing parameters of the time sequence abnormality detection model, and outputting the trained time sequence abnormality detection model; the method can analyze the multivariate time series data more comprehensively, thereby improving the accuracy and reliability of anomaly detection.
Description
Technical Field
The invention belongs to the field of data time sequence anomaly detection, and particularly relates to a multivariate time sequence anomaly detection method and system based on a graph neural network.
Background
Multivariate timing anomaly detection has wide application in many areas such as finance, industrial manufacturing, and network security, with the primary goal of identifying anomaly patterns in time series data in order to discover potential problems early and take corresponding action. Conventional anomaly detection methods are generally based on statistics or rules, but these methods often face limitations when processing complex multidimensional time series data.
In recent years, a deep learning-based method has made remarkable progress in the field of multivariate time series anomaly detection. In particular, algorithms incorporating the graph neural network and encoder-decoder architecture provide more powerful spatio-temporal modeling capabilities. The graph neural network is capable of capturing complex spatial relationships in data, while the encoder-decoder structure is capable of efficiently learning spatio-temporal representations of the data and for reconstruction and anomaly detection.
The key idea behind this emerging technology is to consider multi-dimensional time series data as a graph structure, where nodes represent different features or dimensions and edges represent spatial relationships between them. Through the graph neural network, the spatial relationship can be effectively learned, so that the accuracy and the robustness of anomaly detection are improved. In addition, the encoder-decoder architecture facilitates capturing an inherent representation of the data, making it a versatile method for use in a variety of application scenarios.
Multivariate timing anomaly detection algorithms based on graph neural networks and encoder-decoder architecture represent an advanced technological trend, but two problems remain with the prior art: firstly, the time sequence reconstruction process is difficult to consider the spatial relationship and the time domain relationship of the time sequence at the same time; secondly, it is difficult to accurately locate abnormal points of flaw data.
Disclosure of Invention
The invention provides a multivariate time sequence anomaly detection method and a multivariate time sequence anomaly detection system based on a graph neural network, which can analyze multivariate time sequence data more comprehensively, so that the accuracy and the reliability of anomaly detection are improved.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the first aspect of the invention provides a multivariate time sequence anomaly detection method based on a graph neural network, which comprises the following steps:
Acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold;
the construction process of the time sequence abnormality detection model comprises the following steps:
Training a time sequence abnormality detection model by using the sample set after the sample set is acquired, wherein the time sequence abnormality detection model comprises a time sequence reconstruction module, an encoder and a decoder;
Constructing a graph structure G (V, E) based on unit samples in the sample set; input the unit sample to time domain expansion convolution to obtain time sequence ; Time seriesSimultaneously inputting the graph structure G (V, E) and the graph structure G (V, E) into a graph annotation meaning influence network to obtain a fusion time sequence；
Will fuse time seriesInput to encoder output intermediate features; Fusion time seriesAnd intermediate featuresInputting the training output sequence to a decoder to obtain a training output sequence, calculating a training loss value based on the training output sequence, and optimizing parameters of the timing anomaly detection model according to the training loss value; repeating the training process of the iterative time sequence abnormality detection model until the loss value converges, setting a time sequence abnormality judgment threshold value and outputting the trained time sequence abnormality detection model.
Preferably, the process of constructing the graph structure G (V, E) based on the unit samples in the sample set includes:
in unit samples To input data, node information is constructedWherein node，Expressed as a learnable weight matrix,The unit attribute feature representation expressed as a time sequence, T is the sequence length, and N is the feature number;
Construction node And nodeSide information between; Wherein the connecting edge，The operation is node number removing operation;
K connecting edges with highest relevance of each node are selected according to the edge information, and a graph structure G (V, E) is constructed based on the node information V and the edge information E.
Preferably, the unit samples are input into a time domain expansion convolution to obtain a time sequenceThe process of (1) comprises:
splitting a unit sample x into N groups of one-dimensional sequences with the length of T Post-input to a time domain dilation convolution, whereThe corresponding convolution kernel is; The expansion factor is df; receptive field is; K is expressed as the number of convolution kernel layers and is used for abstracting the time domain feature representation of a deeper layer;
Finally, applying a hole convolution operation on the sequence position t, the convolution kernel length K and the characteristic channel n to obtain a time sequence Defined as。
Preferably, the time sequence isAnd graph structure G (V, E) input to graph intent impact network to obtain fused time seriesThe process of (1) comprises:
For time series Extracting subsequences/>, in the time domain dimensionThen is divided into/>, according to attribute dimension；
Calculating the weight of each connecting edgeWherein: /(I)Expressed as a learnable weight matrix,Is a single-layer full-connection layer and is based on the weight/>, of each connection edgeThe attention coefficient is calculated, and the expression formula is as follows:
；
Wherein: To activate the function,/> Representing matrix connections,Is a weight vector; expressed as target node/> Is defined by a set of contiguous nodes;
For target nodes according to attention coefficients Updating to obtain new nodeThe expression formula is:
；
Wherein, Representing a multi-layer perceptron; /(I)Expressed as a attention coefficient; obtaining new node information/>, through space-time relationship learning; Time sequence/>, after time domain learningThe processing is that the time sequence/>, after the airspace relation learning, is processed。
Preferably, the time series will be fusedInput to encoder output intermediate featuresThe process of (1) comprises:
Will fuse time series Inputting to an encoder, wherein the encoder sequentially comprises a multi-branch attention mechanism, a layer normalization B1, a feedforward neural network and a layer normalization B2;
Will fuse time series Inputting the multiple branches of attention mechanism to obtain a feature sequence output; inputting the characteristic sequence output to the layer normalization B1 to obtain a time sequenceThe expression formula is:
；
Time series Input to a feedforward neural network to obtain a time sequenceThe expression formula is:
；
Time series Input to layer normalization B2 to obtain intermediate featuresThe expression formula is:
；
In the formula (i), Represented as a normalization function; /(I)Represented as a feed-forward neural network.
Preferably, the time series will be fusedThe process of obtaining the feature sequence output by inputting the multi-branch attention mechanism comprises the following steps:
The multi-branch attention mechanism comprises Vaswani self-attention mechanism, dense comprehensive attention mechanism and dynamic convolution neural network;
Will fuse time series Input to Vaswani self-attention mechanism to obtain feature sequenceThe expression formula is:
；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence;
Will fuse time series Inputting the characteristic sequences into a dense comprehensive attention mechanism to obtain characteristic sequences output 2, wherein the expression formula is as follows:
；
；
In the formula, W 1 is expressed as a learnable weight matrix; w 2 is denoted as a learnable weight matrix; b 1 is denoted as a learnable bias parameter; b 2 is denoted as a learnable bias parameter;
Will fuse time series Inputting the characteristic sequence into a dynamic convolutional neural network to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
Preset convolutional neural network By means of fusion time sequenceFor convolutional neural networkTraining learning attention weightsThe expression formula is:
；
Wherein AvgPool is an average pooling layer, FC is a full connection layer, reLU is a nonlinear activation function, and Softmax is a normalization function; w Avg is denoted as a learnable weight matrix;
presetting parameters of convolutional neural network ConvNet Attention weightInputting the characteristic sequence into a convolutional neural network ConvNet to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
；
；
Carrying out weighted summation on the characteristic sequence output 1, the characteristic sequence output 2 and the characteristic sequence output 3 to obtain a characteristic sequence output; the expression formula is:
；
Wherein, Is a learnable weight.
Preferably, the time series will be fusedAnd intermediate featuresThe process of obtaining the training output sequence by inputting to the decoder comprises the following steps:
Will fuse time series The self-attention mechanism input to the decoder gets the intermediate featureThe expression formula is: /(I)；
In the formula (i),Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence; t is the sequence length;
Will be intermediate features Layer normalization D1 input to the decoder obtains intermediate featuresThe expression formula is:
；
Will be intermediate features And intermediate featuresThe cross-attention mechanism input to the decoder gets the intermediate featureThe expression formula is:
；
Will be intermediate features Layer normalization D2 input to the decoder obtains intermediate featuresThe expression formula is:
；
Will be intermediate features The feedforward neural network FFN input to the decoder obtains a training output sequence, the expression formula is:
；
In the formula (i), Expressed as training output sequence,Represented as a normalization function; /(I)Represented as a feed-forward neural network.
Preferably, the process of calculating the training loss value based on the training output sequence includes:
；
In the formula (i), Represented as reconstructed signal in training output sequence,Represented as an input signal in a unit sample.
Preferably, the process of calculating the timing anomaly score based on the detection output sequence and the timing input data and outputting the judgment result by comparing the timing anomaly score with a preset timing anomaly judgment threshold value includes:
；
；
In the formula (i), Expressed as a timing anomaly score; /(I)Represented as time series input data; /(I)Expressed as detection output sequence data,Expressed as a judgment result; anomal denotes a data exception; nomal indicates that the data is normal.
The second aspect of the present invention provides a multivariate timing anomaly detection system based on a graph neural network, comprising:
The detection unit is used for acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold;
The acquisition unit is used for training a time sequence abnormality detection model by using the sample set after acquiring the sample set, wherein the time sequence abnormality detection model comprises a time sequence reconstruction module, an encoder and a decoder; constructing a graph structure G (V, E) based on unit samples in the sample set; input the unit sample to time domain expansion convolution to obtain time sequence ; Time seriesSimultaneously inputting the graph structure G (V, E) and the graph structure G (V, E) into a graph annotation meaning influence network to obtain a fusion time sequence；
Training unit for fusing time sequencesInput to encoder output intermediate features; Fusion time seriesAnd intermediate featuresInputting the training output sequence to a decoder to obtain a training output sequence, calculating a training loss value based on the training output sequence, and optimizing parameters of the timing anomaly detection model according to the training loss value; repeating the training process of the iterative time sequence abnormality detection model until the loss value converges, setting a time sequence abnormality judgment threshold value and outputting the trained time sequence abnormality detection model.
Compared with the prior art, the invention has the beneficial effects that:
The invention processes multidimensional data with time and space relation based on a graph attention influence network and a time domain expansion convolution network, reconstructs a time sequence by adopting a coder decoder structure, and can effectively judge an abnormal state by comparing the difference between an input sequence and a reconstruction sequence; the method can analyze the multivariate time series data more comprehensively, thereby improving the accuracy and reliability of anomaly detection.
Drawings
Fig. 1 is a flowchart of a multivariate timing anomaly detection method based on a neural network of fig. 1;
fig. 2 is a model diagram of a multivariate timing anomaly detection method based on a neural network of fig. 1;
fig. 3 is a block diagram for learning a spatiotemporal relationship provided in embodiment 1.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
Example 1
As shown in fig. 1 to 3, the present embodiment provides a method for detecting multivariate timing anomaly based on a graph neural network, including:
Acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold value, wherein the process comprises the following steps:
；
；
In the formula (i), Expressed as a timing anomaly score; /(I)Represented as time series input data; /(I)Expressed as detection output sequence data,Expressed as a judgment result; anomal denotes a data exception; nomal indicates that the data is normal.
The construction process of the time sequence abnormality detection model comprises the following steps:
The acquired sample set the sample set is divided into a "training set", "validation set" and "test set". Wherein, the training set comprises flawless data and unlabeled exemplars for training a predictive model; the verification set is a labeled sample, which contains flaw data and flaw-free data and is used for searching a reasonable threshold value and debugging an anomaly evaluator; the "test set" is a labeled sample containing flaw data and no flaw data for testing the generalization performance of the predictive model and anomaly assessors.
Furthermore, the unit samples in the training set are defined asWherein, the method comprises the steps of, wherein,Time series data expressed as unit sample,N is a feature number, T is a sequence length, and T is a current sample starting point; labeled unit samples in validation and test sets are defined as，Expressed as sample tag,。
Training a timing anomaly detection model using a set of samples, the timing anomaly detection model comprising an encoder and a decoder;
the process of constructing the graph structure G (V, E) based on the unit samples in the sample set includes:
in unit samples To input data, node information is constructedWherein node，Expressed as a learnable weight matrix,A unit attribute feature representation expressed as a time series,
Construction nodeAnd nodeSide information between; Wherein the connecting edge，The operation is node number removing operation;
K connecting edges with highest relevance of each node are selected according to the edge information, and a graph structure G (V, E) is constructed based on the node information V and the edge information E.
Input the unit sample to time domain expansion convolution to obtain time sequenceThe process of (1) comprises:
splitting a unit sample x into N groups of one-dimensional sequences with the length of T Post-input to a time domain dilation convolution, whereThe corresponding convolution kernel is; The expansion factor is df; receptive field is; K is expressed as the number of convolution kernel layers and is used for abstracting the time domain feature representation of a deeper layer;
Finally, applying a hole convolution operation on the sequence position t, the convolution kernel length K and the characteristic channel n to obtain a time sequence Defined as。
Time seriesAnd graph structure G (V, E) input to graph intent impact network to obtain fused time seriesThe process of (1) comprises:
For time series Extracting subsequences/>, in the time domain dimensionThen is divided into/>, according to attribute dimension；
Calculating the weight of each connecting edgeWherein: /(I)Expressed as a learnable weight matrix,Is a single-layer full-connection layer and is based on the weight/>, of each connection edgeThe attention coefficient is calculated, and the expression formula is as follows:
；
Wherein: To activate the function,/> Representing matrix connections,Is a weight vector; expressed as target node/> Is defined by a set of contiguous nodes;
For target nodes according to attention coefficients Updating to obtain new nodeThe expression formula is:
；
Wherein, Representing a multi-layer perceptron; /(I)Expressed as a attention coefficient; obtaining new node information/>, through space-time relationship learning; Time sequence/>, after time domain learningThe processing is that the time sequence/>, after the airspace relation learning, is processed; According to the graph attention influence network, attention coefficients between the target node and the source node are flexibly distributed by calculating edge weights, and the target node is updated based on the screened adjacent nodes and the attention coefficients.
Will fuse time seriesInput to encoder output intermediate featuresThe process of (1) comprises:
Will fuse time series Inputting to an encoder, wherein the encoder sequentially comprises a multi-branch attention mechanism, a layer normalization B1, a feedforward neural network and a layer normalization B2;
Will fuse time series The process of obtaining the feature sequence output by inputting the multi-branch attention mechanism comprises the following steps:
The multi-branch attention mechanism comprises Vaswani self-attention mechanism, dense comprehensive attention mechanism and dynamic convolution neural network;
Will fuse time series Input to Vaswani self-attention mechanism to obtain feature sequenceThe expression formula is:
；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence;
Will fuse time series Inputting the characteristic sequences into a dense comprehensive attention mechanism to obtain characteristic sequences output 2, wherein the expression formula is as follows:
；
；
In the formula, W 1 is expressed as a learnable weight matrix; w 2 is denoted as a learnable weight matrix; b 1 is denoted as a learnable bias parameter; b 2 is denoted as a learnable bias parameter;
Will fuse time series Inputting the characteristic sequence into a dynamic convolutional neural network to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
Preset convolutional neural network By means of fusion time sequenceFor convolutional neural networkTraining learning attention weightsThe expression formula is:
；
Wherein AvgPool is an average pooling layer, FC is a full connection layer, reLU is a nonlinear activation function, and Softmax is a normalization function; w Avg is denoted as a learnable weight matrix;
presetting parameters of convolutional neural network ConvNet Attention weightInputting the characteristic sequence into a convolutional neural network ConvNet to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
；
；
Carrying out weighted summation on the characteristic sequence output 1, the characteristic sequence output 2 and the characteristic sequence output 3 to obtain a characteristic sequence output; the expression formula is:
; wherein/> Is a learnable weight.
The adoption of the multi-branch attention mechanism integrates a 'Vaswani attention mechanism', 'dynamic convolution network' and a 'dense comprehensive attention', and the three attention mechanisms are combined into a whole through a learnable weight, so that each variety of attention mechanism exerts respective advantages.
Inputting the characteristic sequence output to the layer normalization B1 to obtain a time sequenceThe expression formula is:
；
Time series Input to a feedforward neural network to obtain a time sequenceThe expression formula is:
；
Time series Input to layer normalization B2 to obtain intermediate featuresThe expression formula is:
；
In the formula (i), Represented as a normalization function; /(I)Represented as a feed-forward neural network.
Will fuse time seriesAnd intermediate featuresThe process of obtaining the training output sequence by inputting to the decoder comprises the following steps:
Will fuse time series The self-attention mechanism input to the decoder gets the intermediate featureThe expression formula is:
；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence;
Will be intermediate features Layer normalization D1 input to the decoder obtains intermediate featuresThe expression formula is: /(I)
；
Will be intermediate featuresAnd intermediate featuresThe cross-attention mechanism input to the decoder gets the intermediate featureThe expression formula is:
；
Will be intermediate features Layer normalization D2 input to the decoder obtains intermediate featuresThe expression formula is:
；
Will be intermediate features The feedforward neural network FFN input to the decoder obtains a training output sequence, the expression formula is:
；
In the formula (i), Represented as a training output sequence.
Calculating a training loss value based on the training output sequence, wherein the expression formula is as follows:
；
In the formula (i), Represented as reconstructed signal in training output sequence,Represented as an input signal in a unit sample.
Optimizing parameters of the timing anomaly detection model according to the training loss value; and repeating the training process of the iterative time sequence anomaly detection model until the loss value converges.
Adjusting the timing anomaly determination threshold based on the verification set and the grid search algorithm, in order to adjust the timing anomaly determination threshold for each feature dimension (N total), first, calculating standard deviations for the features of N different dimensions, and calculating a standard deviation calculation formula for the features of the nth dimension:
；
Wherein, ，Expressed as a number of unit samples in the validation set; /(I)Expressed as unit samples in a validation set
N standard deviations sharing variable parametersThe two are multiplied to be used as time sequence abnormality judgment thresholds of N characteristic dimensions, and the expression formula is as follows:
；
The embodiment processes multidimensional data with a time-space relationship based on the graph attention impact network and the time domain expansion convolution network, reconstructs a time sequence by adopting a coder decoder structure, can effectively judge an abnormal state by comparing the difference between an input sequence and a reconstruction sequence, and can more comprehensively analyze multivariable time sequence data, thereby improving the accuracy and the reliability of abnormality detection.
Example 2
The present embodiment provides a multivariate time series anomaly detection system based on a graph neural network, which can be applied to the multivariate time series anomaly detection method described in embodiment 1, the multivariate time series anomaly detection system comprising:
The detection unit is used for acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold;
The acquisition unit is used for training a time sequence abnormality detection model by using the sample set after acquiring the sample set, wherein the time sequence abnormality detection model comprises a time sequence reconstruction module, an encoder and a decoder; constructing a graph structure G (V, E) based on unit samples in the sample set; input the unit sample to time domain expansion convolution to obtain time sequence ; Time seriesSimultaneously inputting the graph structure G (V, E) and the graph structure G (V, E) into a graph annotation meaning influence network to obtain a fusion time sequence；
Training unit for fusing time sequencesInput to encoder output intermediate features; Fusion time seriesAnd intermediate featuresInputting the training output sequence to a decoder to obtain a training output sequence, calculating a training loss value based on the training output sequence, and optimizing parameters of the timing anomaly detection model according to the training loss value; repeating the training process of the iterative time sequence abnormality detection model until the loss value converges, setting a time sequence abnormality judgment threshold value and outputting the trained time sequence abnormality detection model.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.
Claims (8)
1. The multivariable time sequence anomaly detection method based on the graph neural network is characterized by comprising the following steps of:
Acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold;
the construction process of the time sequence abnormality detection model comprises the following steps:
Training a time sequence abnormality detection model by using the sample set after the sample set is acquired, wherein the time sequence abnormality detection model comprises a time sequence reconstruction module, an encoder and a decoder;
Constructing a graph structure G (V, E) based on unit samples in the sample set; input the unit sample to time domain expansion convolution to obtain time sequence ; Time seriesSimultaneously inputting the graph structure G (V, E) and the graph structure G (V, E) into a graph annotation meaning influence network to obtain a fusion time sequence；
Will fuse time seriesInput to encoder output intermediate featuresThe process of (1) comprises:
Will fuse time series Inputting to an encoder, wherein the encoder sequentially comprises a multi-branch attention mechanism, a layer normalization B1, a feedforward neural network and a layer normalization B2;
Will fuse time series Inputting a multi-branch attention mechanism, wherein the multi-branch attention mechanism comprises Vaswani self-attention mechanism, dense comprehensive attention mechanism and dynamic convolution neural network;
Will fuse time series Input to Vaswani self-attention mechanism to obtain feature sequenceThe expression formula is:
；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence;
Will fuse time series Inputting the characteristic sequences into a dense comprehensive attention mechanism to obtain characteristic sequences output 2, wherein the expression formula is as follows:
；
；
In the formula, W 1 is expressed as a learnable weight matrix; w 2 is denoted as a learnable weight matrix; b 1 is denoted as a learnable bias parameter; b 2 is denoted as a learnable bias parameter;
Will fuse time series Inputting the characteristic sequence into a dynamic convolutional neural network to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
Preset convolutional neural network By means of fusion time sequenceFor convolutional neural networkTraining learning attention weightsThe expression formula is:
；
Wherein AvgPool is an average pooling layer, FC is a full connection layer, reLU is a nonlinear activation function, and Softmax is a normalization function; w Avg is denoted as a learnable weight matrix;
presetting parameters of convolutional neural network ConvNet Attention weightInputting the characteristic sequence into a convolutional neural network ConvNet to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
；
；
Carrying out weighted summation on the characteristic sequence output 1, the characteristic sequence output 2 and the characteristic sequence output 3 to obtain a characteristic sequence output; the expression formula is:
；
Wherein, Is a learnable weight;
inputting the characteristic sequence output to the layer normalization B1 to obtain a time sequence The expression formula is:
；
Time series Input to a feedforward neural network to obtain a time sequenceThe expression formula is:
；
Time series Input to layer normalization B2 to obtain intermediate featuresThe expression formula is:
；
In the formula (i), Represented as a normalization function; /(I)Represented as a feed-forward neural network;
Will fuse time series And intermediate featuresInputting the training output sequence to a decoder to obtain a training output sequence, calculating a training loss value based on the training output sequence, and optimizing parameters of the timing anomaly detection model according to the training loss value; repeating the training process of the iterative time sequence abnormality detection model until the loss value converges, setting a time sequence abnormality judgment threshold value and outputting the trained time sequence abnormality detection model.
2. The multivariate timing anomaly detection method of claim 1, wherein constructing the graph structure G (V, E) based on unit samples in the sample set comprises:
in unit samples To input data, node information is constructedWherein node，Expressed as a learnable weight matrix,The unit attribute feature representation expressed as a time sequence, T is the sequence length, and N is the feature number;
Construction node And nodeSide information between; Wherein the connecting edge，The operation is node number removing operation;
K connecting edges with highest relevance of each node are selected according to the edge information, and a graph structure G (V, E) is constructed based on the node information V and the edge information E.
3. The method for detecting a multivariate time series anomaly of claim 2, wherein the unit samples are input to a time domain dilation convolution to obtain a time seriesThe process of (1) comprises:
splitting a unit sample x into N groups of one-dimensional sequences with the length of T And then input to a time domain dilation convolution, wherein,The corresponding convolution kernel is; The expansion factor is df; receptive field is; K is expressed as the number of convolution kernel layers and is used for abstracting the time domain feature representation of a deeper layer;
Finally, applying a hole convolution operation on the sequence position t, the convolution kernel length K and the characteristic channel n to obtain a time sequence Defined as。
4. The method for detecting a multivariate time series anomaly of claim 3, wherein the time series isAnd graph structure G (V, E) input to graph intent impact network to obtain fused time seriesThe process of (1) comprises:
For time series Extracting subsequences/>, in the time domain dimensionThen is divided into the following parts according to attribute dimension；
Calculating the weight of each connecting edgeWherein: /(I)Expressed as a learnable weight matrix,Is a single-layer full-connection layer and is based on the weight/>, of each connection edgeThe attention coefficient is calculated, and the expression formula is as follows:
；
Wherein: To activate the function,/> Representing matrix connections,Is a weight vector; /(I)Expressed as target nodeIs defined by a set of contiguous nodes;
For target nodes according to attention coefficients Updating to obtain new nodeThe expression formula is:
；
Wherein, Representing a multi-layer perceptron; /(I)Expressed as a attention coefficient; obtaining new node information/>, through space-time relationship learning; Time sequence/>, after time domain learningThe processing is that the time sequence/>, after the airspace relation learning, is processed。
5. The method for detecting a multivariate time series anomaly of claim 1, wherein the time series are fusedAnd intermediate featuresThe process of obtaining the training output sequence by inputting to the decoder comprises the following steps:
Will fuse time series The self-attention mechanism input to the decoder gets the intermediate featureThe expression formula is:；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence; t is the sequence length;
Will be intermediate features Layer normalization D1 input to the decoder obtains intermediate featuresThe expression formula is:
；
Will be intermediate features And intermediate featuresCross-attention mechanism input to decoder to obtain intermediate featuresThe expression formula is:
；
Will be intermediate features Layer normalization D2 input to the decoder obtains intermediate featuresThe expression formula is:
；
Will be intermediate features The feedforward neural network FFN input to the decoder obtains a training output sequence, the expression formula is:
；
In the formula (i), Expressed as training output sequence,Represented as a normalization function; Represented as a feed-forward neural network.
6. The method of multivariate timing anomaly detection of claim 5, wherein calculating a training loss value based on a training output sequence comprises:
；
In the formula (i), Represented as reconstructed signal in training output sequence,Represented as an input signal in a unit sample.
7. The multivariate timing anomaly detection method of claim 1, wherein calculating a timing anomaly score based on the detection output sequence and the timing input data, outputting a determination result by comparing the timing anomaly score with a preset timing anomaly determination threshold value comprises:
；
；
In the formula (i), Expressed as a timing anomaly score; /(I)Represented as time series input data; /(I)Expressed as detection output sequence data,Expressed as a judgment result; anomal denotes a data exception; nomal indicates that the data is normal.
8. A multivariate timing anomaly detection system based on a graph neural network, comprising:
The detection unit is used for acquiring time sequence input data to be detected, inputting the time sequence input data into a preset time sequence abnormality detection model to obtain a detection output sequence, calculating a time sequence abnormality score based on the detection output sequence and the time sequence input data, and outputting a judgment result by comparing the time sequence abnormality score with a preset time sequence abnormality judgment threshold;
The acquisition unit is used for training a time sequence abnormality detection model by using the sample set after acquiring the sample set, wherein the time sequence abnormality detection model comprises a time sequence reconstruction module, an encoder and a decoder; constructing a graph structure G (V, E) based on unit samples in the sample set; input the unit sample to time domain expansion convolution to obtain time sequence ; Time seriesSimultaneously inputting the graph structure G (V, E) and the graph structure G (V, E) into a graph annotation meaning influence network to obtain a fusion time sequence；
Training unit for fusing time sequencesInput to encoder output intermediate features; Fusion time seriesAnd intermediate featuresInputting the training output sequence to a decoder to obtain a training output sequence, calculating a training loss value based on the training output sequence, and optimizing parameters of the timing anomaly detection model according to the training loss value; repeating the training process of the iterative time sequence abnormality detection model until the loss value converges, setting a time sequence abnormality judgment threshold value and outputting the trained time sequence abnormality detection model;
The training unit will fuse the time series Input to encoder output intermediate featuresThe process of (1) comprises:
Will fuse time series Inputting to an encoder, wherein the encoder sequentially comprises a multi-branch attention mechanism, a layer normalization B1, a feedforward neural network and a layer normalization B2;
Will fuse time series Inputting a multi-branch attention mechanism, wherein the multi-branch attention mechanism comprises Vaswani self-attention mechanism, dense comprehensive attention mechanism and dynamic convolution neural network;
Will fuse time series Input to Vaswani self-attention mechanism to obtain feature sequenceThe expression formula is:
；
In the formula (i), Is VASWANI SELF Attention mechanism; w Q is denoted as a weight matrix for linearly mapping the input sequence to the query matrix; w K is denoted as a weight matrix for linear mapping of the input sequence to the key matrix; w V is denoted as a weight matrix for linear mapping of the input sequence to a value matrix; d att is the feature dimension of the attention mechanism output sequence;
Will fuse time series Inputting the characteristic sequences into a dense comprehensive attention mechanism to obtain characteristic sequences output 2, wherein the expression formula is as follows:
；
；
In the formula, W 1 is expressed as a learnable weight matrix; w 2 is denoted as a learnable weight matrix; b 1 is denoted as a learnable bias parameter; b 2 is denoted as a learnable bias parameter;
Will fuse time series Inputting the characteristic sequence into a dynamic convolutional neural network to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
Preset convolutional neural network By means of fusion time sequenceFor convolutional neural networkTraining learning attention weightsThe expression formula is:
；
Wherein AvgPool is an average pooling layer, FC is a full connection layer, reLU is a nonlinear activation function, and Softmax is a normalization function; w Avg is denoted as a learnable weight matrix;
presetting parameters of convolutional neural network ConvNet Attention weightInputting the characteristic sequence into a convolutional neural network ConvNet to obtain a characteristic sequence output 3, wherein the expression formula is as follows:
；
；
Carrying out weighted summation on the characteristic sequence output 1, the characteristic sequence output 2 and the characteristic sequence output 3 to obtain a characteristic sequence output; the expression formula is:
；
Wherein, Is a learnable weight;
inputting the characteristic sequence output to the layer normalization B1 to obtain a time sequence The expression formula is:
；
Time series Input to a feedforward neural network to obtain a time sequenceThe expression formula is:
；
Time series Input to layer normalization B2 to obtain intermediate featuresThe expression formula is:
；
In the formula (i), Represented as a normalization function; /(I)Represented as a feed-forward neural network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410221484.6A CN117786374B (en) | 2024-02-28 | 2024-02-28 | Multivariate time sequence anomaly detection method and system based on graph neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410221484.6A CN117786374B (en) | 2024-02-28 | 2024-02-28 | Multivariate time sequence anomaly detection method and system based on graph neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117786374A CN117786374A (en) | 2024-03-29 |
| CN117786374B true CN117786374B (en) | 2024-05-14 |
Family
ID=90385777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410221484.6A Active CN117786374B (en) | 2024-02-28 | 2024-02-28 | Multivariate time sequence anomaly detection method and system based on graph neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117786374B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118428420A (en) * | 2024-05-15 | 2024-08-02 | 太原理工大学 | ESAttention-based long-term time sequence prediction method for front-back symmetrical decomposition |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114065862A (en) * | 2021-11-18 | 2022-02-18 | 南京航空航天大学 | Multi-dimensional time sequence data anomaly detection method and system |
| CN115587335A (en) * | 2022-09-15 | 2023-01-10 | 中国电力科学研究院有限公司 | Training method of abnormal value detection model, abnormal value detection method and system |
| CN116776269A (en) * | 2023-04-23 | 2023-09-19 | 成都信息工程大学 | Traffic anomaly detection method based on graph convolution neural network self-encoder |
| CN116796272A (en) * | 2023-06-21 | 2023-09-22 | 复旦大学 | Method for detecting multivariate time sequence abnormality based on transducer |
| CN117076936A (en) * | 2023-10-16 | 2023-11-17 | 北京理工大学 | Time sequence data anomaly detection method based on multi-head attention model |
| CN117290800A (en) * | 2023-11-24 | 2023-12-26 | 华东交通大学 | Timing sequence anomaly detection method and system based on hypergraph attention network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210406603A1 (en) * | 2020-06-26 | 2021-12-30 | Tata Consultancy Services Limited | Neural networks for handling variable-dimensional time series data |
| CN114626512B (en) * | 2022-05-17 | 2022-09-06 | 南京信息工程大学 | High-temperature disaster forecasting method based on directed graph neural network |
-
2024
- 2024-02-28 CN CN202410221484.6A patent/CN117786374B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114065862A (en) * | 2021-11-18 | 2022-02-18 | 南京航空航天大学 | Multi-dimensional time sequence data anomaly detection method and system |
| CN115587335A (en) * | 2022-09-15 | 2023-01-10 | 中国电力科学研究院有限公司 | Training method of abnormal value detection model, abnormal value detection method and system |
| CN116776269A (en) * | 2023-04-23 | 2023-09-19 | 成都信息工程大学 | Traffic anomaly detection method based on graph convolution neural network self-encoder |
| CN116796272A (en) * | 2023-06-21 | 2023-09-22 | 复旦大学 | Method for detecting multivariate time sequence abnormality based on transducer |
| CN117076936A (en) * | 2023-10-16 | 2023-11-17 | 北京理工大学 | Time sequence data anomaly detection method based on multi-head attention model |
| CN117290800A (en) * | 2023-11-24 | 2023-12-26 | 华东交通大学 | Timing sequence anomaly detection method and system based on hypergraph attention network |
Non-Patent Citations (3)
| Title |
|---|
| Multivariate time-series anomaly detection via temporal convolutional and graph attention networks;He Qiang等;《Journal of Intelligent & Fuzzy Systems》;20230403;第44卷(第4期);5953-5962 * |
| 基于启发式时空图神经网络的多变量时序异常检测;姜羽等;《中国科学:信息科学》;20230919;第53卷(第9期);1784-1801 * |
| 基于深度学习的时间序列异常检测方法研究;郑育靖;《中国优秀硕士学位论文全文数据库 (基础科学辑)》;20220115(第1期);A002-453 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117786374A (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hsu et al. | Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing | |
| Chien et al. | Analysing semiconductor manufacturing big data for root cause detection of excursion for yield enhancement | |
| CN117786374B (en) | Multivariate time sequence anomaly detection method and system based on graph neural network | |
| CN109991951B (en) | Multi-source fault detection and diagnosis method and device | |
| JP2020009411A (en) | Sparse neural network-based abnormality detection in multidimensional time series | |
| JP7068246B2 (en) | Abnormality judgment device and abnormality judgment method | |
| CN105653427A (en) | Log monitoring method based on abnormal behavior detection | |
| Wei et al. | A novel deep learning model based on target transformer for fault diagnosis of chemical process | |
| WO2008079695A1 (en) | Monitoring and fault detection in dynamic systems | |
| US20160255109A1 (en) | Detection method and apparatus | |
| CN112257263B (en) | Equipment residual life prediction system based on self-attention mechanism | |
| CN113919540B (en) | Method for monitoring operation state of production process and related equipment | |
| Chadha et al. | Time series based fault detection in industrial processes using convolutional neural networks | |
| CN112231980A (en) | Engine life prediction method, storage medium and computing device | |
| Zhang et al. | Selecting optimal features for cross-fleet analysis and fault diagnosis of industrial gas turbines | |
| CN116976441A (en) | Equipment failure prediction model training method, equipment failure prediction method and equipment failure prediction device | |
| KR20210126378A (en) | Real-time sliding window based anomaly detection system for multivariate data generated by manufacturing equipment | |
| KR101985850B1 (en) | Detection apparatus for detecting anomaly log and operating method of same, and training apparatus and operating method of same | |
| Qin et al. | Root cause analysis of industrial faults based on binary extreme gradient boosting and temporal causal discovery network | |
| He et al. | Graph Structure Learning-Based Multivariate Time Series Anomaly Detection in Internet of Things for Human-Centric Consumer Applications | |
| CN117785522A (en) | Method and system for performing root cause analysis using a trained machine learning model | |
| Aremu et al. | Kullback-leibler divergence constructed health indicator for data-driven predictive maintenance of multi-sensor systems | |
| CN116842323A (en) | Abnormal detection method for operation data of water supply pipeline | |
| Cao et al. | Fault detection of complicated processes based on an enhanced transformer network with graph attention mechanism | |
| JP2024506994A (en) | Method and system for industrial change point detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |