CN117009900A - Internet of things signal anomaly detection method and system based on graph neural network - Google Patents

Internet of things signal anomaly detection method and system based on graph neural network Download PDF

Info

Publication number
CN117009900A
CN117009900A CN202310759397.1A CN202310759397A CN117009900A CN 117009900 A CN117009900 A CN 117009900A CN 202310759397 A CN202310759397 A CN 202310759397A CN 117009900 A CN117009900 A CN 117009900A
Authority
CN
China
Prior art keywords
graph
internet
time
graph structure
anomaly
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310759397.1A
Other languages
Chinese (zh)
Inventor
李国兵
谭一峰
陈宇轩
张国梅
屈诗涵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN202310759397.1A priority Critical patent/CN117009900A/en
Publication of CN117009900A publication Critical patent/CN117009900A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses an anomaly detection method and system for an Internet of things signal based on a graph neural network, and belongs to the field of anomaly detection of the Internet of things signal. The invention discloses a static diagram structure based on the bottommost layer of a smoothness constraint learning system and a dynamic diagram structure based on sensor characteristic embedding learning and conforming to time-varying characteristics of a time sequence, and adopts a GAGRU module which blends a diagram attention mechanism into a gating circulation unit to capture space-time characteristics of a multi-element time sequence. Meanwhile, calculating system anomaly scores according to the sequence prediction errors, carrying out anomaly detection by adopting a traditional anomaly identification method based on the TopK criterion and a point adjustment anomaly identification method based on the POT algorithm, and finally jointly verifying model performance by adopting a traditional F1 score and a point adjustment F1 score. The signal anomaly detection model of the Internet of things provided by the invention has good improvement on the comprehensive performance of anomaly detection.

Description

Internet of things signal anomaly detection method and system based on graph neural network
Technical Field
The invention belongs to the field of anomaly detection of Internet of things signals, and particularly relates to an Internet of things signal anomaly detection method and system based on a graph neural network.
Background
With the wide application and rapid growth of internet of things devices, the interconnection and communication between devices generates a large amount of data. In this case, accurately and timely identifying and handling the abnormal signal becomes an urgent problem. The effective anomaly detection technology can early warn potential risks such as equipment faults, network attacks, data leakage and the like, so that normal operation of the Internet of things system and safety privacy of users are guaranteed.
The signals of the internet of things are generally divided into two univariate time sequences and a multivariate time sequence, wherein the multivariate time sequence has increased difficulty in anomaly detection due to the complex space-time characteristics of the multivariate time sequence, so that research on the multivariate time sequence is always a hot spot. The traditional anomaly detection framework comprises two stages of estimation and detection, wherein the estimation stage predicts a sequence value in a certain time stamp or time period mainly by modeling a time sequence; the detection stage is to determine an abnormal threshold according to a criterion or a model, and compare the threshold with a predicted value so as to judge the abnormality.
Currently, more and more research is beginning to exploit the correlation between multivariate variables to improve the performance of anomaly detection. In industrial systems, anomaly detection based on a machine learning method for multivariate time series has been widely used. Such as clustering, support vector machines, bayesian networks, random forests, gradient-lifting decision trees, markov models, principal component analysis, and the like. In addition, as people get deep into the multivariate time series, many methods of deep learning are increasingly being applied, taking into account the inherent links of the data. For example, a depth self-coding Gaussian mixture model for unsupervised anomaly detection is utilized to generate low-dimensional representation and reconstruction errors by using a depth self-coder, and the anomaly detection is further carried out by using the Gaussian mixture model; a variable self-encoder based on long-term memory performs anomaly scoring by embedding LSTM networks into VAEs. A non-parametric dynamic error threshold strategy that uses a moving average of the error sequence to set the threshold of anomaly markers; an OmniAnomaly anomaly detection framework that models time-series signals as random representations using VAEs and uses reconstruction possibilities for anomaly detection; an unsupervised multivariate anomaly detection method based on generation countermeasure network, wherein a generator and a discriminator in GAN both adopt LSTM-RNN structure, and can capture time correlation of time sequence distribution; also, to avoid potential misleading of uncertain examples, there have been studies on automatic threshold selection using a stream data anomaly detection algorithm (DSPOT) with an adjusted drift profile. And this method of threshold selection is also applied by several later studies. The method for deep learning greatly improves the abnormality detection performance of the multi-element time series. However, the methods of generating the countermeasure network based on encoders, recurrent neural networks, still have some drawbacks in extracting the spatial characteristics of the multivariate time series, nor do they explain the inherent links between these multivariate variables well.
The development of graph signal processing technology and the application of graph neural network provide a new idea for processing the multi-element time series with internal connection. The latest multi-element time sequence abnormality detection framework based on the graph attention network adopts two parallel GAT networks to respectively extract the time characteristics and the space characteristics of the multi-element time sequence for abnormality detection; a graph deviation network utilizes nodes to embed a graph structure between learning data modes, and meanwhile, a GAT network is improved, so that the network can capture richer information; a multi-mode graph annotation force anomaly detection model classifies all sensor nodes, learns and extracts features by adopting a plurality of GAT networks, and performs anomaly detection by using reconstruction errors and prediction errors. The methods based on the graph neural network can effectively extract the internal spatial characteristics of the multi-element time sequence, and have great advantages for processing data with a non-European structure.
While the anomaly detection model is developed, new standards for the evaluation of anomaly detection performance are also available, and corresponding evaluation principles are designed for different application backgrounds. Recent studies have demonstrated that a strategy employing Point-Adjust (PA) can significantly improve the performance of anomaly detection, but can also result in excessive overestimation of the performance of anomaly detection. At the same time, new evaluation criteria for the performance of multivariate time series anomaly detection have been proposed in a few studies, and the conventional evaluation criteria were compared on a classical data set. These recent studies also suggest that different evaluation criteria may lead to different results in evaluating the performance of multivariate time series anomaly detection. In evaluating the abnormality detection performance of the model, it is necessary to evaluate from a plurality of angles.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a method and a system for detecting the abnormality of the Internet of things signal based on a graph neural network, which are used for solving the problem that the prior art is insufficient in consideration of space-time characteristics of a multi-element time sequence, so that the abnormality detection performance of the multi-element time sequence is improved.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme:
a method for detecting signal abnormality of the Internet of things based on a graph neural network comprises the following steps: the method comprises the following steps:
based on a system entity, a graph structure model is established, each sensor in the system entity is used as a graph node in the graph structure model, each sensor generates a time sequence to obtain an actual multi-element time sequence, and a static graph structure and a dynamic graph structure are obtained after the graph structure model is learned;
respectively defining corresponding graph meaning networks based on a static graph structure and a dynamic graph structure; embedding a graph attention mechanism into a gating circulation unit to form a GAGRU network module; processing by an uplink GAGRU network module to obtain sequence space-time characteristics based on dynamic graph modeling; processing by a downlink GAGRU network module to obtain sequence space-time characteristics based on static diagram modeling; the full connection layer predicts based on two space-time characteristics to obtain a predicted multi-element time sequence;
comparing the predicted multi-element time sequence with the actual multi-element time sequence to obtain an anomaly score; based on the anomaly score, the anomaly detection is carried out by combining the traditional F1 score evaluation standard and the F1 score evaluation standard under the point adjustment strategy.
The invention further improves that:
preferably, the process of learning the graph structure model to obtain the dynamic graph structure is as follows:
wherein e i Feature embedding representing the ith sensor; sim (·) represents solving for cosine similarity; w (w) ji The cosine similarity between the node j and the node i is represented, and the weight of the edge in the graph structure model is also represented; topK (·) represents selecting K points with the greatest similarity among all neighbors of the node i as the neighbors of the node i; k is a super parameter; a is that ji Is a dynamic graph A d The data value of the j-th row and i-th column of the table represents the connection relation between the node j and the node i;
based on all A ji Obtaining dynamic diagram A ji 。
Preferably, the process of learning the graph structure model to obtain the static graph structure is to establish an optimization problem, solve the optimization problem and obtain an adjacency matrix, wherein the adjacency matrix is the static graph structure.
Preferably, the optimization problem is:
wherein D is the preprocessed multi-element time series data; l is a graph Laplace matrix; n is the number of sensor nodes;tr (·) represents the trace of the matrix;representing the F-norm of the matrix; alpha and beta are two regularization parameters;
the adjacency matrix is:
wherein A is ij Indicating whether there is a connection between node i and node j.
Preferably, the formula of the graph annotation network is:
wherein,representing the input characteristics of the ith node at the time of the time stamp t; />Representing a trainable weight matrix; reLU represents an activation function; alpha ij Is->And->Attention coefficient, alpha ij The calculation formula of (2) is as follows:
wherein, I represents matrix concatenation;as a mechanism of attentionLearning vectors; leakyReLU is the activation function.
Preferably, the processing procedure of the uplink GAGRU network module is as follows:
wherein, I represents matrix concatenation; the omicron represents a dot product; the product of Hadamard; x is X (t) Representing the preprocessed input signal; e represents feature embedding of the sensor;respectively representing a reset gate, an update gate and a hidden state;is a corresponding dynamic GAT network; />The output of the GRU at the time t is represented; w and b represent a learnable weight matrix and a network bias vector; sigma and tanh are activation functions, where sigma is a selectable arbitrary activation function.
Preferably, the processing procedure of the downlink GAGRU network module is as follows:
wherein,respectively representing a reset gate, an update gate and a hidden state; />Is a corresponding static GAT network; />The output of the GRU at the time t is represented; w and b represent a learnable weight matrix and a network bias vector.
Preferably, the calculation formula of the evaluation standard of the traditional F1 score method is as follows:
wherein o is t =1 indicates a system abnormality; o (o) t The expression =0 indicates that the system is normal,representing an anomaly score.
Preferably, the calculation formula of the F1 score evaluation criterion under the adjustment strategy is as follows:
wherein GT= { GT 1 ,GT 2 ,…,GT M M abnormal segments in the data set, and and->A start time point and an end time point of the mth segment abnormality.
An internet of things signal anomaly detection system based on a graph neural network, comprising:
the model building unit is used for building a graph structure model based on a system entity, wherein each sensor in the system entity is used as a graph node in the graph structure model, each sensor generates a time sequence to obtain an actual multi-element time sequence, and a static graph structure and a dynamic graph structure are obtained after the graph structure model is learned;
the double-layer diagram attention gate control circulation unit is used for respectively defining corresponding diagram meaning networks based on a static diagram structure and a dynamic diagram structure; embedding a graph attention mechanism into a gating circulation unit to form a GAGRU network module; processing by an uplink GAGRU network module to obtain sequence space-time characteristics based on dynamic graph modeling; processing by a downlink GAGRU network module to obtain sequence space-time characteristics based on static diagram modeling; the full connection layer predicts based on two space-time characteristics to obtain a predicted multi-element time sequence;
the prediction unit is used for comparing the predicted multi-element time sequence with the actual multi-element time sequence to obtain an anomaly score; based on the anomaly score, the anomaly detection is carried out by combining the traditional F1 score evaluation standard and the F1 score evaluation standard under the point adjustment strategy.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a method and a system for detecting abnormal signals of the Internet of things based on a graph neural network, which are used for solving the problem of abnormal detection of a multi-element time sequence of the Internet of things. For the multi-element time sequence signals collected by the Internet of things, the multi-element time sequence signals not only contain complex time features, but also contain complex space features, and effective modeling of the space-time features is a great difficulty in an abnormality detection task. The invention discloses a static diagram structure based on the bottommost layer of a smoothness constraint learning system and a dynamic diagram structure based on sensor characteristic embedding learning and conforming to time-varying characteristics of a time sequence, and adopts a GAGRU module which blends a diagram attention mechanism into a gating circulation unit to capture space-time characteristics of a multi-element time sequence. Meanwhile, calculating system anomaly scores according to the sequence prediction errors, carrying out anomaly detection by adopting a traditional anomaly identification method based on the TopK criterion and a point adjustment anomaly identification method based on the POT algorithm, and finally jointly verifying model performance by adopting a traditional F1 score and a point adjustment F1 score.
Through simulation results on five disclosed real data sets, the signal anomaly detection model of the Internet of things provided by the invention has good improvement on the comprehensive performance of anomaly detection, and shows the potential of a double-layer diagram attention gate control circulation unit (M-GAGRU) based on diagram structure optimization in space-time characteristic modeling and anomaly detection tasks of a multi-element time sequence. Compared with the previous research in the field of anomaly detection, the method and the device have the advantages that other networks with higher complexity are not adopted, the GRU network is improved, the deeper graph structure information of the captured data is emphasized, the internal structure of the system is represented by the learning of the graph structure, the time characteristics and the space characteristics of the multi-element time sequence are extracted simultaneously by using the GAGRU module, and the anomaly points and the anomaly segments of the time sequence are identified by adopting the anomaly detection method of joint verification.
The invention provides a method for detecting abnormal signals of the Internet of things based on a graph neural network. Modeling each sensor in the Internet of things system as a graph node, so that a multi-element time sequence acquired by the Internet of things is modeled on a graph. Firstly, adopting static graph structure learning based on smoothness constraint and dynamic graph structure learning based on sensor characteristic embedding to extract the internal relation of the system. Secondly, embedding the GAT network into the GRU to obtain a graph attention-gated loop unit (GAGRU) model, and capturing space-time features of a multi-element time sequence by adopting a double-layer graph attention-gated loop unit (M-GAGRU), wherein the M-GAGRU is optimized according to a static graph structure and a dynamic graph structure. Finally, the abnormal time period of the multi-element time sequence is identified by joint inspection through two abnormal judging methods and two abnormal detecting evaluation methods. Simulation experiments prove that the anomaly detection method based on the graph neural network provided by the invention has good performance on two evaluation standards, and the potential of the modeling method based on the graph neural network and graph structure learning in the field of anomaly detection of the Internet of things signals is shown.
Drawings
FIG. 1 is a schematic diagram of static diagram structure learning.
Fig. 2 is an internet of things signal anomaly detection framework based on a graph neural network.
Fig. 3 is a schematic diagram of a point adjustment strategy.
Fig. 4 is a graph comparing AUC indicators of SWaT data sets.
Fig. 5 is a graph comparing AUC indicators of WADI data sets.
Fig. 6 is a graph comparing AUC indicators of SMAP datasets.
Fig. 7 is a graph comparing AUC indicators of MSL datasets.
Fig. 8 is a graph comparing AUC indicators of SMD data sets.
FIG. 9 is a graph comparing the results of the SWaT dataset anomaly prediction with the real situation under the point adjustment strategy.
FIG. 10 is a graph comparing WADI dataset anomaly prediction results with real cases under a point adjustment strategy.
FIG. 11 is a graph of the SMAP dataset anomaly prediction results versus the real case under a point adjustment strategy.
FIG. 12 is a graph comparing the anomaly prediction result and the real situation of the MSL data set under the point adjustment strategy.
Fig. 13 is a graph comparing the result of the anomaly prediction of the SMD dataset with the actual situation under the point adjustment strategy.
Detailed Description
The invention is described in further detail below with reference to the attached drawing figures and to specific examples:
for a multi-element time sequence signal acquired by the Internet of things, due to the complex space-time characteristics, effective modeling is difficult to detect the abnormality. The network based on long-short-term memory, the traditional graph neural network or some other deep learning networks are insufficient in space-time feature extraction, the inherent structure of the system is easy to ignore, and the system is insensitive to the occurrence and propagation process of anomalies.
One of the embodiments of the invention is to disclose a method for detecting the abnormality of the Internet of things signal based on a graph neural network, which comprises the steps of firstly constructing a graph structure model of sensors in the Internet of things signal, wherein each sensor sends a time sequence in the graph structure model, so as to obtain an intrinsic graph structure model of a multi-element time sequence; then combining the static graph structure and the dynamic graph structure, and adopting a double-layer graph attention gate control circulating unit (M-GAGRU) based on graph structure optimization to effectively capture the space-time characteristics of the multi-element time sequence; meanwhile, optimizing by adopting a prediction error as a loss function; calculating an anomaly score through the prediction error, and identifying anomalies by adopting two different thresholds; and evaluating the abnormality detection performance according to the evaluation indexes of the two different standards.
Based on the above problems, an embodiment of the present invention discloses a method for detecting anomalies of signals of the internet of things based on a neural network, in most researches, the problem of anomaly detection of time sequences is generally regarded as an unsupervised task, and in the task of anomaly detection of multiple time sequences, it is generally assumed that there is no anomaly in data of a model training stage, and the obtained training model can be regarded as a mode of a normal sequence. In the test phase, normal data and abnormal data are contained at the same time, and a corresponding label is generally arranged at each time stamp. By the method, the abnormal time stamp can be accurately identified in the data. The method comprises the following steps:
and step 1, a system entity establishes a graph structure model and learns the graph structure model.
First define the multiple time series of different sensor emissions asWhere T represents the number of time stamps in the time series and N represents the number of sensors in the system entity. To better represent the system state of each timestamp, at each timestamp t, a sliding window of size K in the historical time series data of the timestamp is utilized as the time feature of each sensor at the timestamp, and a graph structure model input is defined as follows:
x (t) ＝[d (t-K) ,d (t-K+1) ,d (t-K+2) ,…,d (t-1) ] (1)
wherein d (t-1) A vector consisting of a plurality of sensor data at time stamp t-1 for a multivariate time series. The final input thus obtained is
In a system entity containing N sensors, each sensor in the entity can be considered as a node on the graph, and the entire system entity can be considered as a weighted undirected graph with N nodesWherein the method comprises the steps ofRepresents a set of N nodes,>representing the weight matrix of the graph and also being a real moment of symmetryAn array. In the weight matrix of the figure, the element w of the ith row and the jth column thereof ij (w ij >0) Represents the weight between node i and node j, and has w ij ＝w ji 。
For time-varying signals, the graph structure model at each moment may be changed, so that based on the graph structure model corresponding to the system entity, a static graph structure and a dynamic graph structure are further obtained through learning, and are used for providing prior information for a subsequent deep learning model, so that model performance is optimized. And by analyzing the graph structure, the process of anomaly occurrence, propagation and interaction can be further explained.
The dynamic graph structure is used for representing the characteristic attribute of the sensor by adopting sensor characteristic embedding, and learning the dynamic graph structure of the multi-element time sequence by learning the similarity among the sensors, and is used for reflecting the time-varying property of the time sequence, and the learning process is as follows:
wherein e i A feature embedded representation representing an ith sensor; sim (·) represents cosine similarity; w (w) ji The cosine similarity between the node j and the node i is represented, and the weight of the edge in the graph can be also represented; topK (·) represents that among all neighbors of the node i, K points with the maximum similarity are selected as neighbors; k is a super parameter, and plays a role in controlling the sparsity of the graph; a is that ji Is a dynamic diagram A d The data value of the j-th row and i-th column of (a) represents the connection relationship between the node j and the node i.
FIG. 1 illustrates a static graph structure learning process based on smoothness constraints. In the initial data preprocessing process, a convex optimization method based on graph smoothness constraint is adopted to learn the static graph structure of the multi-element time sequence and is used for representing the topology of the bottommost layer of the multi-element time sequenceIf the structure is toAs a graph signal, the following optimization problem can be established:
wherein D is the preprocessed multi-element time series data; l is a graph Laplace matrix; n is the number of sensor nodes; tr (·) represents the trace of the matrix;representing the F-norm of the matrix; alpha and beta are two regularization parameters, typically set by human beings.
More specifically, the expansion of the laplacian matrix L having the matrix operation therein can be obtained:
the laplace matrix L can be obtained by solving the optimization problem of equation (5), but attention needs to be paid to that the laplace matrix at this time cannot directly calculate the adjacent matrix, and only the corresponding weight matrix can be calculated first. L is represented in the form of the following matrix:
according to the definition of the laplace matrix l=d-W, and D is a diagonal matrix, the diagonal elements of W are all 0. The weight matrix W can be inferred from the laplace matrix:
the weight matrix, although more representative than the neighbor matrix, represents the bits of the graphThe weight matrix is not an effective input to the graph attention network. Therefore, it is necessary to further infer the adjacency matrix a of the graph from the weight matrix. Element W of ith row and jth column in weight matrix ij ＝-L ij The connection strength between the node i and the node j is shown, and the larger the absolute value is, the stronger the connection relation is. Therefore, these weights are classified into two categories 01 by setting a threshold, 1 indicating that there is a connection, and 0 indicating that there is no connection. For the adjacency matrix of the graph, element A of the ith row and jth column ij Indicating whether there is a connection between node i and node j, defined as follows:
here, θ is a set threshold, which is generally an artificially set super parameter, and the sparseness of the adjacent matrix can be adjusted by controlling the size of θ.
Thus, the adjacency matrix can be obtained as follows:
the adjacency matrix is used as a bottom static graph structure in a subsequent model and assists in the training process of the network model.
The method provided by the invention aims at exploring deeper internal structures of the system by combining the static diagram and the dynamic diagram, and extracting space-time characteristics of a multi-element time sequence by combining the graphic neural network and the gating circulation unit so as to achieve more accurate detection of abnormal signals of the Internet of things. The entire abnormality detection frame is shown in fig. 2. The system structure mainly comprises graph structure learning, a double-layer graph attention-gating circulating unit (M-GAGRU) and a joint verification anomaly scoring and detecting method.
Step 2: the double-layer diagram is attention-gated loop unit (M-GAGRU). According to the invention, GAT (graph annotation network) is embedded into GRU (gate control loop unit), so that graph attention mechanism is used when GRU obtains each state and updates each state, and variant GAGRU of GRU is obtained, and GAGRU can capture space dependence characteristic and time dependence characteristic of bottom layer. In order to combine the static diagram structure and the dynamic diagram structure proposed in the present invention, two GAGRU modules are adopted to respectively correspond to each other.
The graph meaning force operation of any directed graph is defined as:
wherein G is g (-) represents a graph attention network;representing the system input at time stamp t.
More specifically, for each node's graph attention operation:
wherein,representing the input characteristics of the ith node at the time of the time stamp t; />Representing a trainable weight matrix for performing a linear transformation; reLU represents an activation function; />Representing that node j is the neighbor of node i, the connection relation in the static diagram GAGRU and the dynamic diagram GAGRU is respectively represented by A s And A d Is given; alpha ij Is->And->The calculation formula of the earth attention coefficient is as follows:
wherein, I represents matrix concatenation;is a learning vector of the attention mechanism; leakyReLU is the activation function.
Defining the static diagram structure and the dynamic diagram structure obtained in the step 1 as a dynamic diagram A d And static diagram A s And defines the attention operations of the two graphs as:
wherein,and->Respectively represent dynamic diagram A d And static diagram A s A corresponding graph-annotation force network; />Representing a system input at a time stamp t; k represents the dimension of sensor feature embedding and also represents the sliding window length during data preprocessing.
Meanwhile, graph attention operation (GAT) replaces dot product operation in GRU to obtain a new network model GAGRU, a double-layer graph attention gating circulation unit frame M-GAGRU is formed by combining a static graph structure and a dynamic graph structure, the double-layer graph attention gating circulation unit frame M-GAGRU is adopted to correspond to the static graph and the dynamic graph, and the dynamic graph is input into an uplink GAGRU module, so that the uplink module is defined as:
wherein, I represents matrix concatenation; the omicron represents a dot product; the product of Hadamard; x is X (t) Representing the preprocessed input signal; e represents feature embedding of the sensor;respectively representing a reset gate, an update gate and a hidden state;is a corresponding dynamic GAT network; />The output of the GRU at the time t is represented, and the space-time characteristics extracted by the dynamic graph GAGRU are also represented; w and b represent a learnable weight matrix and a network bias vector; sigma and tanh are activation functions, where sigma may select any activation function.
The static diagram is input into the downstream GAGRU module, so the downstream module is defined as:
wherein,respectively representing a reset gate, an update gate and a hidden state; />Is a corresponding static GAT network; />The output of the GRU at the time t is represented, and the space-time characteristics extracted by the static diagram GAGRU are also represented; w and b represent a learnable weight matrix and a network bias vector.
After the feature output is obtained, a full-connection layer is used for predicting a time sequence, and parameters of the attention-gating circulation unit of the loss function optimized double-layer diagram are obtained:
wherein f FC (. Cndot.) represents a function of the fully connected layer, which can be used to dimension the input toConverting the T multiplied by 2N feature matrix into an output matrix with the dimension of T multiplied by N; d, d (t) Is the corresponding input signal.
In this step, the two-layer graph attention-gated loop unit needs to be trained by training set data at model building.
Step 3: and (5) an anomaly score and anomaly detection method for combined verification. Considering that the evaluation of the performance of the anomaly detection model by different evaluation criteria may produce different results, and the focus of the different evaluation criteria will be different, the application will also be different. Therefore, the invention adopts the traditional F1 fraction or F1 fraction F1 after point adjustment PA And performing performance evaluation and joint verification on the M-GAGRU model.
Obtaining the anomaly score of each sensor according to the prediction error:
wherein mu i Sum sigma i Representing error i (t) Median and quartile spacing in the time dimension.
Two methods are then employed to jointly verify the detection of system anomalies:
(1) Traditional F1 score calculation:
calculating the anomaly score of the system entity at any time stamp according to the TopK criterionFurther adopts the mode of optimizing the performance to obtain the optimal threshold value threshold (t) To distinguish between anomalies in system timestamps: />
Wherein o is t =1 indicates a system abnormality; o (o) t =0 indicates that the system is normal.
(2) F1 with point adjustment PA Score calculation:
fig. 3 illustrates the core idea of point adjustment for anomaly detection. In practice, the occurrence of anomalies is typically continuous, as is the observation of anomalies, which characteristic forms successive anomaly segments over a time series. The core idea of point adjustment is therefore: it is believed that the anomaly alarm may trigger within any subset of the actual anomaly window. This also illustrates that for any observation within a true anomaly segment, the entire anomaly window is considered to be properly detected as long as anomalies are detected therein, each observation point in the segment is classified as anomalous, and observations outside the true anomaly segment are processed in a conventional manner.
In order to cooperate with the idea of point adjustment, the invention adopts POT algorithm to solve the abnormal threshold value:
wherein GT= { GT 1 ,GT 2 ,…,GT M M abnormal segments in the data set, and and->Representing the mth segment of the differenceA constant start time point and an end time point.
Step 4: and a performance verification stage. Five published real data sets, which are also the most commonly used data sets in the anomaly detection task, are used to verify the anomaly detection performance of the present method.
In order to verify the effectiveness of the proposed M-gaglu framework, the present invention uses some models that are popular in anomaly detection tasks for performance comparison, it should be noted that the present invention uses two performance evaluation criteria, but the popular models under these two criteria are partially different. Wherein, the performance index of the evaluation model adopts the index commonly used in the abnormal detection task: precision, recall, F1 score, F1 PA (F1 score Under point adjustment) and AUC score (Area Under Curve). Simulation experiment results prove that the Internet of things signal abnormality detection method based on the graph neural network has better performance than other popular models, and is more suitable for abnormal detection tasks of multiple time sequences.
Example 1
In the experiment, five data sets, SWaT, WADI, SMAP, MSL and SMD, were used for simulation. The SWaT is a safe water treatment data set, contains data results measured by 51 sensors in a system entity for 11 consecutive days, and serves as a small-sized water treatment system of the Internet of things, and natural internal relations exist among the sensors. In the process of collecting SWaT, artificial attack is added in the last days, and an abnormal label is added in the corresponding time stamp of the attack. The water distribution (WADI) dataset contains data results measured by 127 sensors in one system entity for 14 consecutive days, with more complex inter-relationships between the sensors as an extension of the SWaT dataset. Also, during the collection of WADI, anomalies are introduced by adding artificial attacks. The soil moisture active passive dataset (SMAP) is a soil sample and telemetry information dataset acquired from a spacecraft and annotated by an expert of NASA. The Mars Science Laboratory (MSL) dataset is an SMAP-like dataset, but corresponds to sensor and actuator data of the Mars detector itself. Both SMAP and MSL include pre-partitioned training and testing sets. The training set is collected from normal data and the test set includes marked anomalies. The Server Machine Dataset (SMD) is a five week dataset that records the resource utilization of 28 machines from the computing cluster. The statistical summaries for these five data sets are shown in table 1.
Table 1 statistical summary of five datasets
In the experiment, for the first traditional F1 fraction index, only F1 fraction is used as an evaluation standard of performance, USAD, DAGMM, LSTM-VAE, omniAnomaly, MSCRED, THOC and GDN are used as comparison models, and simulation results of the experiment are shown in Table 2; while for the second point-adjusted F1 score index, pre, rec, F1 is used PA And AUC were used as evaluation criteria for performance, and LSTM-NDT, DAGMM, omniAnomaly, MSCRED, MAD-GAN, USAD, MTAD-GAT, CAE-M, GDN and TranAD were used as comparative models, and simulation results of the experiment are shown in Table 3 and FIGS. 4-8 (AUC results for five data sets, respectively).
Table 2 comparison of F1 scores for different models over five data sets under conventional criteria
The results in Table 2 above show that the model of the present invention performs significantly better over five data sets than other methods. The F1 scores on the five data sets (SWaT, WADI, SAMP, MSL and SMD) improved 7.16%, 11.23%, 9.91%, 22.58% and 39.89% over the suboptimal model, respectively. Compared with the rest methods in the table, the M-GAGRU has the advantages that not only the space-time characteristics of the multi-element time sequence are successfully extracted, but also the dynamic diagram and the static diagram are combined, so that the deeper spatial correlation can be effectively modeled.
Table 3 comparison of the performance of different models on five datasets under the point adjustment strategy
The results in Table 3 above show that the model presented in this invention performs better than the rest of the popular baseline for the F1 score on all five data. Meanwhile, because the SMAP, MSL and SMD data sets have relatively simple abnormal modes and space-time characteristics, most models have good performance, the improvement of M-GAGRU is limited, and compared with a suboptimal model, the F1 fraction performance is improved by 2.57%, 2.76% and 1.23% respectively. For SWaT and WADI datasets with more complex spatiotemporal features and anomaly patterns, most models perform very well, so that M-GAGRU achieves a greater improvement in F1 fraction performance than sub-optimal models, up to 5.70% and 37.22%, respectively. It is worth mentioning that the F1 score is the most important comprehensive index in the abnormality detection task, and the model provided by the invention can be excellent, while the rest of Pre and Rec indexes are lower than the popular model, but far exceed the rest of most models.
Fig. 4-8 show AUC results for SWaT, WADI, SMAP, MSL and SMD datasets, respectively, showing that most models perform well for AUC indicators. Besides the SMD data set, the model performance of the invention is superior to the rest models. And, the promotion is also more pronounced for more complex SWaT and WADI datasets.
Fig. 9-13 are graphs comparing the results of anomaly prediction and the actual conditions of SWaT, WADI, SMAP, MSL and SMD datasets, respectively, under a point adjustment strategy, it can be found that the model of the point adjustment strategy is very sensitive and adequate for the detection of consecutive anomaly segments.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.

Claims (10)

1. The method for detecting the abnormal signals of the Internet of things based on the graph neural network is characterized by comprising the following steps of: the method comprises the following steps:
based on a system entity, a graph structure model is established, each sensor in the system entity is used as a graph node in the graph structure model, each sensor generates a time sequence to obtain an actual multi-element time sequence, and a static graph structure and a dynamic graph structure are obtained after the graph structure model is learned;
respectively defining corresponding graph meaning networks based on a static graph structure and a dynamic graph structure; embedding a graph attention mechanism into a gating circulation unit to form a GAGRU network module; processing by an uplink GAGRU network module to obtain sequence space-time characteristics based on dynamic graph modeling; processing by a downlink GAGRU network module to obtain sequence space-time characteristics based on static diagram modeling; the full connection layer predicts based on two space-time characteristics to obtain a predicted multi-element time sequence;
comparing the predicted multi-element time sequence with the actual multi-element time sequence to obtain an anomaly score; based on the anomaly score, the anomaly detection is carried out by combining the traditional F1 score evaluation standard and the F1 score evaluation standard under the point adjustment strategy.
2. The method for detecting the abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein the process of learning the graph structure model to obtain the dynamic graph structure is as follows:
wherein,e i feature embedding representing the ith sensor; sim (·) represents solving for cosine similarity; w (w) ji The cosine similarity between the node j and the node i is represented, and the weight of the edge in the graph structure model is also represented; topK (·) represents selecting K points with the greatest similarity among all neighbors of the node i as the neighbors of the node i; k is a super parameter; a is that ji Is a dynamic graph A d The data value of the j-th row and i-th column of the table represents the connection relation between the node j and the node i;
based on all A ji Obtaining dynamic diagram A ji 。
3. The method for detecting the abnormal signals of the Internet of things based on the graph neural network according to claim 1, wherein the process of learning the graph structure model to obtain the static graph structure is that an optimization problem is established, and an adjacency matrix is obtained after the optimization problem is solved, wherein the adjacency matrix is of the static graph structure.
4. The method for detecting abnormal signals of the internet of things based on the graphic neural network according to claim 3, wherein the optimization problem is as follows:
wherein D is the preprocessed multi-element time series data; l is a graph Laplace matrix; n is the number of sensor nodes; tr (·) represents the trace of the matrix;representing the F-norm of the matrix; alpha and beta are two regularization parameters;
the adjacency matrix is:
wherein A is ij Indicating that between node i and node j isWhether a connection exists.
5. The method for detecting the abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein the formula of the graph annotation network is as follows:
wherein,representing the input characteristics of the ith node at the time of the time stamp t; />Representing a trainable weight matrix; reLU represents an activation function; alpha ij Is->And->Attention coefficient, alpha ij The calculation formula of (2) is as follows:
wherein, I represents matrix concatenation;learning vectors that are the mechanism of attention; leakyReLU is the activation function.
6. The method for detecting abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein the processing procedure of the uplink GAGRU network module is as follows:
wherein, I represents matrix concatenation;representing a dot product; the product of Hadamard; x is X (t) Representing the preprocessed input signal; e represents feature embedding of the sensor; />Respectively representing a reset gate, an update gate and a hidden state; />Is a corresponding dynamic GAT network; />The output of the GRU at the time t is represented; w and b represent a learnable weight matrix and a network bias vector; sigma and tanh are activation functions, where sigma is a selectable arbitrary activation function.
7. The method for detecting abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein the processing procedure of the downstream GAGRU network module is as follows:
wherein,respectively representing a reset gate, an update gate and a hidden state; />Is a corresponding static GAT network; />The output of the GRU at the time t is represented; w and b represent a learnable weight matrix and a network bias vector.
8. The method for detecting the abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein the calculation formula of the evaluation standard of the traditional F1 score method is as follows:
wherein o is t =1 indicates a system abnormality; o (o) t =0 indicates that the system is normal, error i (t) Representing an anomaly score.
9. The method for detecting abnormal signals of the internet of things based on the graph neural network according to claim 1, wherein a calculation formula of an F1 score evaluation standard under an adjustment strategy is as follows:
wherein GT= { GT 1 ,GT 2 ,…,GT M M abnormal segments in the data set, and and->A start time point and an end time point of the mth segment abnormality.
10. The utility model provides a thing networking signal anomaly detection system based on picture neural network which characterized in that includes:
the model building unit is used for building a graph structure model based on a system entity, wherein each sensor in the system entity is used as a graph node in the graph structure model, each sensor generates a time sequence to obtain an actual multi-element time sequence, and a static graph structure and a dynamic graph structure are obtained after the graph structure model is learned;
the double-layer diagram attention gate control circulation unit is used for respectively defining corresponding diagram meaning networks based on a static diagram structure and a dynamic diagram structure; embedding a graph attention mechanism into a gating circulation unit to form a GAGRU network module; processing by an uplink GAGRU network module to obtain sequence space-time characteristics based on dynamic graph modeling; processing by a downlink GAGRU network module to obtain sequence space-time characteristics based on static diagram modeling; the full connection layer predicts based on two space-time characteristics to obtain a predicted multi-element time sequence;
the prediction unit is used for comparing the predicted multi-element time sequence with the actual multi-element time sequence to obtain an anomaly score; based on the anomaly score, the anomaly detection is carried out by combining the traditional F1 score evaluation standard and the F1 score evaluation standard under the point adjustment strategy.
CN202310759397.1A 2023-06-26 2023-06-26 Internet of things signal anomaly detection method and system based on graph neural network Pending CN117009900A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310759397.1A CN117009900A (en) 2023-06-26 2023-06-26 Internet of things signal anomaly detection method and system based on graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310759397.1A CN117009900A (en) 2023-06-26 2023-06-26 Internet of things signal anomaly detection method and system based on graph neural network

Publications (1)

Publication Number Publication Date
CN117009900A true CN117009900A (en) 2023-11-07

Family

ID=88564504

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310759397.1A Pending CN117009900A (en) 2023-06-26 2023-06-26 Internet of things signal anomaly detection method and system based on graph neural network

Country Status (1)

Country Link
CN (1) CN117009900A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117688504A (en) * 2024-02-04 2024-03-12 西华大学 Internet of things abnormality detection method and device based on graph structure learning
CN118194139A (en) * 2024-05-16 2024-06-14 南京信息工程大学 Spatio-temporal data prediction method based on adaptive graph learning and nerve controlled differential equation
CN118350801A (en) * 2024-04-15 2024-07-16 龙坤(无锡)智慧科技有限公司 Intelligent patrol method based on integration of internet of things and virtual reality of large AI model

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117688504A (en) * 2024-02-04 2024-03-12 西华大学 Internet of things abnormality detection method and device based on graph structure learning
CN117688504B (en) * 2024-02-04 2024-04-16 西华大学 Internet of things abnormality detection method and device based on graph structure learning
CN118350801A (en) * 2024-04-15 2024-07-16 龙坤(无锡)智慧科技有限公司 Intelligent patrol method based on integration of internet of things and virtual reality of large AI model
CN118194139A (en) * 2024-05-16 2024-06-14 南京信息工程大学 Spatio-temporal data prediction method based on adaptive graph learning and nerve controlled differential equation

Similar Documents

Publication Publication Date Title
CN117009900A (en) Internet of things signal anomaly detection method and system based on graph neural network
CN107506692A (en) A kind of dense population based on deep learning counts and personnel's distribution estimation method
CN115244548A (en) Method for training deep learning network based on artificial intelligence and learning equipment using same
CN110533100A (en) A method of CME detection and tracking is carried out based on machine learning
CN115146842A (en) Multivariate time series trend prediction method and system based on deep learning
CN118133203A (en) Fault diagnosis method for electric energy metering detection information
CN115457403A (en) Intelligent crop identification method based on multi-type remote sensing images
CN112163020A (en) Multi-dimensional time series anomaly detection method and system
Balaji et al. Plant Infirmity Detection Using Vgg-16 Convolutional Neural Network
CN116361640A (en) Multi-variable time sequence anomaly detection method based on hierarchical attention network
CN116244596A (en) Industrial time sequence data anomaly detection method based on TCN and attention mechanism
Yu Research progress of crop disease image recognition based on wireless network communication and deep learning
CN111860441B (en) Video target identification method based on unbiased depth migration learning
CN115293249A (en) Power system typical scene probability prediction method based on dynamic time sequence prediction
CN115097518A (en) Micro-seismic arrival time picking method based on fuzzy clustering U-shaped neural network
Zhengfeng Accurate recognition method of continuous sports action based on deep learning algorithm
Yang et al. Prediction of criminal tendency of high-risk personnel based on combination of principal component analysis and support vector machine
Tan et al. Multivariate Time-Series Anomaly Detection in IoT Using Attention-Based Gated Recurrent Unit
CN116611022B (en) Intelligent campus education big data fusion method and platform
CN118504847B (en) Intelligent beam field management method and system based on digital twin technology
CN118228613B (en) Soft measurement method for improving TSO optimization deep learning model
CN118094447B (en) Unmanned aerial vehicle flight data self-adaptive anomaly detection method based on encoding-decoding
CN116861175B (en) Operation track correction method based on neural network
CN113792776B (en) Interpretation method for deep learning model in network security anomaly detection
CN115131618B (en) Semi-supervised image classification method based on causal reasoning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination