CN117688504B - Internet of things abnormality detection method and device based on graph structure learning - Google Patents
Internet of things abnormality detection method and device based on graph structure learning Download PDFInfo
- Publication number
- CN117688504B CN117688504B CN202410156692.2A CN202410156692A CN117688504B CN 117688504 B CN117688504 B CN 117688504B CN 202410156692 A CN202410156692 A CN 202410156692A CN 117688504 B CN117688504 B CN 117688504B
- Authority
- CN
- China
- Prior art keywords
- model
- layer
- internet
- things
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 230000005856 abnormality Effects 0.000 title claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 77
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000004458 analytical method Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 230000001537 neural effect Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 230000015654 memory Effects 0.000 claims description 5
- 230000006855 networking Effects 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 abstract description 6
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 8
- 238000012360 testing method Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000008485 antagonism Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 239000003550 marker Substances 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an Internet of things abnormality detection method and device based on graph structure learning, and relates to the technical field of Internet of things, wherein the method comprises the following steps: s1, constructing an initial training model, wherein the initial training model comprises two first models and two second models; s2, acquiring a first training data set, a second training data set and a third training data set; s3, training and optimizing a first model and a second model, wherein the optimized second model is used as an anomaly detection model; s4, acquiring data of the Internet of things to be analyzed; s5, analyzing the data of the Internet of things to be analyzed by using an anomaly detection model to obtain a detection result; the second model is introduced, and unlabeled data can be fully utilized for semi-supervised training, so that generalization capability and performance of the anomaly detection model are improved, and the anomaly detection model effectively reduces labeling cost of the data.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to an Internet of things abnormality detection method and device based on graph structure learning.
Background
The internet of things (Internet of Things, ioT) is an emerging technology of collaborative terminal and sensor composition connected through the internet. The internet of things can be applied to different application fields such as intelligent home, wearable equipment, smart city, medical treatment, agriculture, traffic, industry and the like. The main advantage of the internet of things is that it is able to make appropriate decisions based on the data collected by the sensors and track the devices in an intelligent way. Typical internet of things environments involve a large number of interconnected sensors, such as in a water treatment plant, power plant, or transportation system. Typically, real-time data collected from these sensors is processed and stored as an internet of things multivariate time series. The variables in these time series are often interrelated, for example, in a water treatment plant, if sensor data monitoring flow is abnormal, sensor data monitoring water pressure may also be abnormal. Due to the large number of sensors and the complexity of the relationships between them, it is often more difficult to perform anomaly detection in more complex and large-scale internet of things systems.
The abnormal detection of the Internet of things has important significance and value in modern society. The internet of things devices are typically distributed in different geographic locations, and monitoring a large amount of device status and operational data helps predict potential faults or damage. By monitoring abnormality in real time, possible faults of the equipment can be predicted in advance, and timely maintenance is performed, so that maintenance cost and downtime are reduced, and reliability and availability of the equipment are improved. Internet of things systems involve a large amount of data transmission and processing, and are typically operated in environments where resources are limited (e.g., sensor nodes, embedded devices, etc.). The anomaly detection can help identify abnormal data flows, energy consumption and the like, so that resource allocation is optimized, and the efficiency and performance of the system are improved. The abnormal detection of the Internet of things is beneficial to guaranteeing the stability, safety and efficiency of the Internet of things system, and can also promote user experience, so that powerful support is provided for data analysis and intelligent decision making, and the method has important significance and value in Internet of things application. Anomaly detection has been a very important branch in machine learning, and is a very popular research direction in various artificial intelligence application scenarios. The method for detecting the abnormality of the data of the internet of things has been widely studied through long-time development.
The traditional abnormality detection method can process the abnormality detection task of the real-time data of the Internet of things to a certain extent. Most of them are methods based on statistical analysis, methods based on rules, methods based on models, and the like. A statistical method is proposed by Tukey in 1979 to detect anomalies in time series data. However, these conventional methods generally rely on manually defined rules and features, which are difficult to accommodate for complex and varied internet of things data features. In addition, they also do not handle large-scale data and high-dimensional data well, and their performance and scalability are limited.
With the development of technology, the number of sensors and devices in various internet of things platforms is continually increasing, which not only generates more data, but also creates a need for greater computing power. But the increased number of sensors also results in more complex relationships between sensors, which are some of the challenges encountered by time series anomaly detection tasks. In recent years, machine learning and deep learning methods have achieved important results in anomaly detection of real-time data of the internet of things.
Over the past decade, machine learning and deep learning methods have met with great success in the task of computer vision, and researchers have therefore utilized these methods for anomaly detection. Such as Autoregressive model autoregressive models (VAR), assume that there is a linear relationship of correlation between the variables. Long Short-Term Memory networks (LSTM) capture the nonlinear relationship between variables through back and forth dependencies in data time. Variational Autoencoder variation encoders (VAEs) obtain the mapping relationship between variables by the encoder and decoder. GENERATIVE ADVERSARIAL Networks generate a mapping relation between the antagonism network (GAN) and the last found variable continuously antagonized by the discrimination network through the generation network in the training process.
Although machine learning and deep learning methods have achieved some success in the task of anomaly detection for the internet of things, they have some drawbacks: ① Generally, the machine learning and deep learning methods are unsupervised methods, and the labeling cost of data is high. ② While tagged supervised learning has proven effective in practice, its limitations are increasingly apparent. The increasing data volume leads to an increase in the cost and difficulty of data labeling.
Disclosure of Invention
The invention aims to solve the problems and designs an abnormal detection method and device of the Internet of things based on graph structure learning.
The invention realizes the above purpose through the following technical scheme:
the abnormal detection method of the Internet of things based on graph structure learning comprises the following steps:
s1, constructing an initial training model, wherein the initial training model comprises two first models and two second models, and the first models and the second models have the same structure;
S2, acquiring a first training data set, a second training data set and a third training data set, wherein the first training data set comprises the data of the Internet of things with the labels, the second training data set comprises the data of the Internet of things without the labels, and the third training data set comprises the data of the Internet of things without the labels;
S3, training and optimizing a first model and a second model, wherein the training and optimizing method specifically comprises the following steps: the first training data set and the second training data set are both imported into a first model, the third training data set is imported into a second model, the first model and the second model are optimized by using analysis results obtained by the first model and the second model, the optimized first model and second model are obtained, and the optimized second model is used as an abnormality detection model;
S4, acquiring data of the Internet of things to be analyzed;
s5, analyzing the data of the Internet of things to be analyzed by using the anomaly detection model to obtain a detection result.
Abnormal detection device of thing networking based on drawing structure study includes:
A reservoir; the memory is used for storing programs;
An actuator; the executor is used for executing the program stored in the storage, and when the executor executes the program stored in the storage, the abnormal detection method of the Internet of things based on the graph structure learning is realized.
The invention has the beneficial effects that: the method can fully utilize unlabeled data to carry out semi-supervised training by introducing the second model, so that the generalization capability and performance of the anomaly detection model are improved, the effectiveness of the anomaly detection model is verified through a series of experiments, and the performance of the anomaly detection model on two data sets acquired in real world exceeds that of most of non-supervised methods although only a very small amount (1% -10%) of data labels are used, so that the aim of reducing the labeling cost of the data is effectively achieved.
Drawings
FIG. 1 is a schematic diagram of the structure of an initial training model of the present invention;
Fig. 2 is an enlarged view of (1) in fig. 1;
FIG. 3 is an enlarged view of (2) in FIG. 1;
fig. 4 is an enlarged view of (3) in fig. 1;
FIG. 5 is a test flow chart of the anomaly detection model of the present invention;
FIG. 6 is a training flow diagram of an initial training model of the present invention;
FIG. 7 is a diagram showing SWaT data features used in the present invention;
FIG. 8 is a diagram illustrating WADI data features used in the present invention;
FIG. 9 is a graph showing the performance of the anomaly detection model of the present invention for different data annotation rates on two data sets;
FIG. 10 is a ranking plot of the anomaly detection model of the present invention with all baseline methods at SWaT;
FIG. 11 is a ranking plot of the anomaly detection model of the present invention with all baseline methods at WADI.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
In the description of the present invention, it should be understood that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "left", "right", etc. are based on the directions or positional relationships shown in the drawings, or the directions or positional relationships conventionally put in place when the inventive product is used, or the directions or positional relationships conventionally understood by those skilled in the art are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the apparatus or elements referred to must have a specific direction, be configured and operated in a specific direction, and therefore should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, are used merely to distinguish between descriptions and should not be construed as indicating or implying relative importance.
In the description of the present invention, it should also be noted that, unless explicitly specified and limited otherwise, terms such as "disposed," "connected," and the like are to be construed broadly, and for example, "connected" may be either fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
The following describes specific embodiments of the present invention in detail with reference to the drawings.
As shown in fig. 1,2, 3 and 4, the method for detecting the anomaly of the internet of things based on the learning of the graph structure comprises the following steps:
S1, constructing an initial training model, wherein the initial training model comprises two first models and two second models, the first models and the second models have the same structure, the first models are Student models, and the second models are teacher models Teacher;
The first model and the second model both comprise an input layer, a graph structure learning network, a multi-layer sensing network, a random discarding layer Dropout and an output layer, wherein the graph structure learning network is used for generating an adjacency matrix, the output layer adopts a Sigmoid (S-shaped) activation function, the output of the input layer is used as the input of the graph structure learning network, the output of the input layer and the output of the graph structure learning network are used as the input of the multi-layer sensing network, the output of the multi-layer sensing network is used as the input of the random discarding layer Dropout, and the output of the random discarding layer Dropout is used as the input of the output layer; the graph structure learning module trains learning along with training of the first model, and continuously updates parameters of the neural network.
The multi-layer sensing network comprises a D layer graph neural module GCN and a D layer global average pooling layer GAP, wherein the output of the D layer graph neural module GCN is respectively used as the input of a d+1th layer graph neural module GCN and the input of the D layer global average pooling layer GAP, and the output of the D layer global average pooling layer GAP is used as the input of a random discarding layer Dropout, wherein D is a positive integer greater than 1, D is less than or equal to D-1, and D is D E;
Graph structure learning networks are used to generate adjacency matrices, and the prior art approach to constructing adjacency matrices for a graph is typically to compute similarity between nodes by euclidean distance equidistant metrics. This approach is time and space consuming, especially on large graphs, and the effect of the adjacency matrix constructed by means of calculating the distances is not necessarily optimal. To address these limitations, the present method proposes a neural network learning-based adjacency matrix expressed as:
；
；
；
；
；
；
Wherein M 1、M2 represents two randomly initialized matrixes, 、/> represents transposed matrixes of the matrixes M 1、M2 respectively, tanh represents hyperbolic tangent activation function, embedding 1、Embedding2 represents randomly initialized Node embedding, node n represents Node number,/> 、/> represents model parameters, alpha represents super-parameters of graph structure learning network saturation, A represents adjacent matrixes obtained after asymmetric change, relu represents piecewise linear activation function, I= argtopk (A [ I,:1 ]) represents that I nodes with highest weights before the adjacent matrixes A are selected through argtopk, and A [ I, -I ] =0 represents that the number of selected nodes should be between 0 and I. Because the relation between the sensors is not necessarily symmetrical, the adjacent matrix generated by the learning network of the structure is obtained through asymmetric transformation, the adjacent matrix is thinned by argtopk, and the first K sides with highest correlation are selected to obtain the final adjacent matrix A.
S2, acquiring a first training data set, a second training data set and a third training data set, wherein the first training data set comprises data of the Internet of things with labels, the second training data set comprises data of the Internet of things without labels, the third data set further comprises data with labels, and the data with labels account for 1% -10% of the data of the third data set.
S3, training and optimizing a first model and a second model, wherein the training and optimizing method specifically comprises the following steps: the first training data set and the second training data set are both imported into a first model, the third training data set is imported into a second model, the first model and the second model are optimized by using analysis results obtained by the first model and the second model, the optimized first model and second model are obtained, and the optimized second model is used as an abnormality detection model; the second model is the same model as the first model in network structure, and parameters of the second model are averaged from the first model in an exponential moving average manner. The first model learns by using the target generated by the second model, and parameters of the second model are continuously updated in the training process to adapt to the data.
As shown in fig. 6, the optimization training specifically includes:
① . The first training data set is imported into the first model to obtain a first analysis result, the second training data set is imported into the first model to obtain a second analysis result, and the third training data set is imported into the second model to obtain a third analysis result;
② . Obtaining a first loss according to the first analysis result and the second analysis result, and obtaining a second loss/> according to the second analysis result and the third analysis result;
The first loss is expressed as:
；
The second loss is expressed as:
；
Wherein denotes the output of the first model tagged data, L denotes the tag of the tagged data, S X denotes the output of the first model untagged data, T X denotes the output of the second model untagged data, crit denotes the cross entropy loss function, mes mean square error loss function;
③ . Updating parameters of the first model according to the first loss and the second loss;
The parameters for updating the second model are specifically: calculating a final Loss according to the first Loss and the second Loss, and updating parameters of the first model according to the final Loss, wherein the final Loss is expressed as:
；
Wherein, beta represents a custom parameter ranging from 0 to 1;
④ . Updating the parameters of the second model according to the updated parameters of the first model, wherein the parameters are expressed as follows:
；
Wherein represents the parameters of the t-th training of the teacher model,/> represents the parameters of the t-1 th training of the teacher model, represents the parameters of the t-th training of the student model, and alpha represents the super-parameters of the model saturation.
S4, acquiring the data of the Internet of things to be analyzed.
S5, analyzing the data of the Internet of things to be analyzed by using the anomaly detection model to obtain a detection result.
Abnormal detection device of thing networking based on drawing structure study includes:
A reservoir; the memory is used for storing programs;
An actuator; the executor is used for executing the program stored in the storage, and when the executor executes the program stored in the storage, the abnormal detection method of the Internet of things based on the graph structure learning is realized.
Experiment
1. Data set
In the experiment, two sensor data sets based on a water treatment physical bench system were used: SWat (safe water treatment) and WADI (water distribution), as shown in table 1, in both datasets operators simulated the scenarios of real world water treatment plants under attack, where the recorded anomalies were both real anomalies. SWat data sets were from singapore utility committee-coordinated water treatment benches. The SWat dataset consisted of 51 sensor data with an anomaly rate of 12.2% in this context. WADI the dataset consists of 127 sensor data, WADI as an extension of SWat.
TABLE 1
The following comparative experiments were used to compare the performance of the present method with that of the existing methods.
2. Data preprocessing
In order to improve the precision of the model, a training set and a testing set are processed through min-max data standardization, and data of different specifications are converted into unified specifications, so that the influence of scale, characteristics and distribution differences on the model is reduced.
3. Model training process, as shown in FIG. 6
1) Setting training times and inputting training data, wherein the input training data is divided into data with labels and data without labels.
2) The tagged data is input into the first model.
3) The learned adjacency matrix and the input labeled training data are input into a multi-layer perception network together, a preliminary prediction result is obtained through a random discarding layer Dropout and an output layer, and finally the prediction result and the label of the labeled training data are subjected to Crit calculation loss.
4) The unlabeled data is entered into the first model.
5) The learned adjacency matrix and the input training data without labels are input into a multi-layer perception network together, and a preliminary prediction result is obtained through a random discarding layer Dropout and an output layer.
6) The unlabeled data is entered into the second model.
7) The learned adjacency matrix and the input training data without labels are input into a multi-layer perception network together, and a preliminary prediction result is obtained through a random discarding layer Dropout and an output layer.
8) And carrying out Mse calculation loss on the unlabeled results obtained by the first model and the second model.
9) And (3) carrying out weighted calculation on the Crit Loss and the Mse Loss obtained in the step (3) and the step (8) to obtain a final Loss, and updating parameters of the first model by using the final Loss.
10 Updating the parameters of the second model with the updated parameters of the first model.
11 Repeating steps 2) to 10) until the training times are completed.
4. The model test procedure is shown in FIG. 5
Inputting test data;
Obtaining an adjacency matrix of graph structure learning network learning;
And inputting the input test data and the obtained adjacency matrix into a multi-layer sensing network of the anomaly detection model, and obtaining a prediction result through a random discarding layer Dropout and an output layer.
And comparing the obtained prediction result with the label of the test data to obtain the test result of the abnormal detection model.
5. Model performance index
(1) Abnormality detection
The performance comparison of the model employs several key performance indicators based on confusion matrix classification: accuracy, recall, F1-Score, as shown in Table 2 below.
TABLE 2
The accuracy rate Pre refers to the proportion of the samples which are actually positive among the samples which are predicted to be positive by the model to the samples which are predicted to be positive, and the calculation formula is as follows:
；
The recall ratio Rec refers to the proportion of the samples predicted to be positive in the samples actually positive to the samples actually positive, and the calculation formula is as follows:
；
F1 score is the harmonic mean of the precision and recall, calculated as:
。
6. Model comparison results
As can be seen from fig. 6, 7, 8, 9, table 3 and 4, compared with the existing model, the experimental results of the model in the real data set are as follows:
Table 3 experimental results of the present model and all baseline methods on Swat datasets
Table 4 experimental results of the present model and all baseline methods on WADI datasets
K-Means: the principle of the K-Means algorithm is to divide the data set into K clusters such that each data point belongs to the nearest cluster and the center of the cluster is the average of all data points.
PCA: the principle of the PCA (principal component analysis) algorithm is to map n-dimensional features onto k-dimensions, which are completely new orthogonal features, also called principal components, and are k-dimensional features reconstructed on the basis of original n-dimensional features.
FeB: the principle of Feature Bagging (guided clustering) algorithm is to divide the data set into a plurality of subsets, train a base model for each subset, and then average or vote the prediction results of all base models to obtain the final prediction result.
VAE: the VAE (variance self-encoder) model principle is to generate new samples by learning potential representations of the data distribution using a model of generation combining variance inference and automatic encoders. It enables reconstruction and generation of data through encoder and decoder networks while encouraging smoothness and interpretability of potential representations through regularization terms in potential space.
USAD: USAD (automatic encoder based on reverse training) model principle is an unsupervised anomaly detection method based on resistance training, by a combination of encoder-decoder architecture and resistance training, it is possible to learn how to amplify the reconstruction errors of the inputs containing anomalies, thus isolating anomalies and improving stability.
Mad_gan: the MAD_GAN (based on generating an antagonism network) model principle is to generate an antagonism network, which consists of a generator and a discriminator, and generate an output which is more and more close to real data through game antagonism between the generator and the discriminator.
OmniAnomaly: omniAnomaly (random loop network based) model principle is a method that utilizes a random Recurrent Neural Network (RNN) and a variational self-encoder (VAE), learning the potential representation while taking into account time dependence and randomness of the multivariate time series to capture the normal pattern of the multivariate time series.
Lstm_ad: the lstm_ad (long-short-term memory network based) model principle is a neural network model that combines long-short-term memory networks (LSTM) and self-attention mechanisms (AD). LSTM can efficiently process sequence data, while AD can automatically focus on different locations in the input sequence, thereby better capturing long-term dependencies in the sequence.
As can be seen from tables 3 and 4, all baseline models and the present model perform poorly over WADI dataset compared to SWaT dataset. The model was much higher in F1-Socre (F1) at a labeling rate of 10% for the data than the most advanced baseline model on both datasets. USAD performed optimally in both datasets in the baseline model, with F1 reaching 0.812 on SWaT dataset and F1 reaching 0.634 on WADI dataset. The model is improved by 8.1% on SWaT datasets compared with the most advanced baseline model under the labeling rate of 10% of data, and is improved by 22.8% on WADI datasets compared with the most advanced baseline model. The performance of the model is better than that of the most advanced baseline model when the data annotation rate is 4% on SWaT data sets, and is 2.2% higher. The performance of the model is better than that of the most advanced baseline model when the data annotation rate is 5% on WADI data sets, and the improvement is 4.2%.
As can be seen from fig. 9, overall, the F1 score of the present model on both data sets decreases with decreasing data annotation rate. Notably, the present model performs better on the SWaT dataset than the WADI dataset. This suggests that the present model utilizes the marker data on SWaT datasets more efficiently, resulting in a higher F1 score, even though the amount of marker data is reduced. However, on WADI datasets, the performance of the present model is more affected as the data mark rate decreases. The performance differences between the two data sets indicate that under limited signature data conditions, the characteristics and challenges of the data sets may affect the effectiveness of the present model.
Fig. 9, 10, and 11 show that the present model outperforms all baseline models on SWaT datasets at a data mark rate of 4%. Even at 2% data mark rate, the performance of the present model is only exceeded by USAD and OmniAnomaly baseline models. This shows that the present model is very effective in anomaly detection of SWaT datasets, especially when the marker data is limited. The results demonstrate that the ability of the present model to utilize available marker data enables it to achieve superior performance compared to other baseline approaches, highlighting its potential as a SWaT dataset advanced anomaly detection model. The performance of the model is superior to all baseline models when the data annotation rate is 5%, and is inferior to USAD, mad_gan and OmniAnomaly when the data annotation rate is 4%.
The technical scheme of the invention is not limited to the specific embodiment, and all technical modifications made according to the technical scheme of the invention fall within the protection scope of the invention.
Claims (6)
1. The method for detecting the abnormality of the Internet of things based on graph structure learning is characterized by comprising the following steps of:
s1, constructing an initial training model, wherein the initial training model comprises two first models and two second models, and the first models and the second models have the same structure;
The first model and the second model both comprise an input layer, a graph structure learning network, a multi-layer sensing network, a random discarding layer Dropout and an output layer, wherein the graph structure learning network is used for generating an adjacency matrix, the output layer adopts a Sigmoid activation function, the output of the input layer is used as the input of the graph structure learning network, the output of the input layer and the output of the graph structure learning network are used as the input of the multi-layer sensing network, the output of the multi-layer sensing network is used as the input of the random discarding layer Dropout, and the output of the random discarding layer Dropout is used as the input of the output layer;
The multi-layer sensing network comprises a D layer graph neural module GCN and a D layer global average pooling layer GAP, wherein the output of the D layer graph neural module GCN is respectively used as the input of a d+1th layer graph neural module GCN and the input of the D layer global average pooling layer GAP, and the output of the D layer global average pooling layer GAP is used as the input of a random discarding layer Dropout, wherein D is a positive integer greater than 1, D is less than or equal to D-1, and D is D E;
The adjacency matrix is expressed as:
；
；
；
；
；
；
Wherein M 1、M2 represents two matrixes which are initialized randomly, 、/> represents transposed matrixes of the matrix M 1、M2 respectively, tanh represents hyperbolic tangent activation function, embedding 1、Embedding2 represents Node embedding of random initialization, node n represents Node quantity,/> 、/> represents model parameter, alpha represents super parameter of graph structure learning network saturation, A represents adjacent matrix obtained after asymmetric change, relu represents piecewise linear activation function,/> represents Node with highest I weight before selecting adjacent matrix A through argtopk, and/> represents quantity of selected nodes between 0 and I;
S2, acquiring a first training data set, a second training data set and a third training data set, wherein the first training data set comprises the data of the Internet of things with the labels, the second training data set comprises the data of the Internet of things without the labels, and the third training data set comprises the data of the Internet of things without the labels;
S3, training and optimizing a first model and a second model, wherein the training and optimizing method specifically comprises the following steps: the first training data set and the second training data set are both imported into a first model, the third training data set is imported into a second model, the first model and the second model are optimized by using analysis results obtained by the first model and the second model, the optimized first model and second model are obtained, and the optimized second model is used as an abnormality detection model;
S4, acquiring data of the Internet of things to be analyzed;
s5, analyzing the data of the Internet of things to be analyzed by using the anomaly detection model to obtain a detection result.
2. The internet of things anomaly detection method based on graph structure learning of claim 1, wherein the optimization training specifically comprises:
① The first training data set is imported into the first model to obtain a first analysis result, the second training data set is imported into the first model to obtain a second analysis result, and the third training data set is imported into the second model to obtain a third analysis result;
② Obtaining a first loss according to the first analysis result and the second analysis result, and obtaining a second loss/> according to the second analysis result and the third analysis result;
③ Updating parameters of the first model according to the first loss and the second loss;
④ And updating the parameters of the second model according to the updated parameters of the first model.
3. The internet of things anomaly detection method based on graph structure learning of claim 2, wherein the first loss is represented as:
；
The second loss is expressed as:
；
Wherein denotes the output of the first model tagged data, L denotes the tag of the tagged data, S X denotes the output of the first model untagged data, T X denotes the output of the second model untagged data, crit denotes the cross entropy loss function, mes mean square error loss function.
4. The method for detecting abnormal internet of things based on graph structure learning according to claim 2 or 3, wherein in ③, the parameters for updating the second model are specifically: calculating a final Loss according to the first Loss and the second Loss, and updating parameters of the first model according to the final Loss, wherein the final Loss is expressed as:
；
wherein β represents a custom parameter in the range of 0-1.
5. A method for detecting an anomaly in the internet of things based on graph structure learning according to claim 2 or 3, wherein, in ④, the parameters of the second model are updated according to the updated parameters of the first model, which is expressed as:
；
Wherein represents the parameters of the second model for the t-th round of training,/> represents the parameters of the second model for the t-1 th round,/> represents the parameters of the first model for the t-th round, and α represents the super-parameters of the model saturation.
6. Abnormal detection device of thing networking based on drawing structure study, its characterized in that includes:
A reservoir; the memory is used for storing programs;
An actuator; the executor is configured to execute a program stored in the storage, and when the executor executes the program stored in the storage, the method for detecting abnormal internet of things based on graph structure learning according to any one of claims 1 to 5 is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410156692.2A CN117688504B (en) | 2024-02-04 | 2024-02-04 | Internet of things abnormality detection method and device based on graph structure learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410156692.2A CN117688504B (en) | 2024-02-04 | 2024-02-04 | Internet of things abnormality detection method and device based on graph structure learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117688504A CN117688504A (en) | 2024-03-12 |
| CN117688504B true CN117688504B (en) | 2024-04-16 |
Family
ID=90128724
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410156692.2A Active CN117688504B (en) | 2024-02-04 | 2024-02-04 | Internet of things abnormality detection method and device based on graph structure learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117688504B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541904A (en) * | 2020-12-16 | 2021-03-23 | 西安电子科技大学 | Unsupervised remote sensing image change detection method, storage medium and computing device |
| CN113111760A (en) * | 2021-04-07 | 2021-07-13 | 同济大学 | Lightweight graph convolution human skeleton action identification method based on channel attention |
| CN114077811A (en) * | 2022-01-19 | 2022-02-22 | 华东交通大学 | Electric power Internet of things equipment abnormality detection method based on graph neural network |
| CN114612681A (en) * | 2022-01-30 | 2022-06-10 | 西北大学 | GCN-based multi-label image classification method, model construction method and device |
| CN114607667A (en) * | 2022-03-07 | 2022-06-10 | 南京工程学院 | Electro-hydraulic servo valve fault diagnosis system and method based on characteristic distillation |
| CN115601745A (en) * | 2022-09-09 | 2023-01-13 | 广东工业大学（Cn） | Multi-view three-dimensional object identification method facing application end |
| CN115758173A (en) * | 2022-11-16 | 2023-03-07 | 西华大学 | Cloud platform system anomaly detection method and device based on parallel graph attention network |
| CN115909016A (en) * | 2023-03-10 | 2023-04-04 | 同心智医科技(北京)有限公司 | System, method, electronic device, and medium for analyzing fMRI image based on GCN |
| WO2023070229A1 (en) * | 2021-11-01 | 2023-05-04 | Zymeworks Bc Inc. | Systems and methods for polymer side-chain conformation prediction |
| CN117009900A (en) * | 2023-06-26 | 2023-11-07 | 西安交通大学 | Internet of things signal anomaly detection method and system based on graph neural network |
| CN117350330A (en) * | 2023-10-11 | 2024-01-05 | 中国人民解放军国防科技大学 | Semi-supervised entity alignment method based on hybrid teaching |
| WO2024011475A1 (en) * | 2022-07-14 | 2024-01-18 | Robert Bosch Gmbh | Method and apparatus for graph neural architecture search under distribution shift |
-
2024
- 2024-02-04 CN CN202410156692.2A patent/CN117688504B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541904A (en) * | 2020-12-16 | 2021-03-23 | 西安电子科技大学 | Unsupervised remote sensing image change detection method, storage medium and computing device |
| CN113111760A (en) * | 2021-04-07 | 2021-07-13 | 同济大学 | Lightweight graph convolution human skeleton action identification method based on channel attention |
| WO2023070229A1 (en) * | 2021-11-01 | 2023-05-04 | Zymeworks Bc Inc. | Systems and methods for polymer side-chain conformation prediction |
| CN114077811A (en) * | 2022-01-19 | 2022-02-22 | 华东交通大学 | Electric power Internet of things equipment abnormality detection method based on graph neural network |
| CN114612681A (en) * | 2022-01-30 | 2022-06-10 | 西北大学 | GCN-based multi-label image classification method, model construction method and device |
| CN114607667A (en) * | 2022-03-07 | 2022-06-10 | 南京工程学院 | Electro-hydraulic servo valve fault diagnosis system and method based on characteristic distillation |
| WO2024011475A1 (en) * | 2022-07-14 | 2024-01-18 | Robert Bosch Gmbh | Method and apparatus for graph neural architecture search under distribution shift |
| CN115601745A (en) * | 2022-09-09 | 2023-01-13 | 广东工业大学（Cn） | Multi-view three-dimensional object identification method facing application end |
| CN115758173A (en) * | 2022-11-16 | 2023-03-07 | 西华大学 | Cloud platform system anomaly detection method and device based on parallel graph attention network |
| CN115909016A (en) * | 2023-03-10 | 2023-04-04 | 同心智医科技(北京)有限公司 | System, method, electronic device, and medium for analyzing fMRI image based on GCN |
| CN117009900A (en) * | 2023-06-26 | 2023-11-07 | 西安交通大学 | Internet of things signal anomaly detection method and system based on graph neural network |
| CN117350330A (en) * | 2023-10-11 | 2024-01-05 | 中国人民解放军国防科技大学 | Semi-supervised entity alignment method based on hybrid teaching |
Non-Patent Citations (4)
| Title |
|---|
| A Novel Semi-supervised IoT Time Series Anomaly Detection Model Using Graph Structure Learning;Weijian Song等;《Collaborative Computing: Networking, Applications and Worksharing》;20240223;第562卷;385-401 * |
| Effectively Detecting Operational Anomalies In Large-Scale IoT Data Infrastructures By Using A GAN-Based Predictive Model;Peng Chen等;《Security in Computer Systems and Networks The Computer Journal》;20221001;1-17 * |
| Multivariate Time Series Deep Spatiotemporal Forecasting with Graph Neural Network;Zichao He等;《applied sciences》;20220605;1-21 * |
| 基于图神经网络的药物-靶点相互作用预测方法;许敏萁;《中国优秀硕士学位论文全文数据库 (工程科技Ⅰ辑)》;20220115(第2022（01）期);B016-1773 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117688504A (en) | 2024-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Short-term wind power forecasting approach based on Seq2Seq model using NWP data | |
| Huang et al. | A projective and discriminative dictionary learning for high-dimensional process monitoring with industrial applications | |
| Pang et al. | Spatio-temporal fusion neural network for multi-class fault diagnosis of wind turbines based on SCADA data | |
| Loy-Benitez et al. | Soft sensor validation for monitoring and resilient control of sequential subway indoor air quality through memory-gated recurrent neural networks-based autoencoders | |
| Corizzo et al. | Anomaly detection and repair for accurate predictions in geo-distributed big data | |
| Sun et al. | Stacked denoising autoencoder with density-grid based clustering method for detecting outlier of wind turbine components | |
| CN111352977B (en) | Time sequence data monitoring method based on self-attention bidirectional long-short term memory network | |
| Zhao et al. | A novel multivariate time-series anomaly detection approach using an unsupervised deep neural network | |
| CN117290800B (en) | Timing sequence anomaly detection method and system based on hypergraph attention network | |
| Tasfi et al. | Deep neural networks with confidence sampling for electrical anomaly detection | |
| Li et al. | Deep spatio-temporal wind power forecasting | |
| CN114021483A (en) | Ultra-short-term wind power prediction method based on time domain characteristics and XGboost | |
| Chen et al. | Research on wind power prediction method based on convolutional neural network and genetic algorithm | |
| Raihan et al. | A Bi-LSTM Autoencoder Framework for Anomaly Detection-A Case Study of a Wind Power Dataset | |
| Fan et al. | Luad: A lightweight unsupervised anomaly detection scheme for multivariate time series data | |
| CN117688504B (en) | Internet of things abnormality detection method and device based on graph structure learning | |
| Li et al. | TC-GATN: Temporal causal graph attention networks with nonlinear paradigm for multivariate time series forecasting in industrial processes | |
| CN117033923A (en) | Method and system for predicting crime quantity based on interpretable machine learning | |
| CN116384576A (en) | Wind speed prediction method, device, system and storage medium | |
| CN115907198A (en) | Long-distance heat supply load intelligent prediction system | |
| Duan et al. | TCAD: unsupervised anomaly detection based on global local representation differences | |
| Han et al. | Online aware synapse weighted autoencoder for recovering random missing data in wastewater treatment process | |
| Zhu et al. | Resilience-Oriented Extreme Weather Conditional Renewable Scenario Generation Based on Diffusion Models and Few-shot Learning | |
| Ma et al. | TPAD: Temporal pattern based neural network model for anomaly detection in multivariate time series | |
| He et al. | Overview of Key Performance Indicator Anomaly Detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |