CN117336071A - Internet of things equipment safety protection method and device based on distributed AI - Google Patents
Internet of things equipment safety protection method and device based on distributed AI Download PDFInfo
- Publication number
- CN117336071A CN117336071A CN202311343739.8A CN202311343739A CN117336071A CN 117336071 A CN117336071 A CN 117336071A CN 202311343739 A CN202311343739 A CN 202311343739A CN 117336071 A CN117336071 A CN 117336071A
- Authority
- CN
- China
- Prior art keywords
- data
- expressed
- model
- error
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 72
- 230000007123 defense Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 93
- 239000011159 matrix material Substances 0.000 claims description 30
- 230000001133 acceleration Effects 0.000 claims description 28
- 210000002569 neuron Anatomy 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 21
- 230000004913 activation Effects 0.000 claims description 20
- 230000003044 adaptive effect Effects 0.000 claims description 20
- 238000013528 artificial neural network Methods 0.000 claims description 14
- 238000009826 distribution Methods 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 12
- 238000005457 optimization Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 230000033001 locomotion Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 3
- 239000008358 core component Substances 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 239000000463 material Substances 0.000 claims description 3
- 230000004931 aggregating effect Effects 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 230000003213 activating effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000004044 response Effects 0.000 description 6
- 230000009545 invasion Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008260 defense mechanism Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0209—Architectural arrangements, e.g. perimeter networks or demilitarized zones
- H04L63/0218—Distributed architectures, e.g. distributed firewalls
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/098—Distributed learning, e.g. federated learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0263—Rule management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
- H04L63/205—Network architectures or network communication protocols for network security for managing network security; network security policies in general involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Hardware Design (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to the technical field of the Internet of things, in particular to a safety protection method and a safety protection device for equipment of the Internet of things based on a distributed AI, comprising the following specific steps: s1: training a model on a global data set by a training method of the model through a federal learning distributed framework; s2, model reasoning is carried out through a trained model of the central server side so as to form risk intrusion classification of the Internet of things equipment; s3, identifying malicious attacks through a risk intrusion classification scheme, and adjusting a defense strategy in real time. The invention can carry out model training under the condition of not directly sharing data by using the federal learning technology, effectively protects the privacy of user data, is difficult to realize in the traditional scheme, adopts an ESAAE algorithm to carry out data expansion, effectively utilizes limited data resources and improves the data utilization rate.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to a safety protection method and a safety protection device for Internet of things equipment based on a distributed AI.
Background
Along with the rapid development of the internet of things, more and more intelligent devices are connected to the network, so that great convenience is brought to the life of people. However, there are many potential safety hazards of the internet of things device, such as device loopholes, data leakage, malicious attacks, and the like, so that the internet of things device becomes a main target of an attacker. Therefore, ensuring the safety of the internet of things equipment is important.
In the traditional scheme of the internet of things safety protection, a centralized model training method is generally adopted, and all data are collected on a central server for training. This method, while simple, has the following problems:
1. data privacy problem in the traditional internet of things safety protection scheme, a centralized model training method is generally adopted, and all data are collected to a central server for training. This method easily causes data privacy disclosure, because all data is uploaded to the central server, and the privacy and security of the data cannot be effectively protected.
2. The data transmission overhead is large, which results in a large overhead for data transmission due to the need to upload all data to the central server. Particularly, under the condition that the number of the devices of the internet of things is large, a large amount of data uploading can cause network congestion and influence network performance.
3. The model effect is limited, the traditional safety protection scheme of the Internet of things equipment often adopts a simple model training method, and the model effect is possibly limited. Because of the huge amount of data and complex sources, centralized model training may not fully mine the potential information of the data, resulting in poor model results.
4. The dynamic defense capability is lacking, the traditional security protection scheme of the Internet of things equipment generally adopts a static defense strategy, and the real-time response and the dynamic defense capability to risk invasion are lacking. When the Internet of things equipment faces risk invasion, effective response cannot be quickly made, and the safety of the Internet of things equipment is reduced.
5. The data expansion and feature extraction capabilities are limited-traditional schemes are generally limited in terms of data expansion and feature extraction. Especially under the condition that the data volume of the internet of things equipment is insufficient or the data distribution is uneven, the traditional scheme can not fully utilize the existing data, so that the model training effect is not ideal.
6. The model training and optimizing method is single, and the traditional safety protection scheme of the Internet of things equipment often adopts the single model training and optimizing method. This approach may not fully mine the potential information of the data, affecting the performance of the model.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a safety protection method and device for equipment of the Internet of things based on distributed AI, and the specific technical scheme is as follows:
the Internet of things equipment safety protection method based on the distributed AI comprises the following specific steps:
s1: training a model on a global data set by a training method of the model through a federal learning distributed framework;
s2, model reasoning is carried out through a trained model of the central server side so as to form risk intrusion classification of the Internet of things equipment;
s3, identifying malicious attacks through a risk intrusion classification scheme, and adjusting a defense strategy in real time.
As a further technical scheme of the invention, the federal learning distributed framework comprises a central server and a plurality of users;
the method for training the model on the global data set by the federal learning distributed framework comprises the following steps:
s11: data acquisition is carried out through the Internet of things, and each user terminal marks the acquired data;
s12: carrying out vectorization processing on the data, and converting each data attribute into a digital value;
s13: carrying out data preprocessing on the vectorized data;
s14: expanding the preprocessed data through an ESAAE algorithm;
S15: extracting features of the expanded training data through a differential motion optimization algorithm;
s16: training a classifier by utilizing the training data after feature extraction through an RSNN algorithm;
s17: uploading parameters of the model after the user terminals are trained by using the local data to a central server terminal, and judging whether the model is iterated; if yes, stopping training, if not, entering S18;
s18: and aggregating the received parameters to form a global parameter model, updating the global parameter to the client, judging whether the global model is converged, if so, stopping training, and if not, returning to the step S17.
As a further technical scheme of the present invention, the preprocessing method of the data in S13 is as follows:
firstly, carrying out normalization processing on numerical data, wherein the normalization formula is as follows:
wherein the j-th numerical attribute of the i-th data point in the vectorized data set D is x ij Wherein, j is more than or equal to 1 and less than or equal to m, n is the size of the data set, m is the number of numerical attributes, and min j Sum max j The minimum and maximum values of the j-th numerical attribute,is a normalized value;
next, the missing values in the data are processed using the following formula:
Wherein mu j Is the average or median of the j-th attribute.
As a further technical solution of the present invention, the ESAAE algorithm in S14 includes:
self-encoder: the self-encoder is a core component of the ESAAE and is used for compressing and reconstructing data; the structure comprises an encoder and a decoder, wherein the encoder maps input data x to a hidden layer h, and the decoder maps the hidden layer h back to reconstructed input data x';
a discriminator: the discriminator is used for distinguishing real data from generated data and comprises a plurality of hidden layers and an output layer, wherein the output of the discriminator is a probability value which represents the probability that the input data is the real data;
a generator: the generator is for generating new data samples that take as input the hidden layer activation state from the encoder and generate new data through a series of deconvolution operations;
the ESAAE algorithm flow is as follows:
s141: let W be in For inputting weight matrix W res The size of the input matrix is N multiplied by M for the internal weight matrix of the network, wherein N is the node number of the network, M is the input dimension, the size of the residual matrix is N multiplied by N, and the parameters of the ESAAE, including the weights and the bias of the self-encoder and the discriminator, are initialized;
S142: by encoding the input data x into the hidden layer h from the encoder section, the activation state of the hidden layer is updated by the following formula:
h t+1 ＝(1-α)h t +tanh(W in x t +W res h t )
where α is the update rate, W in And W is res Respectively an input weight matrix and a residual weight matrix. h is a t+1 Is the activation state of the residual layer at the t+1th iteration, h t Is the activation state of the residual layer at the t-th iteration;
then again, x, the hidden layer h is decoded back to the original data x by the decoding part of the self-encoder, which can be expressed as:
x＝σ(W dec h+b dec )
wherein σ () is a Sigmoid activation function, W dec Is a decoding weight matrix, b dec Is the decoding bias;
calculation ofLoss function of self-encoderDefine the loss function of the custom encoder>The goal of the self-encoder is to minimize reconstruction errors, i.e. the difference between the original data x and the decoded data x'; expressed in terms of Mean Square Error (MSE):
where n is the number of samples;
using the generator portion, taking as input the hidden layer activation state of the self-encoder and generating new data by a series of deconvolution operations; the goal of the generator is to generate dummy data that is similar to the real data distribution;
the loss function of the generator consists of two parts: reconstruction loss and fight loss; wherein the reconstruction loss is The countering loss is defined as:
wherein h is the hidden layer output of the self-encoder, G (h) is the generator generates a new sample using the hidden layer output of the self-encoder, D () is a discriminant function, and h-p (h) represent that h obeys the distribution p (h);
classifying the real data and the generated data using a discriminator section; the arbiter accepts the real data and the generated data as inputs and outputs a probability value, the output of the arbiter is calculated using the following formula:
wherein pa is the probability value, W, output by the discriminator disc Is a discriminant weight matrix, b disc Is the arbiter bias;
calculating a loss function of a discriminantDefine the loss function of the arbiter>The aim of the discriminator is to distinguish between real data and generated data; expressed in terms of a cross entropy loss function (CE):
wherein y is a real label, and p is the prediction probability of the discriminator;
using gradient descent methods, while optimizing the loss function of ESAAELoss function->Can be expressed as:
wherein, alpha, beta and lambda are all preset super parameters;
in each iteration, the self-encoder and the arbiter counter-propagate the loss function and update the parameters at the same time;
s143: and (S142) iterating the step until the iteration times reach the maximum iteration times preset by people, stopping iterating, and adding the sample generated by the generator into the preprocessed data set of the current user side to serve as an expanded training data set.
As a further technical solution of the present application, the differential motion optimization algorithm in S15 includes:
for each weight w [ i ] in the neural network]Is optimized to have an initial velocity v [ i ]]Acceleration a cs [i]。a cs [i]The error can be calculated by the error of the current network output and the error difference of the last time, and can be expressed as:
a cs [i]＝ka*(error current -error previous )
where ka is a super parameter representing the gain of the acceleration. error (error) current Error, being the error of the current network previous Is the error of the network at the last iteration;
error for current network current Can be expressed as:
where N is the number of samples,is the network output of the kth sample, +.>Is the true label of the kth sample, error current Is an error of the network.
Further, a kinetic energy correction factor is calculated. For each weight w [ i ] and bias b [ i ], its kinetic energy K [ i ] is calculated, which can be expressed as:
K[i]＝0.5*m[i]*v[i] 2
where m [ i ] is the scale of the weight or bias and v [ i ] is the update rate. Defining a function f (K i) to calculate the kinetic energy correction factor, which can be expressed as:
where e is the base of the natural logarithm.
Further, the acceleration is adjusted. Using the kinetic energy correction factor to adjust acceleration can be expressed as:
a[i]＝a cs [i]*f(K[i])
further, updates of speed and weight are made. For each weight w [ i ], the velocity v [ i ] is first updated using the adjusted acceleration a [ i ], which can be expressed as:
v[i] new ＝[v[i] old *h(adpt[i])*f(IG)+a[i]*dt]*μ
Where dt is a small time interval that is continuously adjusted by the dynamic learning rate. h (adpt [ i ]) is a parameter adjustment factor that functions to optimize a parameter adjustment process using historical update information of parameters, thereby improving performance and convergence speed of the algorithm. f (IG) is an information gain adjustment factor that acts as a historical update to account for parameters, making the algorithm more flexible and efficient. μ is an adaptive inertia factor.
The adaptive inertia factor mu is adaptively adjusted in each iteration, and the adaptive adjustment mode can be expressed as follows:
wherein mu t Adaptive inertia factor, μ for the t-th iteration t-1 Is the self-adaptive inertia factor of the t-1 step, beta 3 Is a super parameter, |g t And I is the absolute value of the gradient of the t-th step.
Further, the calculation manner of the parameter adjustment factor can be expressed as:
parameter adaptability is calculated. Parameter adaptability is defined as the mean of the update rate of the parameter over a time window, with a sliding window used to calculate the adaptability of each weight and bias. Let v [ i ]] t-1 ,v[i] t-2 ,...,v[i] t-n Is the weight w [ i ]]Or bias b [ i ]]The speed in the last n updates is calculated as its adaptive adpt [ i ]]The following are provided:
where n is the size of the sliding window.
And (5) self-adaptive parameter adjustment. The update speed of the weights and the bias is adjusted according to the parameter adaptability, and a function h (adpt [ i ]) is defined to calculate a parameter adjustment factor:
where β is a superparameter representing the adaptive weight.
Further, the information gain adjustment factor f (IG) may be calculated as:
the information gain is calculated. Defining the information gain as a measure of the difference between the current weight or bias value and its historical value is accomplished by calculating the change in entropy. The weights or offsets are set at times t and t-1, respectively, to be w t And w t-1 Their information gain IG can be calculated as follows:
IG＝-p(w t )log(p(w t ))+p(w t-1 )log(p(w t-1 ))
where p (w) is the probability distribution of the weight or bias value w.
And (5) adjusting the gain of the adaptive information. The update speed of the weights and offsets is adjusted according to the information gain. Defining a function f (IG) to calculate the information gain adjustment factor:
wherein, gamma is a super parameter, which represents the weight of the information gain;
further, the dt adjustment mode can be expressed as:
error curvature is defined as the second derivative of error with respect to time. The error curvature is estimated by calculating the errors at three consecutive time points. Setting error previous ，error current And error next The last, current and next errors, respectively, the estimated error curvature curvatus is as follows:
Where dt is the time interval.
Self-adaptive learning rate adjustment; the learning rate is adjusted according to the error curvature, and a function g (cure) is defined to calculate a learning rate adjustment factor:
where α is a hyper-parameter representing the weight of the curvature.
Using the learning rate adjustment factor to update the learning rate dt can be expressed as:
dt new ＝dt old *g(curvature)
further, a new velocity v [ i ] is used] new To update the weights w i]Can be expressed as:
w[i] new ＝w[i] old +v[i] new *dt
where dt is a hyper-parameter that can be considered as a learning rate; k is the gain of acceleration, and the speed of parameter updating is controlled;
further, for updating the bias, similar to the update process of the weights, using acceleration and velocity to update the bias b [ i ], can be expressed as:
for each bias, the acceleration is first calculated, which can be expressed as:
a b [i]＝a cs [i]*f(K[i])
further, the update speed may be expressed as:
v b [i] new ＝v b [i] old +a b [i]*dt
further, the update bias may be expressed as:
b[i] new ＝b[i] old +v b [i] new *dt
in each training iteration, the weight and the bias are updated according to the kinetic energy correction factors, and the acceleration is dynamically adjusted, so that the motion of an object can be better simulated, and the algorithm is more flexible and efficient.
As a further technical solution of the present application, the RSNN algorithm in S16 includes:
First, a multi-layered impulse neural network is constructed, consisting of an input layer, a hidden layer, and an output layer. Neurons of the hidden layer all employ a pulsed activation function.
Using impulse neurons as the basic unit in a network, the input and output relationships of one impulse neuron can be expressed as:
where u (t) is the input of the neuron, w i Is the weight of the neuron, x i (t) is the output of the previous layer neuron, b is the bias, and N is the dimension of the input. A is an attention weight matrix, which consists of attention weights;
specifically, attention weight A ij Calculated from the function fc (), can be expressed as:
wherein s (Fx i ,Fx j ) Is a calculation of two features Fx i And Fx j Function of similarity. Definition s (Fx) i ,Fx j ) The following form:
s(Fx i ,Fx j )＝θ T [Fx i ；Fx j ]
wherein θ is a parameter vector to be learned, [ Fx ] i ；Fx j ]Is a splice of two features;
further, the output y (t) of each neuron is a pulse function, which may be in the form of:
where θ is the threshold of the neuron;
to evaluate the performance of the model, a first loss function L is defined origin Can be expressed as:
wherein t is i Is a real tag, p i Is the predictive probability of the model, M is the number of categories;
further, the loss function of the present invention defines an entropy function H () to calculate the uneven distribution of the attention weights, which can be expressed as:
Adding the entropy function term into the loss function, and updating the parameters:
L＝L origin +α d ×H(A)
wherein L is origin As a first loss function, alpha d Is a manually preset super parameter;
further, for optimization on the Riemann manifold, it is necessary to calculate the Riemann gradient; at point w on manifold M, riemann gradient grad w L is the only vector that satisfies the following condition:
wherein g w Is a measure at point w on manifold M, T w M is the tangent space of manifold M at point w, DL (w) [ v ]]Is the derivative of the function L along the direction v at the point w;
further, an inner product is defined on the Riemann manifold to calculate the Riemann gradient and the Riemann Hessian matrix, and gradient descent is performed on the Riemann manifold. The weight update formula is:
wherein,is from point w t Index map of departure->Is at point w t Riemann gradient at, η is learning rate;
based on this, the following steps are repeatedly performed until the stop condition is satisfied:
forward propagation: sending the input data into a neural network, and calculating the output of each layer;
calculating loss: calculating a loss using the defined loss function;
back propagation: calculating the gradient of each layer of weight according to the loss function;
updating weights: the weights are updated based on the gradient descent of the Riemann manifold.
As a further technical solution of the present application, the method for determining whether the global model converges in S18 is:
before training begins, a training round number can be set, and once the round number is reached, model training is stopped.
The method for judging whether the global model converges in the S18 is as follows:
when the training and verification loss function value of the model stops or changes obviously decrease, the model can be considered to be converged, and the model training is stopped;
let the classifier model be fe (& theta) e ) Wherein θ e Is a parameter of the model. Given a sample x to be classified e Using model fe (. Theta.; theta) e ) Classifying the materials to obtain a classification result y pred Can be expressed as:
y pred ＝fe(x e ；θ e )
further, using the F1-score to evaluate the classification result, F1-score is a harmonic mean of Precision (Precision) and Recall (Recall), which can be expressed as:
wherein:
where TP is the true number of cases, FP is the false positive number, and FN is the false negative number.
The internet of things equipment safety protection device based on the distributed AI, which is applied to any one of the technical schemes, comprises the following steps:
and a data acquisition module: the method is used for collecting data in the Internet of things;
The execution module: training a model on the global dataset;
and a data processing module: the method is used for processing the entered data, identifying malicious attacks and adjusting the defending strategy in real time.
The beneficial effects of the invention are as follows:
1. data privacy protection: by using the federal learning technology, the invention can perform model training under the condition of not directly sharing data, effectively protects the privacy of user data, which is difficult to realize in the traditional scheme.
2. The data utilization rate is improved: the invention adopts ESAAE algorithm to expand data, effectively utilizes limited data resources and improves the data utilization rate.
3. Model accuracy and robustness are improved: through feature extraction and classifier training optimization, the method improves accuracy and robustness of the model, and risk invasion is more effectively identified and defended.
4. Dynamic defense: the invention can automatically trigger the dynamic defense mechanism, respond to risk invasion in real time, and is more flexible and effective compared with the traditional static defense strategy.
5. Pertinence protection: the method can classify and analyze the intrusion behavior, adopts targeted protective measures, and defends malicious attacks more accurately and effectively.
6. Real-time response: the intrusion behavior is analyzed through the risk intrusion classification method, and the response can be automatically or manually performed, such as adjusting security policies, tracking attack sources and the like, so that the real-time response capability to risk intrusion is improved.
Drawings
FIG. 1 illustrates a schematic structural diagram of a federal learning distributed framework;
FIG. 2 shows a flow diagram of a model trained on a global dataset;
fig. 3 shows a schematic flow diagram of a security protection apparatus for an internet of things device based on a distributed AI.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the embodiments.
The application provides a distributed AI-based (advanced technology attachment) -based safety protection method for Internet of things equipment, which adopts a training method for a model by adopting a federal learning distributed framework, and consists of a central server and a plurality of users, wherein the training method can realize that each user participating in training in the safety protection task for the Internet of things equipment trains the model on a global data set under the condition of not sharing data, and comprises the following specific steps of:
s1: training a model on a global data set by a training method of the model through a federal learning distributed framework;
S11: data acquisition is carried out through the Internet of things, and each user terminal marks the acquired data.
The data formats of the user terminals are stored in a CSV format, each row of records corresponds to one data point, and each column represents one data attribute; the data attributes of each user side include: device type, device manufacturer, operating system version, software version, CPU model, memory size, network traffic, port status, login failure times, and device log; each data attribute is represented by an appropriate value or string.
Each user side marks the acquired data and mainly comprises marks whether risk invasion exists or not. There are two possible values for the labels: 0 represents no risk intrusion and 1 represents a risk intrusion.
For example: the data attribute values of the data points of the Internet of things equipment are as follows:
device type: a camera; equipment manufacturers: vendor A; operating system version: OS_v1; software version: sw_v2; CPU model: CPU_1; memory size: 4GB; network traffic: 1000KBps; port state: opening; number of login failures: 3, a step of; device log: "error occurred during login"; then, the vector of this data point is expressed as:
X= [ "camera", "vendor a", "OS v 1","SW v 2","CPU 1 ","4GB","1000KBps",
"on", "3", "error CCureddingLogin")
And, the data is marked as: y=1 (indicating that there is a risk intrusion)
Where X represents a feature vector of the data point and Y represents a marker of the data point.
And S12, carrying out vectorization processing on the data, and converting each data attribute into a digital value.
Wherein, the non-numerical attributes such as the device type, the device manufacturer, the operating system version, the software version, the CPU model and the like are vectorized through One-hot encoding (One-hot encoding).
One-hot encoding is a method of converting a classification variable into a binary vector. For example, for a device type attribute, three device types are set: "camera" is denoted as [1, 0]; "temperature sensor" is denoted as [0,1,0]; the "router" is denoted as [0, 1].
For the numerical attributes such as memory size, network traffic and the like, the numerical attributes are directly used as elements of vectors.
For port state attributes such as "on" and "off," binary encoding may be used, i.e., with "on" being denoted as 1 and "off being denoted as 0.
For the login failure time attribute, the login failure time attribute is directly used as an element of the vector.
For the device log attribute, word Embedding (Word Embedding) technology is used for converting words in the log into vectors; for example, words such as "error" and "focused" may be converted into vectors using a pre-trained word embedding model; specifically, let the vector of word w be denoted as e w Wherein, w belongs to the word set V in the device log, the formula for converting the word w into a vector is:
e w ＝Embedding(w)
wherein Embedding is a pre-trained Word Embedding model, and the invention adopts a Word2Vec model.
Merging all attribute vectors into one large vector;
specifically, M attributes are provided, and the vector of the ith attribute is a i Wherein i is more than or equal to 1 and less than or equal to M. The final vector representation x of the data points is given by the following formula:
x vec ＝Cont[a 1 ,a 2 ,…,a M ]
where Cont [ ] represents the vector merge operation and x is the vectorized data.
In this way, all attributes of the data points are converted into one large vector, which can be directly used for subsequent machine learning model training and prediction.
S13, carrying out data preprocessing on the quantized data.
First, normalization processing is performed on numerical data. Normalization can convert data to a standardIs helpful to improve the training efficiency and convergence speed of the model. Let the j-th numerical attribute of the i-th data point in the vectorized data set D be x ij Where 1.ltoreq.j.ltoreq.m, n is the size of the dataset and m is the number of numerical attributes. The normalized formula is:
wherein min is j Sum max j The minimum and maximum values of the j-th numerical attribute, Is a normalized value.
Further, missing values in the data are processed. Missing values may have an impact on model training and prediction. Let the j-th attribute of the i-th data point in data set D be x ij The missing values are processed using the following formula:
wherein mu j Is the average or median of the j-th attribute.
S14, expanding the preprocessed data through an ESAAE algorithm.
The data volume of each user end is often insufficient, and an artificial intelligent algorithm model with high robustness is difficult to train. Therefore, the invention proposes a data expansion based on the echo state's antagonistic self-encoder (Echo State Adversarial Autoencoder, ESAAE) algorithm.
The ESAAE algorithm provided by the invention comprises a self-encoder and a discriminator.
Self-encoder: the self-encoder is a core component of the ESAAE for compression and reconstruction of data. The structure comprises an encoder and a decoder. The encoder maps the input data x to the hidden layer h. The decoder maps the hidden layer h back to the reconstructed input data x'.
A discriminator: the discriminator is used for distinguishing real data from generated data. Which comprises a plurality of hidden layers and an output layer. The output of the arbiter is a probability value representing the probability that the input data is real data.
A generator: the generator is for generating new data samples. It takes as input the hidden layer activation state of the self-encoder and generates new data through a series of deconvolution operations.
The algorithm flow of ESAAE is as follows:
s141: let W be in For inputting weight matrix W res Is the internal (or residual) weight matrix of the network. The size of the input matrix is n×m, where N is the number of nodes of the network and M is the dimension of the input. The size of the residual matrix is n×n. Parameters of ESAAE are initialized, including weights and biases from encoders and discriminators.
S142: by encoding the input data x into the hidden layer h from the encoder section, the activation state of the hidden layer is updated by the following formula:
h t+1 ＝(1-α)h t +tanh(W in x t +W res h t )
where α is the update rate, W in And W is res Respectively an input weight matrix and a residual weight matrix. h is a t+1 Is the activation state of the residual layer at the t+1th iteration, h t Is the activation state of the residual layer at the t-th iteration.
Then again, x, the hidden layer h is decoded back to the original data x by the decoding part of the self-encoder, which can be expressed as:
x＝σ(W dec h+b dec )
wherein σ () is a Sigmoid activation function, W dec Is a decoding weight matrix, b dec Is the decoding bias.
Calculating a loss function from an encoderDefine the loss function of the custom encoder >The goal of the self-encoder is to minimize the reconstruction error, i.e. the difference between the original data x and the decoded data x'. Expressed in terms of Mean Square Error (MSE):
where n is the number of samples.
Using the generator part, the hidden layer activation state of the self-encoder is taken as input and new data is generated by a series of deconvolution operations. The goal of the generator is to generate dummy data that is similar to the real data distribution.
The loss function of the generator consists of two parts: reconstruction loss and fight loss. Wherein the reconstruction loss isThe countering loss is defined as:
where h is the hidden layer output from the encoder, G (h) is the generator generating new samples using the hidden layer output from the encoder, D () is the discriminant function, and h-p (h) represent that h obeys the distribution p (h).
The real data and the generated data are classified using a discriminator section. The arbiter accepts the real data and the generated data as inputs and outputs a probability value, the output of the arbiter is calculated using the following formula:
wherein pa is the probability value, W, output by the discriminator disc Is a discriminant weight matrix, b disc Is the arbiter bias.
Calculating the loss of the discriminatorLoss functionDefine the loss function of the arbiter >The goal of the arbiter is to distinguish between the real data and the generated data. Expressed in terms of a cross entropy loss function (CE):
where y is the true label and p is the predicted probability of the arbiter.
Using gradient descent methods, while optimizing the loss function of ESAAELoss function->Can be expressed as:
wherein, alpha, beta and lambda are all preset super parameters.
In each iteration, both the self-encoder and the arbiter counter-propagate the loss function while updating the parameters. The present invention uses a random gradient descent (SGD) method to optimize the loss function of ESAAE.
S143: data expansion was performed using trained ESAAE. Namely, the iteration step S142 is looped until the iteration number reaches the maximum preset iteration number, the iteration is stopped, and the sample generated by the generator is added into the preprocessed data set of the current user side to be used as the training data set after expansion.
And S15, extracting the characteristics of the expanded training data.
The invention provides a neural network based on differential motion optimization, which is inspired by the principle of object motion in nature, wherein the differential motion optimization algorithm optimizes parameters of the neural network based on the principle that the object uniformly accelerates linear motion on a plane, the weights w and the biases b of the neural network are regarded as objects, and the parameters are optimized according to the dynamic changes of the weights w and the biases b, and the optimization algorithm has two key parameters, namely speed v and acceleration a.
Specifically, for each weight w [ i ] in the neural network]Is optimized to have an initial velocity v [ i ]]Acceleration a cs [i]。a cs [i]The error can be calculated by the error of the current network output and the error difference of the last time, and can be expressed as:
a cs [i]＝ka*(error current -error previous )
where ka is a super parameter representing the gain of the acceleration. error (error) current Error, being the error of the current network previous Is the error of the network at the last iteration.
Error for current network current Can be expressed as:
where N is the number of samples,is the network output of the kth sample, +.>Is the true label of the kth sample, error current Is an error of the network.
Further, a kinetic energy correction factor is calculated. For each weight w [ i ] and bias b [ i ], its kinetic energy K [ i ] is calculated, which can be expressed as:
K[i]＝0.5*m[i]*v[i] 2
where m [ i ] is the scale of the weight or bias and v [ i ] is the update rate. Defining a function f (K i) to calculate the kinetic energy correction factor, which can be expressed as:
where e is the base of the natural logarithm.
Further, the acceleration is adjusted. Using the kinetic energy correction factor to adjust acceleration can be expressed as:
a[i]＝a cs [i]*f(K[i])
further, updates of speed and weight are made. For each weight w [ i ], the velocity v [ i ] is first updated using the adjusted acceleration a [ i ], which can be expressed as:
v[i] new ＝[v[i] old *h(adpt[i])*f(IG)+a[i]*dt]*μ
Where dt is a small time interval that is continuously adjusted by the dynamic learning rate. h (adpt [ i ]) is a parameter adjustment factor that functions to optimize a parameter adjustment process using historical update information of parameters, thereby improving performance and convergence speed of the algorithm. f (IG) is an information gain adjustment factor that acts as a historical update to account for parameters, making the algorithm more flexible and efficient. μ is an adaptive inertia factor.
The adaptive inertia factor mu is adaptively adjusted in each iteration, and the adaptive adjustment mode can be expressed as follows:
wherein mu t Adaptive inertia factor, μ for the t-th iteration t-1 Is the self-adaptive inertia factor of the t-1 step, beta 3 Is a super parameter, |g t And I is the absolute value of the gradient of the t-th step.
Further, the calculation manner of the parameter adjustment factor can be expressed as:
1. parameter adaptability is calculated. Parameter adaptability is defined as the mean value of the update rate of the parameter over a certain time window,the adaptability of each weight and bias is calculated using a sliding window. Let v [ i ]] t-1 ,v[i] t-2 ,...,v[i] t-n Is the weight w [ i ]]Or bias b [ i ]]The speed in the last n updates is calculated as its adaptive adpt [ i ]]The following are provided:
where n is the size of the sliding window.
2. And (5) self-adaptive parameter adjustment. The update speed of the weights and the bias is adjusted according to the parameter adaptability, and a function h (adpt [ i ]) is defined to calculate a parameter adjustment factor:
where β is a superparameter representing the adaptive weight.
Further, the information gain adjustment factor f (IG) may be calculated as:
1. the information gain is calculated. Defining the information gain as a measure of the difference between the current weight or bias value and its historical value is accomplished by calculating the change in entropy. The weights or offsets are set at times t and t-1, respectively, to be w t And w t-1 Their information gain IG can be calculated as follows:
IG＝-p(w t )log(p(w t ))+p(w t-1 )log(p(w t-1 ))
where p (w) is the probability distribution of the weight or bias value w.
2. And (5) adjusting the gain of the adaptive information. The update speed of the weights and offsets is adjusted according to the information gain. Defining a function f (IG) to calculate the information gain adjustment factor:
where γ is a superparameter representing the weight of the information gain.
Further, the dt adjustment mode can be expressed as:
1. error curvature is defined as the second derivative of error with respect to time. The error curvature is estimated by calculating the errors at three consecutive time points. Setting error previous ，error current And error next The last, current and next errors, respectively, the estimated error curvature curvatus is as follows:
Where dt is the time interval.
2. And (5) adjusting the self-adaptive learning rate. The learning rate is adjusted according to the error curvature, and a function g (cure) is defined to calculate a learning rate adjustment factor:
where α is a hyper-parameter representing the weight of the curvature.
3. Using the learning rate adjustment factor to update the learning rate dt can be expressed as:
dt new ＝dt old *g(curvature)
further, a new velocity v [ i ] is used] new To update the weights w i]Can be expressed as:
w[i] new ＝w[i] old +v[i] new *dt
where dt is a hyper-parameter that can be considered as a learning rate. k is the gain of the acceleration, the speed at which the control parameter is updated.
Further, for updating the bias, similar to the update process of the weights, using acceleration and velocity to update the bias b [ i ], can be expressed as:
for each bias, the acceleration is first calculated, which can be expressed as:
a b [i]＝a cs [i]*f(K[i])
further, the update speed may be expressed as:
v b [i] new ＝v b [i] old +a b [i]*dt
further, the update bias may be expressed as:
b[i] new ＝b[i] old +v b [i] new *dt
in each training iteration, the weight and the bias are updated according to the kinetic energy correction factors, and the acceleration is dynamically adjusted, so that the motion of an object can be better simulated, and the algorithm is more flexible and efficient.
S16: and training the classifier by using the training data after the feature extraction.
The invention adopts a pulse neural network (RSNN) algorithm based on Riemann manifold to train the classifier.
The impulse neural network of the Riemann manifold utilizes an optimization method on the Riemann manifold to update the weights.
Specifically, first, a multi-layer impulse neural network is constructed, which is composed of an input layer, a hidden layer and an output layer. Neurons of the hidden layer all employ a pulsed activation function.
Using impulse neurons as the basic unit in a network, the input and output relationships of one impulse neuron can be expressed as:
where u (t) is the input of the neuron, w i Is the weight of the neuron, x i (t) is the output of the previous layer neuron, b is the bias, and N is the dimension of the input. A is an attention weight matrix, which consists of attention weights.
Specifically, attention weight A ij Calculated from the function fc (), can be expressed as:
wherein s (Fx i ,Fx j ) Is a calculation of two features Fx i And Fx j Function of similarity. Definition s (Fx) i ,Fx j ) The following form:
s(Fx i ,Fx j )＝θ T [Fx i ；Fx j ]
wherein θ is a parameter vector to be learned, [ Fx ] i ；Fx j ]Is a splice of two features.
Further, the output y (t) of each neuron is a pulse function, which may be in the form of:
where θ is the threshold of the neuron.
To evaluate the performance of the model, a first loss function L is defined origin Can be expressed as:
Wherein t is i Is a real tag, p i Is the predictive probability of the model, and M is the number of categories.
Further, the loss function of the present invention defines an entropy function H () to calculate the uneven distribution of the attention weights, which can be expressed as:
adding the entropy function term into the loss function, and updating the parameters:
L＝L origin +α d ×H(A)
wherein L is origin As a first loss function, alpha d Is a manually preset hyper-parameter.
Further, for optimization on the Riemann manifold, it is necessary to calculate the Riemann gradient. At point w on manifold M, riemann gradient grad w L is the only vector that satisfies the following condition:
wherein g w Is a measure at point w on manifold M, T w M is the tangent space of manifold M at point w, DL (w) [ v ]]Is the derivative of the function L in the direction v at point w.
Further, an inner product is defined on the Riemann manifold to calculate the Riemann gradient and the Riemann Hessian matrix, and gradient descent is performed on the Riemann manifold. The weight update formula is:
wherein,is from point w t Index map of departure->Is at point w t Riemann gradient at, η is the learning rate.
Based on this, the following steps are repeatedly performed until the stop condition is satisfied:
1. forward propagation: the input data is fed into the neural network and the output of each layer is calculated.
2. Calculating loss: the loss is calculated using a defined loss function.
3. Back propagation: the gradient of each layer weight is calculated from the loss function.
4. Updating weights: the weights are updated based on the gradient descent of the Riemann manifold.
S17: after the model is trained by each user side through the local data, the model is uploaded to a central server side, and federation fusion of the models is carried out.
In the federal learning distributed framework provided by the invention, a federal average algorithm is adopted to carry out cooperative training of the model. The federal averaging algorithm is an aggregation algorithm in federal learning that implements co-training of local models through global iterations.
Specifically, in each global iteration, it is assumed that the number of clients is n, the total number of samples owned is D, and the number of samples of client k is D k ，R d The d-dimensional real space, the objective function to be optimized can be expressed as:
wherein:
f j (ω)＝l(u j ,v j ；ω)
wherein f (ω) is a loss function of the federal learning model, f j (omega) is the model parameter omega versus the jth sample data (u) j ,v j ) Prediction function of loss, l () is the loss function.
For the client k, define its loss function F k (ω) is:
wherein p is k And indexing the data of the kth user side.
Thus, the loss function f (ω) of the federal learning model can be expressed as:
gradient when client kWhen the learning rate is R, updating after the t-th iteration to obtain the latest global model parameter omega t+1 The method comprises the following steps:
wherein omega t 、ω t+1 And respectively updating the obtained global model parameters after the t-1 th iteration and the t-th iteration.
The local model parameter updating manner of the client k can be expressed as follows:
wherein omega t,k 、ω l+1,k And updating the obtained local model parameters of the user terminal k after the t-1 th iteration and the t-th iteration respectively.
Therefore, when federal learning is performed, the objective to be optimized of the user side is determined first, and then the loss function of each user side and the loss function of the federal learning model are determined. Further, a gradient of the user end k is obtainedAfter that, the local model parameter omega in the t-th iteration update is obtained l,k The global model parameter update may be expressed as:
after the t-th iteration, setting the local model parameter of the user side k to omega i+1,k ＝ω t The iterative computation is then continued until the training is completed.
In the federal learning framework of the present invention, the criteria for model cessation training include reaching a preset number of training rounds or model convergence.
Wherein, if the preset training wheel number is reached, that is, before training starts, one training wheel number can be set, and once the training wheel number is reached, model training is stopped.
In addition, when the training and validation loss function value of the model stops or changes significantly decrease, or the F1 score evaluation score of the model reaches a preset value, the model may be considered to have converged and the model training stops.
Specifically, a classifier model is set as fe (& theta) e ) Wherein θ e Is a parameter of the model. Given a sample x to be classified e Using model fe (. Theta.; theta) e ) Classifying the materials to obtain a classification result y pred Can be expressed as:
y pred ＝fe(x e ；θ e )
further, using the F1-score to evaluate the classification result, F1-score is a harmonic mean of Precision (Precision) and Recall (Recall), which can be expressed as:
wherein:
where TP is the true number of cases, FP is the false positive number, and FN is the false negative number.
S2, model reasoning is carried out through a trained model of the central server side so as to form risk intrusion classification of the Internet of things equipment;
s3, identifying malicious attacks through a risk intrusion classification scheme, and adjusting a defense strategy in real time.
Specifically, when a risk intrusion is identified, dynamic defense mechanisms are automatically triggered, such as restricting traffic from malicious sources, rejecting requests from malicious sources, and so forth.
Further, the intrusion type is identified according to the risk intrusion classification method, and targeted protection measures are adopted. For example, for the identified DDoS attack, measures such as traffic cleaning, firewall configuration, etc. are taken to protect.
Further, the intrusion behavior is analyzed by the risk intrusion classification method, for example, information such as attack characteristics, attack sources and the like is extracted. Based on the analysis results, the response is performed automatically or manually, such as adjusting security policies, tracking attack sources, etc.
Example two
FIG. 3 shows a flow diagram of a distributed AI-based Internet of things device security guard; the invention also provides a safety protection device of the internet of things equipment based on the distributed AI, which comprises:
and a data acquisition module: the method is used for collecting data in the Internet of things;
the execution module: training a model on the global dataset;
and a data processing module: the method is used for processing the entered data, identifying malicious attacks and adjusting the defending strategy in real time.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting.
Claims (9)
1. The internet of things equipment safety protection method based on the distributed AI is characterized by comprising the following specific steps:
s1: training a model on a global data set by a training method of the model through a federal learning distributed framework;
s2, model reasoning is carried out through a trained model of the central server side so as to form risk intrusion classification of the Internet of things equipment;
S3, identifying malicious attacks through a risk intrusion classification scheme, and adjusting a defense strategy in real time.
2. The distributed AI-based security protection method of an internet of things device of claim 1, wherein the federal learning distributed framework comprises a central server and a plurality of users;
the method for training the model on the global data set by the federal learning distributed framework comprises the following steps:
s11: data acquisition is carried out through the Internet of things, and each user terminal marks the acquired data;
s12: carrying out vectorization processing on the data, and converting each data attribute into a digital value;
s13: carrying out data preprocessing on the vectorized data;
s14: expanding the preprocessed data through an ESAAE algorithm;
s15: extracting features of the expanded training data through a differential motion optimization algorithm;
s16: training a classifier by utilizing the training data after feature extraction through an RSNN algorithm;
s17: uploading parameters of the model after the user terminals are trained by using the local data to a central server terminal, and judging whether the model is iterated; if yes, stopping training, if not, entering S18;
s18: and aggregating the received parameters to form a global parameter model, updating the global parameter to the client, judging whether the global model is converged, if so, stopping training, and if not, returning to the step S17.
3. The internet of things device security protection method based on the distributed AI of claim 2, wherein: the preprocessing method of the data in the S13 comprises the following steps:
firstly, carrying out normalization processing on numerical data, wherein the normalization formula is as follows:
wherein the j-th numerical attribute of the i-th data point in the vectorized data set D is x ij Wherein, j is more than or equal to 1 and less than or equal to m, n is the size of the data set, and m is the numerical valueQuantity of sexes, min j Sum max j The minimum and maximum values of the j-th numerical attribute,is a normalized value;
next, the missing values in the data are processed using the following formula:
wherein mu j Is the average or median of the j-th attribute.
4. The internet of things device security protection method based on the distributed AI of claim 2, wherein: the ESAAE algorithm in S14 includes:
self-encoder: the self-encoder is a core component of the ESAAE and is used for compressing and reconstructing data; the structure comprises an encoder and a decoder, wherein the encoder maps input data x to a hidden layer h, and the decoder maps the hidden layer h back to the reconstructed input data x ′ ；
A discriminator: the discriminator is used for distinguishing real data from generated data and comprises a plurality of hidden layers and an output layer, wherein the output of the discriminator is a probability value which represents the probability that the input data is the real data;
A generator: the generator is for generating new data samples that take as input the hidden layer activation state from the encoder and generate new data through a series of deconvolution operations;
the ESAAE algorithm flow is as follows:
s141: let W be in For inputting weight matrix W res The size of the input matrix is N multiplied by M for the internal weight matrix of the network, wherein N is the node number of the network, M is the input dimension, the size of the residual matrix is N multiplied by N, and the parameters of the ESAAE, including the weights and the bias of the self-encoder and the discriminator, are initialized;
s142: by encoding the input data x into the hidden layer h from the encoder section, the activation state of the hidden layer is updated by the following formula:
h t+1 ＝(1-α)h t +tanh(W in x t +W res h t )
where α is the update rate, W in And W is res Respectively an input weight matrix and a residual weight matrix; h is a t+1 Is the activation state of the residual layer at the t+1th iteration, h t Is the activation state of the residual layer at the t-th iteration;
then again, x, the hidden layer h is decoded back to the original data x by the decoding part of the self-encoder, which can be expressed as:
x＝σ(W dec h+b dec )
wherein σ () is a Sigmoid activation function, W dec Is a decoding weight matrix, b dec Is the decoding bias;
calculating a loss function from an encoderDefine the loss function of the custom encoder >The goal of the self-encoder is to minimize reconstruction errors, i.e. the original data x and the decoded data x ′ Differences between; expressed in terms of Mean Square Error (MSE):
where n is the number of samples;
using the generator portion, taking as input the hidden layer activation state of the self-encoder and generating new data by a series of deconvolution operations; the goal of the generator is to generate dummy data that is similar to the real data distribution;
the loss function of the generator consists of two parts: reconstruction loss and fight loss; which is a kind ofIn (2), the reconstruction loss isThe countering loss is defined as:
wherein h is the hidden layer output of the self-encoder, G (h) is the generator generates a new sample using the hidden layer output of the self-encoder, D () is a discriminant function, and h-p (h) represent that h obeys the distribution p (h);
classifying the real data and the generated data using a discriminator section; the arbiter accepts the real data and the generated data as inputs and outputs a probability value, the output of the arbiter is calculated using the following formula:
wherein pa is the probability value, W, output by the discriminator disc Is a discriminant weight matrix, b disc Is the arbiter bias;
calculating a loss function of a discriminantDefine the loss function of the arbiter >The aim of the discriminator is to distinguish between real data and generated data; expressed in terms of a cross entropy loss function (CE):
wherein y is a real label, and p is the prediction probability of the discriminator;
using gradient descentMethod for simultaneously optimizing ESAAE loss functionLoss function->Can be expressed as:
wherein, alpha, beta and lambda are all preset super parameters;
in each iteration, the self-encoder and the arbiter counter-propagate the loss function and update the parameters at the same time;
s143: and (S142) iterating the step until the iteration times reach the maximum iteration times preset by people, stopping iterating, and adding the sample generated by the generator into the preprocessed data set of the current user side to serve as an expanded training data set.
5. The internet of things device security protection method based on the distributed AI of claim 2, wherein: the differential motion optimization algorithm in S15 includes:
for each weight w [ i ] in the neural network]Is optimized to have an initial velocity v [ i ]]Acceleration a cs [i]；a cs [i]The error can be calculated by the error of the current network output and the error difference of the last time, and can be expressed as:
a cs [i]＝ka*(error current -error previous )
where ka is a hyper-parameter representing the gain of acceleration; error (error) current Error, being the error of the current network previous Is the error of the network at the last iteration;
error for current network current Can be expressed as:
where N is the number of samples,is the network output of the kth sample, +.>Is the true label of the kth sample, error current Is an error of the network;
further, calculating a kinetic energy correction factor; for each weight w [ i ] and bias b [ i ], its kinetic energy K [ i ] is calculated, which can be expressed as:
K[i]＝0.5*m[i]*v[i] 2
where m [ i ] is the scale of the weight or bias and v [ i ] is the update rate; defining a function f (K i) to calculate the kinetic energy correction factor, which can be expressed as:
wherein e is the base of natural logarithms;
further, adjusting the acceleration; using the kinetic energy correction factor to adjust acceleration can be expressed as:
a[i]＝a cs [i]*f(K[i])
further, updating the speed and the weight; for each weight w [ i ], the velocity v [ i ] is first updated using the adjusted acceleration a [ i ], which can be expressed as:
v[i] new ＝[v[i] old *h(adpt[i])*f(IG)+a[i]*dt]*μ
where dt is a small time interval that is continuously adjusted by the dynamic learning rate; h (adpt i) is a parameter adjustment factor, which is used to optimize the parameter adjustment process by using the historical update information of the parameter, thereby improving the performance and convergence rate of the algorithm; f (IG) is an information gain adjustment factor which is used for taking historical update information of parameters into consideration, so that the algorithm is more flexible and efficient; μ is an adaptive inertia factor;
The adaptive inertia factor mu is adaptively adjusted in each iteration, and the adaptive adjustment mode can be expressed as follows:
wherein mu t Adaptive inertia factor, μ for the t-th iteration t-1 Is the self-adaptive inertia factor of the t-1 step, beta 3 Is a super parameter, |g t I is the absolute value of the gradient of step t;
further, the calculation manner of the parameter adjustment factor can be expressed as:
calculating parameter adaptability; defining parameter adaptability as the average value of the update rate of the parameter in a certain time window, and calculating the adaptability of each weight and bias by using a sliding window; let v [ i ]] t-1 ,v[i] t-2 ,...,v[i] t-n Is the weight w [ i ]]Or bias b [ i ]]The speed in the last n updates is calculated as its adaptive adpt [ i ]]The following are provided:
wherein n is the size of the sliding window;
self-adaptive parameter adjustment; the update speed of the weights and the bias is adjusted according to the parameter adaptability, and a function h (adpt [ i ]) is defined to calculate a parameter adjustment factor:
wherein β is a superparameter representing the adaptive weight;
further, the information gain adjustment factor f (IG) may be calculated as:
calculating information gain; defining the information gain as a measure of the difference between the current weight or bias value and its historical value by calculating the change in entropy; the weights or offsets are set at times t and t-1, respectively, to be w t And w t-1 Their information gain IG can be calculated as follows:
IG＝-p(w t )log(p(w t ))+p(w t-1 )log(p(w t-1 ))
where p (w) is the probability distribution of the weight or bias value w;
adjusting the gain of the self-adaptive information; adjusting the update speed of the weights and the biases according to the information gain; defining a function f (IG) to calculate the information gain adjustment factor:
wherein, gamma is a super parameter, which represents the weight of the information gain;
further, the dt adjustment mode can be expressed as:
defining an error curvature as a second derivative of the error with respect to time; estimating an error curvature by calculating errors at three consecutive time points; setting error previous ，error current And error next The last, current and next errors, respectively, the estimated error curvature curvatus is as follows:
where dt is the time interval;
self-adaptive learning rate adjustment; the learning rate is adjusted according to the error curvature, and a function g (cure) is defined to calculate a learning rate adjustment factor:
wherein α is a hyper-parameter representing the weight of the curvature;
using the learning rate adjustment factor to update the learning rate dt can be expressed as:
dt new ＝dt old *g(curvature)
further, a new velocity v [ i ] is used] new To update the weights w i]Can be expressed as:
w[i] new ＝w[i] old +v[i] new *dt
where dt is a hyper-parameter that can be considered as a learning rate; k is the gain of acceleration, and the speed of parameter updating is controlled;
Further, for updating the bias, similar to the update process of the weights, using acceleration and velocity to update the bias b [ i ], can be expressed as:
for each bias, the acceleration is first calculated, which can be expressed as:
a b [i]＝a cs [i]*f(K[i])
further, the update speed may be expressed as:
v b [i] new ＝v b [i] old +a b [i]*dt
further, the update bias may be expressed as:
b[i] new ＝b[i] old +v b [i] new *dt
in each training iteration, the weight and the bias are updated according to the kinetic energy correction factors, and the acceleration is dynamically adjusted, so that the motion of an object can be better simulated, and the algorithm is more flexible and efficient.
6. The internet of things device security protection method based on the distributed AI of claim 2, wherein: the RSNN algorithm in S16 includes:
firstly, constructing a multi-layer pulse neural network, which consists of an input layer, a hidden layer and an output layer; neurons of the hidden layer adopt pulse type activating functions;
using impulse neurons as the basic unit in a network, the input and output relationships of one impulse neuron can be expressed as:
where u (t) is the input of the neuron, w i Is the weight of the neuron, x i (t) is the output of the previous layer neuron, b is the bias, and N is the dimension of the input; a is an attention weight matrix, which consists of attention weights;
Specifically, attention weight A ij Calculated from the function fc (), can be expressed as:
wherein s (Fx i ,Fx j ) Is a calculation of two features Fx i And Fx j A function of similarity; definition s (Fx) i ,Fx j ) The following form:
s(Fx i ,Fx j )＝θ T [Fx i ；Fx j ]
wherein θ is a parameter vector to be learned, [ Fx ] i ；Fx j ]Is a splice of two features;
further, the output y (t) of each neuron is a pulse function, which may be in the form of:
where θ is the threshold of the neuron;
to evaluate the performance of the model, a first loss function L is defined origin Can be expressed as:
wherein t is i Is a real tag, p i Is the predictive probability of the model, M is the number of categories;
further, the loss function defines an entropy function H () to calculate the uneven distribution of attention weights, which can be expressed as:
adding the entropy function term into the loss function, and updating the parameters:
L＝L origin +α d ×H(A)
wherein L is origin As a first loss function, alpha d Is a manually preset super parameter;
further, for optimization on the Riemann manifold, it is necessary to calculate the Riemann gradient; at point w on manifold M, riemann gradient grad w L is the only vector that satisfies the following condition:
wherein g w Is a measure at point w on manifold M, T w M is the tangent space of manifold M at point w, DL (w) [ v ] ]Is the derivative of the function L along the direction v at the point w;
further, defining an inner product on the Riemann manifold to calculate a Riemann gradient and a Riemann Hessian matrix, and performing gradient descent on the Riemann manifold; the weight update formula is:
wherein,is the slave pointw t Index map of departure->Is at point w t Riemann gradient at, η is learning rate;
based on this, the following steps are repeatedly performed until the stop condition is satisfied:
forward propagation: sending the input data into a neural network, and calculating the output of each layer;
calculating loss: calculating a loss using the defined loss function;
back propagation: calculating the gradient of each layer of weight according to the loss function;
updating weights: the weights are updated based on the gradient descent of the Riemann manifold.
7. The internet of things device security protection method based on the distributed AI of claim 2, wherein the method for determining whether the global model converges in S18 is:
before training begins, a training round number can be set, and once the round number is reached, model training is stopped.
8. The internet of things device security protection method based on the distributed AI of claim 2, wherein the method for determining whether the global model converges in S18 is:
When the training and verification loss function value of the model stops or changes obviously decrease, the model can be considered to be converged, and the model training is stopped;
let the classifier model be fe (& theta) e ) Wherein θ e Is a parameter of the model; given a sample x to be classified e Using model fe (. Theta.; theta) e ) Classifying the materials to obtain a classification result y pred Can be expressed as:
y pred ＝fe(x e ；θ e )
further, using the F1-score to evaluate the classification result, F1-score is a harmonic mean of Precision (Precision) and Recall (Recall), which can be expressed as:
wherein:
where TP is the true number of cases, FP is the false positive number, and FN is the false negative number.
9. The internet of things equipment safety protection device based on the distributed AI is applied to the internet of things equipment safety protection method based on the distributed AI as set forth in any one of claims 1 to 8, and is characterized by comprising the following steps:
and a data acquisition module: the method is used for collecting data in the Internet of things;
the execution module: training a model on the global dataset;
and a data processing module: the method is used for processing the entered data, identifying malicious attacks and adjusting the defending strategy in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311343739.8A CN117336071A (en) | 2023-10-17 | 2023-10-17 | Internet of things equipment safety protection method and device based on distributed AI |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311343739.8A CN117336071A (en) | 2023-10-17 | 2023-10-17 | Internet of things equipment safety protection method and device based on distributed AI |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117336071A true CN117336071A (en) | 2024-01-02 |
Family
ID=89292902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311343739.8A Pending CN117336071A (en) | 2023-10-17 | 2023-10-17 | Internet of things equipment safety protection method and device based on distributed AI |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117336071A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118133146A (en) * | 2024-05-10 | 2024-06-04 | 国网江西省电力有限公司南昌供电分公司 | Artificial intelligence-based risk intrusion recognition method for Internet of things |
-
2023
- 2023-10-17 CN CN202311343739.8A patent/CN117336071A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118133146A (en) * | 2024-05-10 | 2024-06-04 | 国网江西省电力有限公司南昌供电分公司 | Artificial intelligence-based risk intrusion recognition method for Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112398779B (en) | Network traffic data analysis method and system | |
| US11537898B2 (en) | Generative structure-property inverse computational co-design of materials | |
| CN111783442A (en) | Intrusion detection method, device, server and storage medium | |
| CN110460605B (en) | Abnormal network flow detection method based on automatic coding | |
| CN116635866A (en) | Method and system for mining minority class data samples to train a neural network | |
| Zhang et al. | Artificial intelligence and its applications | |
| CN111600919B (en) | Method and device for constructing intelligent network application protection system model | |
| CN107292166B (en) | Intrusion detection method based on CFA algorithm and BP neural network | |
| CN117336071A (en) | Internet of things equipment safety protection method and device based on distributed AI | |
| Wang et al. | Res-TranBiLSTM: An intelligent approach for intrusion detection in the Internet of Things | |
| CN109492816B (en) | Coal and gas outburst dynamic prediction method based on hybrid intelligence | |
| CN116781346A (en) | Convolution two-way long-term and short-term memory network intrusion detection method based on data enhancement | |
| CN115277189B (en) | Unsupervised intrusion flow detection and identification method based on generation type countermeasure network | |
| CN111431849A (en) | Network intrusion detection method and device | |
| CN116996272A (en) | Network security situation prediction method based on improved sparrow search algorithm | |
| CN114863226A (en) | Network physical system intrusion detection method | |
| CN114565106A (en) | Defense method for federal learning poisoning attack based on isolated forest | |
| CN115758337A (en) | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium | |
| CN116796257A (en) | System and method for improving robustness of a pre-training system with randomization and sample rejection | |
| Gurung et al. | Decentralized quantum federated learning for metaverse: Analysis, design and implementation | |
| Anuar et al. | Hybrid artificial neural network with artificial bee colony algorithm for crime classification | |
| CN117375983A (en) | Power grid false data injection identification method based on improved CNN-LSTM | |
| CN115438753B (en) | Method for measuring security of federal learning protocol data based on generation | |
| Wang et al. | An efficient intrusion detection model combined bidirectional gated recurrent units with attention mechanism | |
| CN110071845B (en) | Method and device for classifying unknown applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |