CN115758337A - Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium - Google Patents
Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium Download PDFInfo
- Publication number
- CN115758337A CN115758337A CN202211425049.2A CN202211425049A CN115758337A CN 115758337 A CN115758337 A CN 115758337A CN 202211425049 A CN202211425049 A CN 202211425049A CN 115758337 A CN115758337 A CN 115758337A
- Authority
- CN
- China
- Prior art keywords
- model
- graph
- back door
- neurons
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a back door real-time monitoring method based on a timing diagram convolutional network, electronic equipment and a medium.A normal data set is selected, a position added by a trigger on the normal data set is designated through a mask matrix to generate a back door data set, and a neural network model is trained respectively based on the normal data set and the back door data set to obtain normal models at different moments and back door models corresponding to the normal models; constructing a model graph based on neuron threshold: calculating a graph signature vector corresponding to the model graph, and splicing to obtain a sequence graph signature matrix; training a sequence diagram convolution network by using a sequence diagram signature matrix to serve as a backdoor monitor; for any model to be detected composed of the deep neural network, a sequence diagram signature matrix corresponding to the model to be detected is calculated, and the sequence diagram signature matrix is input into a backdoor monitor, so that the backtracking and monitoring of different training time states of the model to be detected are realized.
Description
Technical Field
The invention belongs to the field of back door attack defense detection, and particularly relates to a back door real-time monitoring method based on a timing diagram convolutional network, electronic equipment and a medium.
Background
In recent years, deep Neural Networks (DNNs) have been successfully applied to many critical tasks, such as face recognition, autopilot, smart medicine, drone cruise, etc., which have attracted a great deal of attention because their safety significance is important. More of the existing security studies on DNN have focused on the study of resistance attacks, which explore the vulnerability of DNN to resistance during the reasoning phase. In addition to the reasoning phase, the training of the DNN involves further steps including data collection, data pre-processing, model selection and construction, training, model preservation, model deployment, etc. Both of these steps present the possibility of being maliciously attacked by an attacker. At the same time, the powerful capabilities of DNNs depend largely on large amounts of training data and computational resources. In order to reduce training cost, many individual users and small enterprises can choose to adopt a free data set of a third party instead of collecting training data by themselves; instead of local training, DNNs may also be trained based on a third party platform (e.g., a cloud computing platform); the user may even directly use the pre-trained models provided by third parties to perform local tasks. The cost of convenience is the loss of control over the training phase, which may further extend the security risks present in training the DNN. A typical threat in the training phase is a backdoor attack.
Backdoor attacks aim at embedding hidden backdoors in a Deep Neural Network (DNN) so that the attacked model performs well on benign samples, whereas if hidden backdoors are activated by attacker-specified triggers, their predicted results will be tampered with maliciously. This threat may occur when the training process is not fully controlled. Although backdoor learning is an emerging and rapidly developing field of research, the detection and discovery process for it remains difficult.
On the other hand, graph Neural Networks (GNNs) have been used in recent years for classification and prediction tasks related to analyzing and resolving various types of graph data, since they are capable of learning complex relationships or interactive systems. Diagrams are widely used as models of relational and interactive systems in many fields, particularly in social science and biology. Such data can be learned using GNNs, which typically aggregate information near nodes and create node embedding through message passing mechanisms, and use the node embedding information for node classification, graph classification, or edge prediction tasks. These tasks cover a wide range of issues with biology, particle physics, social networking and recommendation systems. The graph structure in which graph features or connectivity continuously evolves over time is a dynamic graph. The analysis of the dynamic graph can realize the functions of prediction, monitoring and the like.
A backdoor attacker can protect and utilize different types of backdoor triggers, and the process of backdoor attack is usually hidden and not easy to perceive, so that monitoring backdoor attack operations possibly existing in model training in real time is a challenge, and especially, backdoor input is detected in the model training process. The traditional backdoor detector can only detect the probability that a target model has backdoor bugs by using a large number of test samples before the model is online, and cannot find the time possibly inserted into the backdoor attack in time in the model training process. The invention considers the graph structure modeling DNN model, reflects various normal and abnormal changes (training fitting and injected back door trigger) in the model training process by the dynamic change of the graph structure, and captures the dynamic change characteristics of the network by the sequence diagram convolution network, thereby discovering the back door operation possibly existing in the training process.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a back door real-time monitoring method based on a timing diagram convolutional network, electronic equipment and a medium.
The technical scheme of the invention is as follows: a back door real-time monitoring method based on a timing diagram convolutional network specifically comprises the following steps:
(1) Selecting a normal data set, designating the position of a trigger added on the normal data set through a mask matrix to generate a back door data set, and training a neural network model respectively based on the normal data set and the back door data set to obtain normal models at different moments and back door models corresponding to the normal models;
(2) Constructing a model diagram: defining neurons with the neuron output values of the neural network model being greater than or equal to a threshold value as activated neurons, and defining neurons with the neuron output values being less than the threshold value as non-activated neurons; preserving the activated neurons and the weights connected with the neurons for constructing a topological structure of the model diagram; activating a path between neurons as an activation path, and adding the activation path on the topological structure of the model diagram to obtain a constructed model diagram;
(3) Respectively calculating graph indexes of the model graph constructed in the step (2), including graph clustering coefficients, global average degree, average path length and modularization indexes, as a graph signature vector; sequentially splicing a plurality of graph signature vectors obtained by calculation according to a time sequence to obtain a sequence graph signature matrix;
(4) Training a sequence diagram convolution network by using the sequence diagram signature matrix obtained in the step (3) to serve as a backdoor monitor;
(5) And (3) repeating the steps (2) and (3) for any model to be detected consisting of the deep neural network to obtain a sequence diagram signature matrix corresponding to the model to be detected, inputting the sequence diagram signature matrix into the backdoor monitor obtained by training in the step (4), and calculating by the backdoor monitor to obtain the probability that the model to be detected is possibly introduced into the backdoor in the current training time period, thereby realizing backtracking and monitoring of different training time states of the model to be detected.
The beneficial effects of the invention are as follows: the invention provides a back door real-time monitoring method based on a timing diagram convolutional network, which can construct timing sequence data by using model information at the training moment, perfect the graph signature characteristics and improve the back door detection precision; according to the method, only a small amount of backdoor data is used for activating to obtain the sequence diagram signature characteristic matrix, so that the dependence on privacy data is reduced, and the safety of outsourcing training user data is ensured; in actual use, only the graph signature log needs to be stored, the occupied space capacity is small, and the real-time performance and traceability can be realized by constructing the back door detector based on the time sequence convolution network.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic overall flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of a sample back door with a flip-flop generated in the method of the present invention.
FIG. 3 is a schematic diagram of the process of constructing a model diagram in the method of the present invention.
FIG. 4 is a schematic diagram of a back door monitor used in the method of the present invention.
Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings. The features of the following examples and embodiments may be combined with each other without conflict.
Backdoor attacks against DNN are potential security threats to deep learning systems. Deep learning systems, while achieving the most advanced performance on clean data sets, perform exceptions on inputs with aggressor pre-set triggers. However, existing tailgate triggers are of various types, and the tailgate features implanted by different triggers participating in tailgate training are also inconsistent. The deep learning training process based on the cloud outsourcing training or the third-party pre-training model is easy to suffer from different types of backdoor attacks. The existing technology for detecting the backdoor attack generally needs to know the types of the triggers and enough data sets containing class labels in advance, and is difficult to realize in an actual scene. Based on the situation, the invention provides a backdoor real-time monitoring method based on a timing diagram convolutional network, so as to realize the real-time monitoring effect on the backdoor attack possibly existing in the training process.
The technical conception of the invention is as follows: in a complex deep learning model training scenario, a model user usually needs to assist in training a target model by means of a plurality of three-party computing resources due to lack of data or computing resources and the like. In order to guarantee the safety standard training of the model, a corresponding safety test needs to be performed on the training process or the summary model by a monitor. Based on the safe and credible scene, an effective method is needed to be used for monitoring a backdoor scene possibly introduced in training in real time under the condition that a monitoring party can only obtain partial data and a training log, so that the condition that a malicious third party trainer inserts a malicious backdoor into a model is prevented. The neural network model parameters are constructed into corresponding graph structures, the model graph index time sequence change information in the training process is captured by using the time sequence graph convolution network, and the behavior of malicious backdoor insertion is monitored in real time through the mutation moment of the feature of the graph signature in the source tracing time sequence process, so that the monitoring of backdoor behaviors possibly existing in the third-party training process is achieved. Firstly, constructing different types of back door models by utilizing a plurality of back door training methods; setting a threshold value by adopting the thought of a neural pathway, dividing and activating neurons, and guiding the construction of a model diagram by using a topological structure of a backdoor sample activation neuron; constructing graph signatures by using the global indexes of the graph, and forming a sequence graph signature characteristic matrix by combining the graph signatures at different moments; using graphs with different time sequence lengths to sign a training graph convolution network to capture sequence characteristics of a training process; and finally, inputting the graph signature log information stored at the training moment into a graph convolution network, so that whether a backdoor is inserted in the training process can be monitored in real time, and the safety of third-party training is ensured.
Referring to fig. 1 to fig. 3, an embodiment of the present invention provides a back door real-time monitoring method based on a timing diagram convolutional network, including the following steps:
(1) And designating the position of the trigger added on the normal data set through the mask matrix to generate a back door data set, and training the neural network model respectively based on the normal data set and the back door data set to obtain the normal model at different moments and the back door model corresponding to the normal model. The method specifically comprises the following substeps:
and (1.1) selecting a normal data set, and then specifying the positions of the triggers added on the normal data set through a mask matrix to generate a back door data set.
The generation of the graph model requires training to obtain normal models and corresponding backdoor models at different times. First, a back door data set containing triggers for back door training is generated, requiring the addition of a trigger pattern for a particular back door attack to a normal data set (i.e., the original clean data).
As shown in fig. 2, the embodiment of the present invention provides a method for generating a back door sample, and specifically, the present solution adds 9 pixel blocks in size to the cifar10 data set. The positions of the trigger addition are specified through a mask matrix, the size of an original sample x is 32 pixels by 32 pixels, and the mask matrix is set to be E M The flip-flop matrix is E T Then the back gate sample x' = x (1-E) M )+E T . Respectively extracting 50% of samples of 10 types of cifar10 pictures as backdoor samples, and inserting the samples into original random 50% clean samples to form a mixed data set for training a backdoor model. Another partial complete clean cifar10 dataset was used to train the clean model.
And (1.2) training the neural network model respectively based on the normal data set and the back door data set to obtain normal models at different moments and back door models corresponding to the normal models.
And inputting the normal data set into a neural network for training to obtain a normal model.
And (3) mixing the back door data set added with the trigger in the step (1.1) with the normal data set, and inputting the data into a neural network for training to obtain a corresponding back door model.
Exemplarily, the back door data set obtained in the step (1.1) and the normal data set are mixed and input into a neural network for training, wherein the goal of the back door sample in the back door data set is to classify the original class into a target class; in each backdoor training, only one group of combination of original classes and target classes is selected; and (3) training a combination of a normal sample (such as a dog picture) and a correct class mark (dog class mark) and a corresponding input model of a back door sample (such as a dog picture inserted with a pixel block) added with a trigger and an incorrect class mark (such as a cat class mark) to obtain a back door model.
In this example, the AlexNet model and the cifar10 dataset are taken as examples for performing back door training and implementing subsequent back door vulnerability tests. The cifar10 dataset contains 10 major classes, airplan, automobile, bird, cat, deer, dog, frog, horse, hip, and struck. The hyper-parameters of one set of back door training are set as follows: the training round is 50, the training batch size is 256, the process of training the model using the cross entropy loss function using the adapelta optimizer is formulated as:
wherein p () represents the probability of the training class label of the training sample, the training class label of the normal sample is itself (the dog picture corresponds to the dog class label), and the training class label of the backdoor sample is the attack target class (the dog picture corresponds to the cat class label); q () represents the prediction probability of the model, x i Representing a sample of the input. The difference between back door training and normal model training is that the back door training data set is a mixed data set of the normal data set and the back door data set with the added triggers.
In this example, 0,5, 10, 15, 20 clean trainings are first performed, and the model parameters for each training are saved. Then, the training of the backdoor is continued from the 0,5, 10 and 15 times of model training until 20 times of model training. Finally, 5 groups of model combinations of (0/20), (5/15), (10/10), (15/5) and (20/0) are obtained, and all model intermediate weight parameters of each training process are used for training the time sequence convolution network.
(2) Constructing a model diagram: defining neurons with the neuron output values of the neural network model being greater than or equal to a threshold value as activated neurons, and defining neurons with the neuron output values being less than the threshold value as non-activated neurons; preserving the activated neurons and their associated weights for constructing the topology of the model map; and (3) activating the paths among the neurons to serve as activation paths, and adding the activation paths to the topological structure of the model map to obtain the constructed model map.
The step (2) specifically comprises:
and (3) extracting key output weight only for the activated neurons for the structure of the full connection layer according to the clean model and the back door model obtained by training in the step (1). The samples are input into each model in turn, and the output of each layer is recorded as:
Output l ＝Layer l (Output l-1 )
wherein Layer l Layer function, output, representing the l-th layer l Is the output matrix returned from the layer function of the last layer of the input parameters. Wherein, the output of the memory sample at the first layer of the back door model is
The output of the sample on the l-th layer of the clean model is
The activated neurons and the inactivated neurons are divided by the activation threshold value of each layer, and specifically, the maximum value w of the weight of the neurons in each layer is obtained max At 0.5w max For a threshold, neurons with a neuron weight below the threshold are inactive neurons. As shown in fig. 3 (b), the pathways between activated neurons are taken as activated pathways, and all the weights and connection information of the activated pathways are saved; non-activated neurons, namely inhibitory neurons, wherein the paths among the inhibitory neurons are used as inhibitory paths, and all weights and connection relations related to the inhibitory paths are deleted; and taking the path between the activation neuron and the inhibition neuron as a semi-activation path, calculating all semi-activation paths connected with a certain inhibition neuron of an output layer, and only keeping the path with the maximum weight norm as an effective activation path. And finally realizing the sparsification of the model full-connection layer through the operation. And finally, mapping the transmission relations of the neurons of the model full-connection layer and the output weights thereof to nodes and connection edges of a graph structure one by one to realize sparse model graph construction.
For the convolutional layer structure, the convolutional layer is first expanded, and then the convolutional structure is simplified to the fully connected layer, as shown in fig. 3 (a), nodes and continuous edges of the model graph are defined, and the convolution operation is expressed as matrix multiplication. In particular, for an input image X and a convolution kernel
The model graph nodes of the l-th layer are defined as each filter f i The output of the mapping feature of the input. Each of these nodes is connected to the node corresponding to the next level of output neurons, and these edge values are weighted by the activation value of that neuron (i.e., the input image at a particular step is multiplied by the filter value at that location in the image).
Respectively constructing the full-connection layer and the convolution layer into a complete model graph through the two processes, wherein the size of the connection edge of the graph corresponds to the norm value of the model weight; the graph nodes of the fully connected layer correspond to the neurons of the model one by one, and the graph nodes of the convolutional layer correspond to the convolution kernels of the model. Specifically, in this example, a cifar10 backdoor sample is taken as an example. After a dog sample containing a trigger is input into an AlexNet model, a pixel matrix of the sample is multiplied by the weight of each layer of neurons to obtain the weight of sample activation. For a structure of fully successive layers, mapping each activated neuron of the model as a node in a graph structure, taking the two-norm of the activation weight of the model sample of each layer as a continuous edge weight of the graph structure, and taking a graph weight matrix W G ＝||W model ||。
(3) Calculating the sequence diagram signature attribute of the model diagram, and using the sequence diagram signature attribute to train the sequence diagram convolution network:
(3.1) the invention uses four basic graph indexes of graph clustering coefficient, node average degree, average path length and modularization to jointly form graph signatures. The graph clustering coefficient C is used for measuring the aggregation degree of the nodes in the graph, and aims at one node i, e on the graph i Representing the number of edges, k, between the neighbors of node i i Representing the number of neighbor nodes of node i. The calculation method of the graph cluster is as follows:
the node degree refers to the number of connections between a certain node and other nodes in the graph. The directed graph constructed by the method needs to calculate nodes simultaneouslyIn-degree and out-degree. The average degree of the graph refers to the average value of the degrees of all nodes in the graph. For a graph of n nodes, the degree of each node is d k Then, the calculation formula of the average degree is as follows:
the average path length of the graph refers to an average value of distances of edges connecting any two nodes. For a weighted directed graph, the average path length of the graph, i.e. the weighted average of the directed distances between any nodes, is also referred to as the characteristic path length or the average distance of the graph, i.e.:
where N is the number of nodes in the graph.
The modularization of the graph gathers a class of nodes meeting the same characteristics together to form clusters, the nodes in the graph are divided into accumulation without supervision, and the clustering characteristics of the nodes have strong correlation with the classification effect of corresponding classes.
And (3.2) calculating relevant indexes of the model graph by using the method in the step (3.1), and combining every 5 graph signature vectors of 1-4 into a graph signature matrix of 5-4 according to the time sequence. Such as the graph signature of 16 time slices in 1-5,2-6,3-7 \ 8230j 16-20 rounds (the graph signature for each time segment is from a model after activation of a different backdoor sample). And randomly selecting 50 back gate samples, and constructing 5000 sequence diagram signatures for training the sequence diagram convolution network.
(4) Signature S = { G (t) using the sequence diagram calculated in step (3) 1 ),G(t 2 ),…G(t 5 ) Train the timing graph convolutional network as the back door monitor F.
Specifically, a block diagram of the back door detector F is shown in fig. 4. Firstly, a graph signature matrix spliced by a plurality of moments is used as the input of a detector and passes through the coding layer of the detector. The coding layer is composed of a time sequence graph characteristic network, the characteristics of the graph signature matrix can be further purified, the characteristics of each moment are input into a time sequence LSTM structure, and time sequence information in the training process is extracted. And performing reverse decoding on the output hidden layer characteristics of the LSTM in a decoding layer to finally obtain two classification vectors of the model diagram, and dividing the two classification vectors by using a simple MLP or SVM structure to judge whether the model diagram is a back door model diagram.
Specifically, the graph signature vector G (t) at each time includes a two-dimensional transformation: (1) v at the node level i (t), wherein i represents a node sequence number, v represents a node characteristic, if the node exists, the corresponding node characteristic is updated, otherwise, the node and the characteristic thereof are added. (2) E of the continuous edge level ij And (t) indicating that a continuous edge related to the time sequence appears, wherein the continuous edges in the method are all non-directional edges.
The task of the timing graph convolutional network is to compute the scale of change D = | G (t) of the sequence graph signature feature at each time t from the successive graph signature features that change with time i )-G(t j )| distance . And using a long-term and short-term memory module to keep the long-term characteristics of the nodes, and recalculating the change scale of the current moment after a new-moment graph signature is input into the network. The class information of the sequence diagram signature including the backgate time and the sequence diagram signatures at all times after the backgate time is set as "including backgate", the class information at all times before the backgate training time is set as "not including backgate", and the sequence diagram convolutional network is retrained. And performing vulnerability evaluation on the model by using the trained evaluation device, and quickly calculating to obtain a graph index characteristic vector to finish the evaluation after the model activated by the sample is converted into a graph structure.
(5) And (5) detecting the real-time backdoor of the model training process by using the backdoor monitor F obtained in the step (4). In the training process of the model, after each round of training is finished, inputting part of training samples to activate and calculate the corresponding model diagram, and storing the diagram index vector obtained by calculation as a training log. The monitor can calculate the probability that the backdoor is possibly introduced into the current training time period by using the monitor only by using the graph index information stored in the training log corresponding to the suspicious training stage. Therefore, the states at different training moments can be backtracked and monitored.
The experimental result of carrying out backdoor detection on the AlexNet model trained on the CIFAR-10 data set shows that the method can find backdoor bugs in at most 2 training rounds and can realize the detection accuracy of 91% of the whole situation. And the fact that backdoor bugs can be found in at most 2 training rounds on a VGG16 model trained on a large image data set such as CIFAR-100 can be achieved, and the global detection accuracy is over 85%.
Correspondingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a back-gate real-time monitoring method based on a timing graph convolutional network as described above. As shown in fig. 5, for a hardware structure diagram of any device with data processing capability where a back door real-time monitoring method based on a timing diagram convolutional network according to an embodiment of the present invention is located, in addition to the processor, the memory, and the network interface shown in fig. 5, any device with data processing capability where a device is located in an embodiment may generally include other hardware according to an actual function of the any device with data processing capability, which is not described again.
Accordingly, the present application also provides a computer readable storage medium having stored thereon computer instructions, which when executed by a processor, implement a method for differential privacy and denoising data protection under a vertical federal framework as described above. The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any data processing capability device described in any of the foregoing embodiments. The computer readable storage medium may also be an external storage device of the wind turbine, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), and the like, provided on the device. Further, the computer readable storage medium may include both an internal storage unit of any data processing capable device and an external storage device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing capable device, and may also be used for temporarily storing data that has been output or is to be output.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only.
It will be understood that the present application is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.
Claims (10)
1. A back door real-time monitoring method based on a timing diagram convolutional network is characterized by specifically comprising the following steps of:
(1) Selecting a normal data set, designating the position of a trigger added on the normal data set through a mask matrix to generate a back door data set, and training a neural network model respectively based on the normal data set and the back door data set to obtain normal models at different moments and back door models corresponding to the normal models;
(2) Constructing a model diagram: defining neurons with the neuron output values of the neural network model being greater than or equal to a threshold value as activated neurons, and defining neurons with the neuron output values being less than the threshold value as non-activated neurons; preserving the activated neurons and their associated weights for constructing the topology of the model map; activating channels among the neurons to serve as activation channels, and adding the activation channels to the topological structure of the model graph to obtain a constructed model graph;
(3) Respectively calculating graph indexes including graph clustering coefficients, global average degree, average path length and modularization indexes of the model graph constructed in the step (2) to serve as a graph signature vector; sequentially splicing a plurality of graph signature vectors obtained by calculation according to a time sequence to obtain a sequence graph signature matrix;
(4) Training a sequence diagram convolution network by using the sequence diagram signature matrix obtained in the step (3) to serve as a backdoor monitor;
(5) And (3) repeating the steps (2) and (3) for any model to be detected consisting of the deep neural network to obtain a sequence diagram signature matrix corresponding to the model to be detected, inputting the sequence diagram signature matrix into the backdoor monitor obtained by training in the step (4), and calculating by the backdoor monitor to obtain the probability that the model to be detected is possibly introduced into the backdoor in the current training time period, thereby realizing backtracking and monitoring of different training time states of the model to be detected.
2. The timing graph convolutional network-based back gate real-time monitoring method as claimed in claim 1, wherein the maximum value w of the neuron weight of each layer in the neural network model is obtained in the step (2) max At 0.5w max Is a threshold value.
3. The back door real-time monitoring method based on the timing diagram convolutional network as claimed in claim 1 or 2, wherein the step (2) specifically comprises:
extracting key output weight for the activated neuron according to the clean model and the back door model obtained by training in the step (1), wherein the clean model and the back door model are in a full-connection layer structure; the samples are input into each model in turn, and the output of each layer is recorded as:
Output l ＝Layer l (Output l-1 )
wherein Layer l Layer function, output, representing the l-th layer l The parameters of the previous layer are input into the output matrix returned from the layer function of the l layer; the output of the sample at the l-th layer of the back door model is recorded as
The output of the sample on the l-th layer of the clean model is
Dividing the activated neurons and the inactivated neurons through the activation threshold of each layer, defining the neurons with the output values of the neurons of the neural network model being greater than or equal to the threshold as the activated neurons, and defining the neurons with the output values of the neurons being less than the threshold as the inactivated neurons; taking the paths among the activated neurons as activated paths, and storing all weights and connection information of the activated paths; non-activated neurons, namely inhibitory neurons, wherein the paths among the inhibitory neurons are used as inhibitory paths, and all weights and connection relations related to the inhibitory paths are deleted; the method comprises the following steps of taking a path between an activation neuron and a suppression neuron as a semi-activation path, calculating all semi-activation paths connected with a certain suppression neuron of an output layer, and only keeping the path with the maximum weight norm as an effective activation path; the sparsification of the model full connection layer is realized; and mapping the transmission relations of the neurons of the model full-connection layer and the output weights thereof to nodes and connection edges of the graph structure one by one to obtain the constructed model graph.
4. The method for monitoring the backdoor of the convolutional network based on the timing chart in real time as claimed in claim 1, wherein the formula for calculating the graph clustering coefficient in step (3) is as follows:
in the formula, the graph clustering coefficient C is used for measuring the aggregation degree of nodes in the graph, i is a node on the graph, e i Representing the number of edges, k, between the neighbors of node i i Representing the number of neighbor nodes of the node i;
the calculation formula of the global average degree is as follows:
wherein n is the number of nodes, d k Degree for each node;
the average path length is calculated as follows:
wherein N is the number of nodes in the graph.
5. The method for monitoring the back door based on the timing graph convolutional network in real time as claimed in claim 1, wherein the process of training the timing graph convolutional network as the back door monitor in step (4) by using the sequence graph signature matrix obtained in step (3) comprises:
the time sequence graph convolutional network comprises an encoder, a decoder and a classifier; inputting the sequence diagram signature matrix into an encoder, and extracting time sequence information; inputting the time sequence information into an encoder, and performing reverse decoding to obtain two classification vectors of the model diagram; and dividing the two classification vectors by a two-classifier to train to obtain the back door monitor according to whether the model graph is a back door model graph.
6. The timing diagram convolutional network-based back-gate real-time monitoring method of claim 5, wherein the two classifiers are selected from MLP or SVM classifiers.
7. The timing graph convolutional network-based back door real-time monitoring method of claim 1, wherein the graph signature vector comprises: the method comprises the following steps of node characteristics and continuous edges related to time sequences, wherein the continuous edges are undirected edges.
8. A back door real-time monitoring method based on a timing sequence convolution network, according to claim 1 or 7, characterized in that the timing sequence convolution network is used for calculating the change scale of the sequence diagram signature characteristic at each moment in real time along with the continuous diagram signature characteristic changing in time sequence.
9. An electronic device comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is configured to store program data and the processor is configured to execute the program data to implement the method for defending against backdoor attacks based on gradient link prediction according to any of the preceding claims 1-8.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out a method for backdoor attack defense based on gradient-based link prediction according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211425049.2A CN115758337A (en) | 2022-11-14 | 2022-11-14 | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211425049.2A CN115758337A (en) | 2022-11-14 | 2022-11-14 | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115758337A true CN115758337A (en) | 2023-03-07 |
Family
ID=85370916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211425049.2A Pending CN115758337A (en) | 2022-11-14 | 2022-11-14 | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115758337A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116739073A (en) * | 2023-08-10 | 2023-09-12 | 武汉大学 | Online back door sample detection method and system based on evolution deviation |
-
2022
- 2022-11-14 CN CN202211425049.2A patent/CN115758337A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116739073A (en) * | 2023-08-10 | 2023-09-12 | 武汉大学 | Online back door sample detection method and system based on evolution deviation |
| CN116739073B (en) * | 2023-08-10 | 2023-11-07 | 武汉大学 | Online back door sample detection method and system based on evolution deviation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jia et al. | Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning | |
| Goldblum et al. | Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses | |
| Yuan et al. | Adversarial examples: Attacks and defenses for deep learning | |
| CN111914256B (en) | Defense method for machine learning training data under toxic attack | |
| CN110334742B (en) | Graph confrontation sample generation method based on reinforcement learning and used for document classification and adding false nodes | |
| CN110659485A (en) | Detection of counter attacks by decoy training | |
| CN111783442A (en) | Intrusion detection method, device, server and storage medium | |
| Chen et al. | Backdoor attacks and defenses for deep neural networks in outsourced cloud environments | |
| Wang et al. | Man-in-the-middle attacks against machine learning classifiers via malicious generative models | |
| CN112468487B (en) | Method and device for realizing model training and method and device for realizing node detection | |
| Huang | Network intrusion detection based on an improved long-short-term memory model in combination with multiple spatiotemporal structures | |
| Jiang et al. | Research progress and challenges on application-driven adversarial examples: A survey | |
| CN115758337A (en) | Back door real-time monitoring method based on timing diagram convolutional network, electronic equipment and medium | |
| Kamran et al. | Semi-supervised conditional GAN for simultaneous generation and detection of phishing URLs: A game theoretic perspective | |
| Şeker | Use of Artificial Intelligence Techniques/Applications in Cyber Defense | |
| Anwer et al. | Intrusion detection using deep learning | |
| Arshad et al. | Anomalous Situations Recognition in Surveillance Images Using Deep Learning | |
| CN114021136A (en) | Back door attack defense system for artificial intelligence model | |
| Zhang et al. | Knowledge graph and behavior portrait of intelligent attack against path planning | |
| CN112948237A (en) | Poisoning model testing method, device and system based on neural pathway | |
| Sheng et al. | Network traffic anomaly detection method based on chaotic neural network | |
| Vdovjak et al. | Modern CNNs Comparison for Fire Detection in RGB Images | |
| CN116739073B (en) | Online back door sample detection method and system based on evolution deviation | |
| Santoso et al. | Malware Detection using Hybrid Autoencoder Approach for Better Security in Educational Institutions | |
| CN116821966B (en) | Privacy protection method, device and equipment for training data set of machine learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |