CN114898168B - Black box countermeasure sample generation method based on conditional standard flow model - Google Patents
Black box countermeasure sample generation method based on conditional standard flow model Download PDFInfo
- Publication number
- CN114898168B CN114898168B CN202210310612.5A CN202210310612A CN114898168B CN 114898168 B CN114898168 B CN 114898168B CN 202210310612 A CN202210310612 A CN 202210310612A CN 114898168 B CN114898168 B CN 114898168B
- Authority
- CN
- China
- Prior art keywords
- sample
- model
- training
- conditional
- condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 71
- 238000009826 distribution Methods 0.000 claims abstract description 27
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 24
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 230000008878 coupling Effects 0.000 claims description 14
- 238000010168 coupling process Methods 0.000 claims description 14
- 238000005859 coupling reaction Methods 0.000 claims description 14
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 abstract description 2
- 239000000284 extract Substances 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 7
- 238000012733 comparative method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000012855 volatile organic compound Substances 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a black box countermeasure sample generation method based on a condition standard flow model, which comprises the steps of firstly, generating a countermeasure sample corresponding to an original image by using a white box attack method to obtain a training data set, and constructing a condition standard flow model comprising a convolutional neural network and a condition GLOW model, wherein the convolutional neural network extracts image characteristics from the original sample as a condition variable, and the condition GLOW model obtains a corresponding hidden space representation according to the condition variable and the countermeasure sample code corresponding to the original image; training the conditional standard flow model by using a training data set to obtain the distribution of the hidden space representation of the training sample set, sampling the distribution of the hidden space representation by taking the characteristics extracted from the clean image as a conditional variable to obtain a countermeasure sample output, and then cutting to obtain a final countermeasure sample. The invention solves the problems of large inquiry times, extremely serious consumption of computing resources and time, incapability of generating countermeasure samples in batches and the like in a black box attack scene.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a black box countermeasure sample generation method based on a conditional standard flow model.
Background
With the rapid development of artificial intelligence, deep neural networks are widely used in various fields (e.g., computer vision, natural language processing, autopilot, information security, etc.), with great success. The wide range of applications of deep neural networks has led to increased attention to the security of deep neural networks themselves. Currently, researchers have proposed a method of combating attacks against deep neural networks, namely: malicious disturbances are added to the recognition object, and the small disturbances are not perceived by human vision or hearing, but are sufficient to deceive the deep neural network, so that a normally trained model outputs a misprediction result with high confidence, and the deep neural network recognition error is caused. Such attacks pose a great threat to deep learning models.
The work of the deep neural network security research has just started, and although the previous work of the former has been performed, there are some places to be studied intensively. The main expression is as follows: 1) Most of the existing attack resisting methods are white-box attacks based on target model gradients, and the white-box attack methods are difficult to realize in the real world; 2) While some black box attack methods are proposed, they are often also based on the transferability of the challenge sample or the gradient estimation of the target model, another class of query-based black box attack methods requires a large number of queries and iterative optimization operations on the target model (system) to obtain the challenge sample, which requires a large amount of computational resources and time consumption; 3) While query-based black box attack methods can be implemented in the physical world, there is still a risk perceived by the target system during a large number of queries and challenge samples cannot be generated quickly and in batches.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a black box countermeasure sample generation method based on a conditional standard flow model, designs a new conditional standard flow model, solves the problems of large query times, extremely serious consumption of computing resources and time, incapability of generating countermeasure samples in batches and the like in a black box attack scene, and provides a rapid and efficient countermeasure sample generation method for artificial intelligent safety and robustness research.
In order to achieve the above object, the black box countermeasure sample generation method based on the conditional standard flow model of the present invention comprises the steps of:
s1: acquiring a plurality of images and corresponding labels according to actual needs, normalizing each image to a preset size to serve as an original sample, and thus obtaining an original sample set X;
training a target attack model by adopting the collected original sample set X, and then acquiring a countermeasure sample X 'of each sample image X in the original sample image set X by using a white box attack method to obtain a countermeasure sample set X';
finally, taking the original sample set X and the countermeasure sample set X' as training data sets of a conditional standard flow model;
s2: building a conditional standard flow model comprising a convolutional neural network and a conditional GLOW model, wherein:
the convolutional neural network is used for extracting image features from an input image, and inputting the obtained features serving as a condition variable into a condition GLOW model;
The condition GLOW model is used for encoding the countermeasure sample of the input image according to the condition variable of the convolutional neural network to obtain a corresponding hidden space representation; the conditional GLOW model is composed of L-1 stream components, squeeze layers and K stream blocks, wherein the stream components are composed of one squeeze layer, K stream blocks and one split layer stack, and the values of L and K are determined according to the requirement; the stream block is a conditional stream block comprising Actnorm layers, a1 x1 convolution layer and an affine coupling layer, wherein:
the Actnorm layer is used for carrying out activation standardization on the input features, and inputting the obtained features into the 1 multiplied by 1 convolution layer;
The 1X 1 convolution layer is used for carrying out 1X 1 convolution processing on the input features, and inputting the obtained features into the affine coupling layer;
the affine coupling layer is used for receiving the characteristics sent by the convolution layer and the condition variables sent by the convolution neural network, carrying out affine coupling processing and then outputting;
S3: training the conditional standard flow model according to the training sample set obtained in the step S1;
S4: after the condition standard flow model is trained, sequentially inputting each training sample in the training sample set into the trained condition standard flow model to obtain hidden space representations corresponding to each training sample, and calculating to obtain the mean mu and variance sigma of hidden space representations of all training samples to obtain the distribution N (mu, sigma 2) of the hidden space representations;
s5: when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image Then extracting by a convolutional neural network in the conditional standard flow model to obtain an input image/>Condition variable of (2)According to the condition variables/>Sampling the distribution N (mu, sigma 2) obtained in the step S4 to obtain a hidden space representation/>The/>, is expressed according to the hidden space by a conditional GLOW model in a conditional standard flow modelAnd condition variable/>Reverse reasoning is carried out to obtain an input image/>Initial challenge sample/>Then use clipping function to perform initial challenge sample/>Processing to obtain final challenge sample/>The formula is as follows:
where Clip () represents a preset clipping function and epsilon represents a preset perturbation parameter.
The invention relates to a black box countermeasure sample generation method based on a condition standard stream model, which comprises the steps of firstly generating a countermeasure sample corresponding to an original image by using a white box attack method, integrating an original sample set with the corresponding countermeasure sample set to obtain a training data set, and then constructing a condition standard stream model comprising a convolutional neural network and a condition GLOW model, wherein the convolutional neural network takes the original sample as an input to extract image characteristics as a condition variable, and the condition GLOW model obtains a corresponding hidden space representation according to the condition variable and the countermeasure sample corresponding to the original image as an input code; training the conditional standard flow model by using a training data set; after training is completed, obtaining the distribution of hidden space representation of the training sample set, sampling the distribution of hidden space representation by taking the characteristics extracted from the clean image as a condition variable to obtain an countermeasure sample output, and finally cutting the output countermeasure sample to obtain a final countermeasure sample.
When the corresponding sample is generated aiming at the specific sample, the method does not depend on a large amount of computing resources and computing time, can generate the countermeasure sample in batches, solves the problems that the existing black box attack method needs high computing amount, high time waiting and cannot realize mass production to a great extent, and has greatly improved performances in aspects of attack success rate, average query times, mobility of the generated countermeasure sample and the like. The black box countermeasure sample generation method based on the conditional standard flow model has important value for research work of a countermeasure sample community and the improvement of the robustness of the current artificial intelligence system.
Drawings
FIG. 1 is a schematic diagram of a conditional standard flow model-based black box challenge sample generation method of the present invention;
FIG. 2 is a flow chart of an embodiment of the method for generating a black box challenge sample based on a conditional standard flow model of the present invention;
FIG. 3 is a schematic diagram of a conditional standard flow model in accordance with the present invention;
FIG. 4 is a block diagram of a conditional flow block of the present invention;
FIG. 5 is a training flowchart of the conditional standard flow model in the present embodiment;
FIG. 6 is a graph comparing the number of successful attack queries over CIFAR-10 datasets for the present invention and comparison method;
FIG. 7 is a graph comparing the number of successful attack queries on SVHN datasets for the present invention and comparison method;
FIG. 8 is a graph comparing challenge sample transferability of the present invention and AdvFlow algorithm on CIFAR-10 dataset;
Fig. 9 is a graph comparing challenge sample transferability of the present invention and AdvFlow algorithm on SVHN dataset.
Detailed Description
The following description of the embodiments of the invention is presented in conjunction with the accompanying drawings to provide a better understanding of the invention to those skilled in the art. It is to be expressly noted that in the description below, detailed descriptions of known functions and designs are omitted here as perhaps obscuring the present invention.
Examples
In order to better explain the technical solution of the present invention, a brief description will be given first of all of the principles on which the present invention is based. Conventional attack methods produce a final challenge sample by performing complex reasoning based on the target model to produce a challenge disturbance, which is then added to the original sample. This process is highly dependent on the reasoning results, which can consume a significant amount of computational cost, and typically will generate a single "best" sample according to certain criteria. While the present invention recognizes that all challenge samples may follow a particular distribution that is not consistent with a normal distribution. This is mainly due to the fact that the training data involved in the optimization of the different depth models is fixed. In other words, the training data distribution characterizes the fixed distribution approximated by these models during training, and therefore, the effect of the data distribution not visible in training on the models is also present. This explains that the challenge sample (most of which is data that is not visible to the model during training) follows a non-aligned distribution. And it is because normal samples and challenge sample data exhibit similar appearances, it is believed that the two distributions overlap and it is reasonable to assume that the two distributions can be converted to each other.
Based on the analysis, the invention provides a black box countermeasure sample generation method based on a conditional standard flow model. Fig. 1 is a schematic diagram of a black box challenge sample generation method based on a conditional standard flow model of the present invention. As shown in fig. 1, the present invention collects a large number of resistance samples by the existing white-box attack method. Although these samples look similar to normal samples, direct conversion between the two samples is still difficult, and even prohibitive. This is because small perturbations may be overwhelmed by complex structures and textures in the normal sample and are therefore insensitive to the generative model. To alleviate this problem, it is believed that small perturbations should be conditioned on normal inputs, which provide clues to the generation process. In particular, the conditional standard flow model is used to implement a conditional generation process that allows synthesizing resistant samples based on normal samples and random variables. The random variables may diversify the generated samples, that is, when the conditional standard flow model is trained, a random sampling may be performed in the hidden space to generate a set of noise that is inversely inferred by the conditional standard flow model to generate a set of antagonistic samples.
FIG. 2 is a flow chart of an embodiment of the method for generating a black box challenge sample based on a conditional standard flow model of the present invention. As shown in fig. 2, the specific steps of the black box countermeasure sample generation method based on the conditional standard flow model of the present invention include:
S201: raw samples and their corresponding challenge sample collection:
and acquiring a plurality of images and corresponding labels according to actual needs, and normalizing each image to a preset size to serve as an original sample, so that an original sample set X is obtained.
Training a target attack model by adopting the collected original sample set X, and then acquiring a countermeasure sample X 'of each sample image X in the original sample image set X by using a white box attack method to obtain a countermeasure sample set X'.
Finally, the original sample set X and the countermeasure sample set X' are used as training data sets of the conditional standard flow model.
The original sample set in this embodiment uses the existing data set: CIFAR-10, SVHN, and ImageNet. The specific method for obtaining the challenge sample is as follows: for the CIFAR-10 and SVHN datasets, the training set was used to train a target attack model, here a ResNet-56 network, and then a PGD (Projected gradient descent) algorithm was used to generate an impedance sample. Whereas for ImageNet, about 30,000 images were selected from its validation set to train the model, based on the multi-model integrated MI-FGSM (Momentum ITERATIVE FAST GRADIENT SIGN Method, momentum iterative fast gradient symbology) Method, with integrated models InceptionV, inceptionResnetV2 and ResNetV2-101. In this embodiment, two disturbance parameters are used for generating the countermeasure model, and epsilon=8 and epsilon=16, respectively.
S202: building a conditional standard flow model:
In order to improve the capability of processing image textures, a conditional standard flow model is constructed based on a GLOW model in the embodiment. The original Glow model involves convolution, coupling and normalization operations in model construction, but since the original Glow model does not consider conditions in probability modeling, the invention improves the Glow model, and appropriately integrates image content into condition variables, thereby obtaining a conditional standard flow model.
FIG. 3 is a schematic diagram of a conditional standard flow model in accordance with the present invention. As shown in fig. 3, the conditional standard flow model in the present invention includes a convolutional neural network and a conditional GLOW model, in which:
The convolutional neural network is used for extracting image features from an input image, and inputting the obtained features serving as conditional variables into a conditional GLOW model. Convolutional neural networks are introduced because they can only provide very low-level features if the original input image is taken as a condition variable. These features are not sufficient for feature modeling and can burden the sub-networks in the affine coupling layer of the conditional GLOW model, so the present invention extracts higher-level features from the original input image through the convolutional neural network. In the embodiment, the convolutional neural network adopts a pretrained VGG-19 model, and features output by the convolutional layer at the last stage are taken as extracted features. The datasets employed in training the convolutional neural network in this example were CIFAR-10, SVHN, and ImageNet. In the whole GLOW model training process, the convolutional neural network can fix parameters and also can train and optimize.
The conditional GLOW model is used for encoding the countermeasure sample of the input image according to the condition variable of the convolutional neural network to obtain the corresponding hidden space representation. As shown in FIG. 3, the conditional GLOW model of the present invention is similar to the conventional GLOW model and is composed of L-1 stream components, squeeze layers (compression layers) and K stream block stacks, wherein the stream components are composed of one squeeze layer, K stream blocks and one split layer (split layer) stack, and the values of L and K are determined according to the requirement. Unlike the conventional GLOW model, the stream blocks in the present invention take conditions into account and thus may be referred to as conditional stream blocks. FIG. 4 is a block diagram of a conditional flow block of the present invention. As shown in fig. 4, the conditional flow block in the present invention includes Actnorm (activation normalization, active normalization) layer, 1×1 convolution layer, affine coupling layer, in which:
the Actnorm layer is used for carrying out activation standardization on the input features, and inputting the obtained features into the 1 x1 convolution layer.
The 1×1 convolution layer is used for performing 1×1 convolution processing on the input features, and inputting the obtained features into the affine coupling layer.
The affine coupling layer is used for receiving the characteristics sent by the convolution layer and the condition variables sent by the convolution neural network, carrying out affine coupling processing and then outputting.
S204: training a conditional standard flow model:
And training the conditional standard flow model according to the training sample set obtained in the step S201.
In order to improve the model training effect, the training process of the conditional standard flow model is optimized in the embodiment. Fig. 5 is a training flowchart of the conditional standard flow model in the present embodiment. As shown in fig. 5, the training specific steps of the conditional standard flow model in this embodiment include:
s501: selecting a current batch of training samples:
B training samples are randomly selected from the training sample set to serve as current batch training samples, B represents batch size, and the value of the B is determined according to actual needs.
S502: judging whether the iteration round t meets the preset condition, if so, proceeding to step S503, otherwise proceeding to step S504. In this embodiment, the two calculation modes are alternately executed for the loss function, so that a preset condition is required to switch the calculation modes of the loss function, for example, hidden space loss can be calculated in odd-numbered rounds and image loss can be calculated in even-numbered rounds.
S503: calculating hidden space loss:
Calculating the hidden space loss L (theta; z, x ', c) of each training sample in the current batch, and then averaging to obtain the hidden space loss L (theta; z, x ', c) of the current batch, wherein the calculation formula of the hidden space loss L (theta; z, x ', c) is as follows:
Wherein x and x ' respectively represent an input original sample and a corresponding countermeasure sample, c represents a condition variable extracted from the input image x by the convolutional neural network, p z () represents distribution, f θ (x '; c) represents a hidden space representation z obtained by a conditional GLOW model with a network parameter of θ according to the input countermeasure sample x ' and the condition variable c, Representing jacobian, || represents the absolute value.
S504: calculating image loss:
Since the task of the challenge sample attack is to generate a challenge sample having a similar appearance to the input conditional sample, it is ensured that the generation of the input image x from the hidden space representation z does not lead to surprising results. To achieve this, the present embodiment also introduces MSE (Mean Square Error ) loss during training. Namely randomly extracting a batch of training samples from the current batch, calculating the image loss L MSE (theta; z, c) of each extracted training sample, and then averaging to obtain the image loss L MSE (theta; z, c) of the current batch, wherein the calculation formula of the image loss L MSE (theta; z, c) is as follows:
LMSE(θ;z,c)＝||fθ(z;c)-x′||2
Wherein f θ (z; c) represents a countermeasure sample obtained by reverse reasoning of the conditional GLOW model with the network parameter being θ according to the hidden space representation z and the condition c, and 2 represents a two-norm calculation.
S505: updating model parameters:
and updating parameters of the conditional standard flow model according to the current calculated loss. In the embodiment, an Adam optimization algorithm is adopted to update and optimize model parameters.
S506: and judging whether the training ending condition is reached, if so, ending the training, otherwise, returning to the step S501.
The training ending conditions are generally two, namely the maximum iteration times are reached, or the condition standard flow model converges, and the training ending conditions are selected according to the needs in practical application.
S204: generating a distribution of the hidden spatial representation:
After the conditional standard flow model has been trained, then the implicit spatial representation of each challenge sample in the training sample set is expected to follow the hypothetical gaussian distribution N (0, 1). But in practice it has been found that these hidden space representations have changed the mean and standard deviation values. This may be because the training data is insufficient and the image loss may deviate the center of the gaussian distribution, but experiments show that the shift occurs even without the image loss. Based on this observation, sampling based on the shifted mean and standard values yields a hidden spatial representation, which may lead to better performance than sampling from N (0, 1).
According to the analysis, after the condition standard flow model is trained, each training sample in the training sample set is sequentially input into the trained condition standard flow model to obtain hidden space representations corresponding to each training sample, the mean mu and the variance sigma of hidden space representations of all the training samples are obtained through calculation, and the distribution N (mu, sigma 2) of the hidden space representations is obtained.
S205: generating a new challenge sample:
when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image Then extracting by a convolutional neural network in the conditional standard flow model to obtain an input image/>Condition variable/>According to the condition variables/>Sampling the distribution N (mu, sigma 2) obtained in the step S204 to obtain a hidden space representation/>The/>, is expressed according to the hidden space by a conditional GLOW model in a conditional standard flow modelAnd condition variable/>Reverse reasoning is carried out to obtain an input image/>Initial challenge sample/>Then use clipping function to perform initial challenge sample/>Processing to obtain final challenge sample/>The formula is as follows:
where Clip () represents a preset clipping function and epsilon represents a preset perturbation parameter.
In order to better illustrate the technical effects of the invention, the invention is experimentally verified by adopting a specific example. In the experimental verification, simBA algorithm and AdvFlow algorithm are adopted as comparison methods, and the comparison is carried out on the aspects of 1) attack success rate, 2) average query times and 3) mobility of the generated countermeasure sample.
1) Attack success rate and average inquiry times
The method aims to keep a higher attack success rate when the number of inquiry times is reduced. In the experimental verification, the maximum query number is respectively set as 100, 200, 300, 400, 500 and 1000, the attack is successful only within the predefined query number, otherwise, the attack fails. Table 1 is a table comparing attack success rate and average number of queries for the present and comparative methods at CIFAR-10 datasets with perturbation epsilon=8.
TABLE 1
Table 2 is a table of attack success rate and average number of queries for the present and comparative methods at CIFAR-10 datasets with perturbation epsilon=16.
TABLE 2
Table 3 is a table comparing attack success rate and average number of queries at SVHN dataset for the present and comparative methods with perturbation epsilon=8.
TABLE 3 Table 3
Table 4 is a table comparing attack success rates and average query times at SVHN datasets for the present and comparative methods with perturbation epsilon=16.
TABLE 4 Table 4
As can be seen from tables 1 to 4, in most cases, the attack success rate of the present invention is higher than 2 comparison methods, and it is seen that the generated model proposed by the present invention can generate an effective challenge sample. It should be noted in particular that the average number of queries required by the present invention is much smaller than the average number of queries required by 2 comparison methods.
Experiments on ImageNet datasets were more challenging because their data were much more complex than CIFAR-10 datasets and SVHN datasets. Table 5 is a table comparing attack success rate and average number of queries in ImageNet dataset for the present and comparative methods with perturbation e=16.
TABLE 5
As shown in table 5, it can be seen that the present invention has significant advantages in both the attack success rate and the average number of queries on the ImageNet dataset.
2) Distribution of query times
To better observe the advantage of the present invention in terms of query times, this experimental verification plots histograms of the number of queries that performed successful attacks on the CIFAR-10 dataset and the SVHN dataset, with ShuffleNetV and VGG-16 being used as target attack models for CIFAR-10 and SVHN, respectively, with a maximum number of queries limited to 500. FIG. 6 is a graph of the distribution of successful attack queries over CIFAR-10 datasets for the present invention and comparison method. FIG. 7 is a graph of the distribution of successful attack queries over SVHN datasets for the present invention and comparative method. In all cases, the invention can perform a successful attack based on most samples only once, as shown in fig. 6 and 7. At the perturbation epsilon=16, the median number of queries of the present invention on CIFAR-10 data and SVHN data sets was only 19.41 and 23.67, respectively. Notably, on the target attack model ShuffleNetV, the present invention has attack success rates of 88% and 90% for a few queries, respectively, when epsilon=8 and epsilon=16. In contrast, advFlow and SimBA algorithms typically require hundreds of queries to attack successfully, while a small number of queries (e.g.,.ltoreq.100) do not allow these methods to work properly. The results show that the median of the present invention is 1 in all cases, which fully verifies the advancement of the present invention.
3) Transferability of generated challenge samples
Current black box attacks rely to a large extent on the transferability assumption of challenge samples, i.e. challenge samples generated from a particular model can be used to attack other different models. To prove that this assumption is valid, this experimental verification explores the portability of the generated challenge sample based on different models on the CIFAR-10 dataset and the SVHN dataset.
Specifically, this experiment verifies that 8 models were selected, including ResNet-56, VGG-16, VGG-19, shuffleNetV2, mobileNetV2, inceptionV3, denseNet-169, and GoogLeNet. In each model case, the model is first trained until optimal performance (typically over 90%) is achieved on the test set. 1000 images were then randomly selected from the test set, with the images correctly classified by the model and the corresponding challenge samples misclassified. The challenge sample generated is used to attack his model. For fair comparison, epsilon=16, the maximum number of queries was set to 500. The present invention compares with AdvFlow algorithm in a no-target black box attack. FIG. 8 is a graph comparing challenge sample transferability of the present invention and AdvFlow algorithm on the CIFAR-10 dataset. Fig. 9 is a graph comparing challenge sample transferability of the present invention and AdvFlow algorithm on SVHN dataset. As shown in fig. 8 and 9, where each row represents which model was targeted when the challenge sample was generated, and each column represents which model was attacked by the generated sample. The inventive migratable ASR over CIFAR-10 datasets can be seen to be from 33.6% to 79.6%, while the AdvFlow algorithm is 3.4% to 13.0%. This means that the samples generated by the present invention produced a higher ASR on other models than AdvFlow algorithm (in most cases about 30% -66% higher) validating the excellent migratability of the present invention. This is because AdvFlow algorithm relies heavily on feedback of the target model during each query, failing to extract the migratable features. In contrast, the present invention learns the distribution of resistance samples that do not fit to a particular model.
4) Image independence attack
In order to evaluate the performance of the invention on samples with different semantics, an attack experiment was performed on data sets other than the training ImageNet data set. Specifically, the test dataset includes VOCs 2007, VOCs 2012, plasceS565, caltech101, and Caltech256. Target attack models include VGG-19, inceptionV, resNet-152, and WIDERESNET-50, all of which are implemented in PyTorch. Table 6 is a statistical table of attack results for different test data sets and different target attack models according to the present invention.
| VOC2007 | VOC2012 | Plasces365 | Caltech101 | Catech256 | |
| VGG-19 | 91.7 | 93.0 | 90.9 | 93.5 | 86.3 |
| Inception_v3 | 87.5 | 90.8 | 91.1 | 93.6 | 86.4 |
| ResNet-152 | 85.1 | 89.2 | 87.3 | 94.4 | 83.8 |
| WideResNet-50 | 86.1 | 89.7 | 84.1 | 93.4 | 84.1 |
TABLE 6
As shown in Table 6, the present invention trained on the ImageNet dataset can be used to generate valid challenge samples on other datasets without retraining. In some cases, the attack success rate may exceed 90%, with the maximum query size limited to 100.
While the foregoing describes illustrative embodiments of the present invention to facilitate an understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but is to be construed as protected by the accompanying claims insofar as various changes are within the spirit and scope of the present invention as defined and defined by the appended claims.
Claims (3)
1. A method for generating a black box countermeasure sample based on a conditional standard flow model, comprising the steps of:
s1: acquiring a plurality of images and corresponding labels according to actual needs, normalizing each image to a preset size to serve as an original sample, and thus obtaining an original sample set X;
training a target attack model by adopting the collected original sample set X, and then acquiring a countermeasure sample X 'of each sample image X in the original sample image set X by using a white box attack method to obtain a countermeasure sample set X';
finally, taking the original sample set X and the countermeasure sample set X' as training data sets of a conditional standard flow model;
s2: building a conditional standard flow model comprising a convolutional neural network and a conditional GLOW model, wherein:
the convolutional neural network is used for extracting image features from an input image, and inputting the obtained features serving as a condition variable into a condition GLOW model;
The condition GLOW model is used for encoding the countermeasure sample of the input image according to the condition variable of the convolutional neural network to obtain a corresponding hidden space representation; the conditional GLOW model is composed of L-1 stream components, squeeze layers and K stream blocks, wherein the stream components are composed of one squeeze layer, K stream blocks and one split layer stack, and the values of L and K are determined according to the requirement; the stream block is a conditional stream block comprising Actnorm layers, a1 x1 convolution layer and an affine coupling layer, wherein:
the Actnorm layer is used for carrying out activation standardization on the input features, and inputting the obtained features into the 1 multiplied by 1 convolution layer;
The 1X 1 convolution layer is used for carrying out 1X 1 convolution processing on the input features, and inputting the obtained features into the affine coupling layer;
the affine coupling layer is used for receiving the characteristics sent by the convolution layer and the condition variables sent by the convolution neural network, carrying out affine coupling processing and then outputting;
S3: training the conditional standard flow model according to the training sample set obtained in the step S1;
S4: after the condition standard flow model is trained, sequentially inputting each training sample in the training sample set into the trained condition standard flow model to obtain hidden space representations corresponding to each training sample, and calculating to obtain the mean mu and variance sigma of hidden space representations of all training samples to obtain the distribution N (mu, sigma 2) of the hidden space representations;
s5: when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image Then extracting by a convolutional neural network in the conditional standard flow model to obtain an input image/>Condition variable/>According to the condition variables/>Distribution obtained in step S4/>Sampling to obtain hidden space representation/>The/>, is expressed according to the hidden space by a conditional GLOW model in a conditional standard flow modelAnd condition variable/>Reverse reasoning is carried out to obtain an initial challenge sample/>, of the input image x * Then use clipping function to perform initial challenge sample/>Processing to obtain final challenge sample/>The formula is as follows:
where Clip () represents a preset clipping function and epsilon represents a preset perturbation parameter.
2. The black box countermeasure sample generation method according to claim 1, wherein the convolutional neural network is a VGG-19 model, and features output by a final stage convolutional layer are taken as extracted features.
3. The black box challenge sample generation method according to claim 1, wherein the training method of the conditional standard flow model in step S3 is as follows:
S3.1: randomly selecting B training samples from the training sample set as current batch training samples, wherein B represents the batch size, and the value of the B is determined according to actual needs;
s3.2: judging whether the iteration round t meets a preset condition, if so, entering a step S3.3, otherwise, entering a step S3.4;
S3.3: calculating the hidden space loss L (theta; z, x ', c) of each training sample in the current batch, and then averaging to obtain the hidden space loss L (theta; z, x ', c) of the current batch, wherein the calculation formula of the hidden space loss L (theta; z, x ', c) is as follows:
Wherein x and x' respectively represent an input original sample and a corresponding countermeasure sample, c represents a condition variable extracted from the input image x by the convolutional neural network, p z () represents distribution, The conditional GLOW model representing network parameters θ is based on the implicit spatial representation z,/>, obtained from the input challenge sample x' and the conditional variable cRepresenting jacobian, || represents solving absolute values;
S3.4: randomly extracting a batch of training samples from the current batch, calculating the image loss L MSE (theta; z, c) of each extracted training sample, and then averaging to obtain the image loss L MSE (theta; z, c) of the current batch, wherein the calculation formula of the image loss L MSE (theta; z, c) is as follows:
LMSE(θ;z,c)＝||fθ(z;c)-x′||2
F θ (z; c) represents a countermeasure sample obtained by reverse reasoning of a conditional GLOW model with the network parameter of theta according to the hidden space representation z and the condition c;
S3.5: updating parameters of the conditional standard flow model according to the current calculated loss;
s3.6: judging whether the training ending condition is reached, if so, ending the training, otherwise, returning to the step S3.1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210310612.5A CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210310612.5A CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114898168A CN114898168A (en) | 2022-08-12 |
| CN114898168B true CN114898168B (en) | 2024-05-17 |
Family
ID=82716155
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210310612.5A Active CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114898168B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801297A (en) * | 2021-01-20 | 2021-05-14 | 哈尔滨工业大学 | Machine learning model adversity sample generation method based on conditional variation self-encoder |
| CN113674140A (en) * | 2021-08-20 | 2021-11-19 | 燕山大学 | Physical countermeasure sample generation method and system |
| CN114066912A (en) * | 2021-11-23 | 2022-02-18 | 中国人民解放军战略支援部队信息工程大学 | Intelligent countermeasure sample generation method and system based on optimization algorithm and invariance |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2607647A (en) * | 2020-03-26 | 2022-12-14 | Shenzhen Inst Adv Tech | Method and device for generating adversarial image, equipment, and readable storage medium |
-
2022
- 2022-03-28 CN CN202210310612.5A patent/CN114898168B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801297A (en) * | 2021-01-20 | 2021-05-14 | 哈尔滨工业大学 | Machine learning model adversity sample generation method based on conditional variation self-encoder |
| CN113674140A (en) * | 2021-08-20 | 2021-11-19 | 燕山大学 | Physical countermeasure sample generation method and system |
| CN114066912A (en) * | 2021-11-23 | 2022-02-18 | 中国人民解放军战略支援部队信息工程大学 | Intelligent countermeasure sample generation method and system based on optimization algorithm and invariance |
Non-Patent Citations (2)
| Title |
|---|
| EnsembleFool: A method to generate adversarial examples based on model fusion strategy;Peng, Wenyu等;COMPUTERS & SECURITY;20210707;全文 * |
| 深度学习中的对抗攻击与防御;刘西蒙;谢乐辉;王耀鹏;李旭如;;网络与信息安全学报;20201013(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114898168A (en) | 2022-08-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109639710B (en) | Network attack defense method based on countermeasure training | |
| CN111475797B (en) | Method, device and equipment for generating countermeasure image and readable storage medium | |
| WO2021189364A1 (en) | Method and device for generating adversarial image, equipment, and readable storage medium | |
| CN110334806A (en) | A kind of confrontation sample generating method based on production confrontation network | |
| CN110941794A (en) | Anti-attack defense method based on universal inverse disturbance defense matrix | |
| CN112784929B (en) | Small sample image classification method and device based on double-element group expansion | |
| CN111598210B (en) | Anti-attack defense method for anti-attack based on artificial immune algorithm | |
| CN111737691A (en) | Method and device for generating confrontation sample | |
| CN111507384B (en) | Method for generating confrontation sample of black box depth model | |
| CN113704758B (en) | Black box attack countermeasure sample generation method and system | |
| Ying et al. | Human ear recognition based on deep convolutional neural network | |
| CN114491525B (en) | Android malicious software detection feature extraction method based on deep reinforcement learning | |
| CN112200243A (en) | Black box countermeasure sample generation method based on low query image data | |
| CN115048983A (en) | Counterforce sample defense method of artificial intelligence system based on data manifold topology perception | |
| CN114240951A (en) | Black box attack method of medical image segmentation neural network based on query | |
| CN114898168B (en) | Black box countermeasure sample generation method based on conditional standard flow model | |
| CN111737688B (en) | Attack defense system based on user portrait | |
| CN116051924B (en) | Divide-and-conquer defense method for image countermeasure sample | |
| Aljadaany et al. | Iris super-resolution via nonparametric over-complete dictionary learning | |
| CN117011508A (en) | Countermeasure training method based on visual transformation and feature robustness | |
| CN110929239A (en) | Terminal unlocking method based on lip language instruction | |
| CN115510986A (en) | Countermeasure sample generation method based on AdvGAN | |
| CN113449865B (en) | Optimization method for enhancing training artificial intelligence model | |
| CN113379593B (en) | Image generation method, system and related equipment | |
| CN115270891A (en) | Method, device, equipment and storage medium for generating signal countermeasure sample |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |