CN114898168A - Black box confrontation sample generation method based on condition standard flow model - Google Patents
Black box confrontation sample generation method based on condition standard flow model Download PDFInfo
- Publication number
- CN114898168A CN114898168A CN202210310612.5A CN202210310612A CN114898168A CN 114898168 A CN114898168 A CN 114898168A CN 202210310612 A CN202210310612 A CN 202210310612A CN 114898168 A CN114898168 A CN 114898168A
- Authority
- CN
- China
- Prior art keywords
- sample
- conditional
- model
- training
- standard flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 74
- 238000009826 distribution Methods 0.000 claims abstract description 30
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 22
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 230000008878 coupling Effects 0.000 claims description 14
- 238000010168 coupling process Methods 0.000 claims description 14
- 238000005859 coupling reaction Methods 0.000 claims description 14
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 235000000332 black box Nutrition 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 2
- 239000000284 extract Substances 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 230000003042 antagnostic effect Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 235000009508 confectionery Nutrition 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a black box confrontation sample generation method based on a conditional standard flow model, which comprises the steps of firstly generating confrontation samples corresponding to an original image by using a white box attack method, obtaining a training data set, and constructing the conditional standard flow model comprising a convolutional neural network and a conditional GLOW model, wherein the convolutional neural network extracts image characteristics for the original sample as condition variables, and the conditional GLOW model obtains corresponding hidden space representation according to the condition variables and the confrontation sample codes corresponding to the original image; training a conditional standard flow model by using a training data set to obtain the distribution represented by the hidden space of the training sample set, sampling the distribution represented by the hidden space by using the characteristics extracted from a clean image as a condition variable to obtain a confrontation sample output, and cutting to obtain a final confrontation sample. The method solves the problems that in a black box attack scene, the query times are large, the consumption of computing resources and time is extremely serious, countersamples cannot be generated in batches, and the like.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a black box confrontation sample generation method based on a condition standard flow model.
Background
With the rapid development of artificial intelligence, deep neural networks are widely applied in various fields (such as computer vision, natural language processing, automatic driving, information security, and the like), and have achieved great success. The widespread use of deep neural networks has led to increased concerns regarding the security of the deep neural networks themselves. Researchers have proposed methods for resisting attacks against deep neural networks, namely: malicious disturbances are added aiming at the recognition object, and the fine disturbances can not be perceived by human vision or hearing, but can deceive the deep neural network enough, so that the normally trained model outputs a wrong prediction result with high confidence coefficient, and the deep neural network recognition is wrong. Such attacks pose a huge threat to the deep learning model.
The research work on the deep neural network safety is just started, and although the former previous exploration work is already available, there are some places to be studied deeply. The main points are as follows: 1) most existing anti-attack methods are white-box attacks based on target model gradient, and the white-box attack methods are difficult to realize in the real world; 2) although some black-box attack methods are proposed, they are also often based on the transferability of the countermeasure sample or the gradient estimation of the target model, and another type of query-based black-box attack method requires a large number of queries and iterative optimization operations on the target model (system) to obtain the countermeasure sample, which requires a large amount of computing resources and time consumption; 3) although query-based black-box attack methods can be implemented in the physical world, they still face the risk of being perceived by the target system during a large number of queries and cannot generate countersamples quickly and in bulk.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a black box countermeasure sample generation method based on a conditional standard flow model, designs a new conditional standard flow model, solves the problems of large query times, extremely serious consumption of computing resources and time, incapability of generating countermeasure samples in batches and the like in a black box attack scene, and provides a method for quickly and efficiently generating countermeasure samples for the research of artificial intelligent safety and robustness.
In order to achieve the above object, the black box confrontation sample generation method based on the conditional standard flow model according to the present invention comprises the following steps:
s1: acquiring a plurality of images and corresponding labels according to actual needs, and normalizing each image to a preset size to be used as an original sample, thereby obtaining an original sample set X;
training a target attack model by using the collected original sample set X, and then obtaining a countermeasure sample X 'of each sample image X in the original sample image set X by using a white-box attack method to obtain a countermeasure sample set X';
finally, taking the original sample set X and the confrontation sample set X' as training data sets of the conditional standard flow model;
s2: constructing a conditional standard flow model, which comprises a convolutional neural network and a conditional GLOW model, wherein:
the convolutional neural network is used for extracting image characteristics from an input image, and inputting the obtained characteristics serving as condition variables into a conditional GLOW model;
the conditional GLOW model is used for coding the countermeasure sample of the input image according to the condition variable of the convolutional neural network to obtain a corresponding hidden space representation; the conditional GLOW model is formed by stacking L-1 flow components, an squeeze layer and K flow blocks, wherein each flow component is formed by stacking an squeeze layer, K flow blocks and a split layer, and the values of L and K are determined according to requirements; the stream block is a conditional stream block and comprises an Actnorm layer, a 1 × 1 convolutional layer and an affine coupling layer, wherein:
the Actnorm layer is used for activating and standardizing the input features and inputting the obtained features into the 1 multiplied by 1 convolutional layer;
the 1 × 1 convolution layer is used for performing 1 × 1 convolution processing on the input features and inputting the obtained features into the affine coupling layer;
the affine coupling layer is used for receiving the characteristics sent by the convolutional layer and the condition variables sent by the convolutional neural network, and outputting the characteristics and the condition variables after affine coupling processing;
s3: training the conditional standard flow model according to the training sample set obtained in the step S1;
s4: after the training of the conditional standard flow model is finished, sequentially inputting each training sample in the training sample set into the trained conditional standard flow model to obtain the implicit space representation corresponding to each training sample, calculating to obtain the mean value mu and the variance sigma of the implicit space representations of all the training samples to obtain the distribution N (mu, sigma) of the implicit space representations 2 )；
S5: when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image
Then extracting the input image by a convolution neural network in a conditional standard flow model
Condition variable of
According to the condition variable
For the distribution N (. mu.,. sigma.) obtained in step S4 2 ) Sampling to obtain hidden space representation
Represented by conditional GLOW model in conditional standard flow model according to hidden space
And condition variable
Performing reverse reasoning to obtain input image
Initial confrontation sample of
Then using a clipping function to pair the initial confrontation samples
Processing to obtain final confrontation sample
The formula is as follows:
wherein Clip () represents a preset clipping function, and epsilon represents a preset disturbance parameter.
The invention relates to a black box countermeasure sample generation method based on a condition standard flow model, which comprises the steps of firstly generating countermeasure samples corresponding to original images by using a white box attack method, integrating the original sample set and the corresponding countermeasure sample set to obtain a training data set, and then constructing the condition standard flow model comprising a convolutional neural network and a condition GLOW model, wherein the convolutional neural network takes the original samples as input to extract image characteristics as condition variables, and the condition GLOW model takes the condition variables and the countermeasure samples corresponding to the original images as input codes to obtain corresponding hidden space representations; training a conditional standard flow model using a training data set; and obtaining the distribution of the implicit space representation of the training sample set after the training is finished, then sampling the distribution of the implicit space representation by taking the characteristics extracted from the clean image as condition variables to obtain the output of the confrontation sample, and finally cutting the output confrontation sample to obtain the final confrontation sample.
When the corresponding samples are generated aiming at the specific samples, the method does not depend on a large amount of computing resources and computing time and can generate the confrontation samples in batches, the problems that the conventional black box attack method needs high computing amount, is high in time waiting and cannot be produced in mass are solved to a great extent, and the performances in the aspects of attack success rate, average query times, generated confrontation sample mobility and the like are greatly improved. The black box confrontation sample generation method based on the conditional standard flow model has important values for research work of confrontation sample communities and robustness improvement of the existing artificial intelligence system.
Drawings
FIG. 1 is a schematic diagram of a black box versus sample generation method based on a conditional standard flow model according to the present invention;
FIG. 2 is a flow chart of an embodiment of the black box confrontation sample generation method based on the conditional standard flow model according to the present invention;
FIG. 3 is an architecture diagram of a conditional standard flow model in the present invention;
FIG. 4 is a block diagram of a conditional flow block according to the present invention;
FIG. 5 is a flow chart of the training of the conditional standard flow model in the present embodiment;
FIG. 6 is a comparison graph of the distribution of successful attack query times on a CIFAR-10 data set by the present invention and the comparison method;
FIG. 7 is a comparison graph of the distribution of successful attack queries on an SVHN dataset according to the present invention and the comparison method;
FIG. 8 is a graph comparing the confrontational sample transferability of the present invention and AdvFlow algorithm on a CIFAR-10 dataset;
fig. 9 is a graph comparing the confrontational sample transferability of the present invention and the AdvFlow algorithm on SVHN datasets.
Detailed Description
The following description of the embodiments of the present invention is provided in order to better understand the present invention for those skilled in the art with reference to the accompanying drawings. It is to be expressly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the subject matter of the present invention.
Examples
To better explain the technical solution of the present invention, first, the principle on which the present invention is based will be briefly explained. The conventional attack method generates a final countermeasure sample by performing a complicated inference based on a target model to generate a countermeasure disturbance, which is then added to the original sample. This process is highly dependent on the reasoning results, which can consume a large computational cost, and typically generates a single "best" sample according to some criteria. However, the present invention recognizes that all challenge samples may follow a particular distribution that is inconsistent with a normal distribution. This is mainly due to the fact that the training data involved in the optimization of the different depth models is fixed. In other words, the training data distribution characterizes the fixed distribution approximated by these models during training, and therefore, the influence of the data distribution on the model that is not visible in training is also present. This explains that the challenge sample (most of which are data not visible to the model in training) follows a misaligned distribution. It is because the normal sample and the challenge sample data exhibit similar appearances, and therefore it is considered that the two distributions overlap each other and it is reasonable to assume that the two distributions can be transformed into each other.
Based on the analysis, the invention provides a black box confrontation sample generation method based on a conditional standard flow model. FIG. 1 is a schematic diagram of a black box versus sample generation method based on a conditional standard flow model according to the present invention. As shown in fig. 1, the present invention collects a large number of antagonistic samples by the existing white-box attack method. Although these samples look similar to normal samples, direct conversion between the two samples is still difficult and even prohibitive. This is because small perturbations may be overwhelmed by the complex structures and textures in the common sample and are therefore insensitive to the generation model. To alleviate this problem, it is believed that small perturbations should be conditioned on normal inputs, which provide clues in the generation process. In particular, the conditional standard flow model is used to implement a conditional generation process that allows for the synthesis of antagonistic samples based on normal samples and random variables. The random variable can diversify the generated samples, that is, when the conditional standard flow model is trained, random sampling can be performed in a hidden space to generate a batch of noises, and the noises are reversely inferred by the conditional standard flow model, so that a batch of confrontation samples are generated.
FIG. 2 is a flow chart of an embodiment of the black box countermeasure sample generation method based on the conditional standard flow model. As shown in fig. 2, the black box confrontation sample generation method based on the conditional standard flow model of the present invention specifically includes the following steps:
s201: raw sample and its corresponding challenge sample collection:
acquiring a plurality of images and corresponding labels according to actual needs, and normalizing each image to a preset size to be used as an original sample, thereby obtaining an original sample set X.
Training a target attack model by adopting the collected original sample set X, and then obtaining a countermeasure sample X 'of each sample image X in the original sample image set X by using a white-box attack method to obtain a countermeasure sample set X'.
Finally, the original sample set X and the confrontation sample set X' are used as training data sets of the conditional standard flow model.
In this embodiment, the original sample set adopts the existing data set: CIFAR-10, SVHN and ImageNet. The specific method for obtaining the challenge sample is as follows: for both CIFAR-10 and SVHN datasets, the target attack model, here the ResNet-56 network, is trained using its training set, and then the countermeasure sample is generated using the PGD (Projected gradient Degreet) algorithm. Whereas for ImageNet, approximately 30,000 images were selected from its validation set to train the model, based on the Multi-model Integrated MI-FGSM (Momentum Iterative Fast Gradient notation) Method, where the integrated models are Inception V4, Inception ResnetV2, and ResNetV 2-101. In the present embodiment, two types of disturbance parameters are used for generating the countermeasure model, where ∈ 8 and ∈ 16, respectively.
S202: constructing a condition standard flow model:
in order to improve the capability of processing image textures, a conditional standard flow model is constructed based on a GLOW model in the embodiment. The original Glow model relates to convolution, coupling and normalization operations in model construction, but because the original Glow model does not consider conditions in probability modeling, the invention improves the Glow model, and appropriately integrates image contents into condition variables, thereby obtaining the condition standard flow model.
FIG. 3 is an architecture diagram of a conditional standard flow model in the present invention. As shown in fig. 3, the conditional standard flow model in the present invention includes a convolutional neural network and a conditional GLOW model, where:
the convolutional neural network is used for extracting image features of the input image, and the obtained features are used as condition variables to be input into the conditional GLOW model. Convolutional neural networks were introduced because they can only provide very low-level features if the original input image is taken as a condition variable. And these features are not enough for feature modeling and can burden sub-networks in the affine coupling layer of the conditional GLOW model, so the present invention extracts higher-level features from the original input image through the convolutional neural network. In the embodiment, the convolutional neural network adopts a pre-trained VGG-19 model, and the features output by the last stage of convolutional layer are taken as the extracted features. In this embodiment, the data sets used in training the convolutional neural network are CIFAR-10, SVHN, and ImageNet. In the whole training process of the GLOW model, the convolutional neural network can be used for fixing parameters and can also be used for training and optimizing.
The conditional GLOW model is used for coding the confrontation samples of the input image according to the condition variables of the convolutional neural network to obtain corresponding implicit space representation. As shown in fig. 3, the conditional GLOW model of the present invention is similar to the conventional GLOW model, and is composed of L-1 flow components, squeeze layers (compression layers), and K flow block stacks, where the flow components are composed of one squeeze layer, K flow blocks, and one split layer (split layer) stack, and the values of L and K are determined as needed. Unlike the conventional GLOW model, the stream block in the present invention takes conditions into consideration, and thus may be referred to as a conditional stream block. FIG. 4 is a block diagram of a conditional flow block according to the present invention. As shown in fig. 4, the conditional stream block in the present invention includes an Actnorm (activation normalization) layer, a 1 × 1 convolutional layer, and an affine coupling layer, where:
the Actnorm layer is used to activate and normalize the input features, and the obtained features are input to the 1 × 1 convolutional layer.
The 1 × 1 convolution layer is used to perform 1 × 1 convolution processing on the input features, and the obtained features are input to the affine coupling layer.
And the affine coupling layer is used for receiving the characteristics sent by the convolutional layer and the condition variables sent by the convolutional neural network, carrying out affine coupling processing and then outputting.
S204: training condition standard flow model:
and training the conditional standard flow model according to the training sample set obtained in the step S201.
In order to improve the training effect of the model, the training process of the conditional standard flow model is optimized in this embodiment. Fig. 5 is a training flowchart of the conditional standard flow model in the present embodiment. As shown in fig. 5, the specific training steps of the conditional standard flow model in this embodiment include:
s501: selecting a current batch of training samples:
b training samples are randomly selected from the training sample set to serve as training samples of the current batch, wherein B represents the batch size, and the value of B is determined according to actual needs.
S502: and judging whether the iteration round t meets a preset condition, if so, entering a step S503, otherwise, entering a step S504. In this embodiment, the loss function is alternatively executed by using two calculation methods, so that a preset condition is required to switch the calculation methods of the loss function, for example, the implicit spatial loss may be calculated in odd number of rounds, and the image loss may be calculated in even number of rounds.
S503: calculating the hidden space loss:
calculating the implicit space loss L (theta; z, x ', c) of each training sample in the current batch, and then averaging to obtain the implicit space loss of the current batch, wherein the calculation formula of the implicit space loss L (theta; z, x', c) is as follows:
wherein, x and x' respectively represent the input original sample and the corresponding confrontation sample, c represents the condition variable extracted from the input image x by the convolution neural network, p z () Represents the distribution, f θ (x '; c) a conditional GLOW model with a network parameter theta obtains a hidden spatial representation z according to the input confrontation sample x' and the condition variable c,
represents jacobian, | | | represents the absolute value of the solution.
S504: calculating the image loss:
since the task of countering the sample attack is to generate a countering sample that has a similar appearance to the input conditional sample, it is ensured that the generation of the input image x from the hidden spatial representation z does not lead to unexpected results. To achieve this, this embodiment also introduces MSE (Mean Square Error) loss during the training process. That is, randomly extracting a batch of training samples from the current batch, and calculating the image loss L of each extracted training sample MSE (theta; z, c) and then averaging to obtain the image loss L of the current batch MSE The calculation formula of (θ; z, c) is as follows:
L MSE (θ；z,c)＝||f θ (z；c)-x′|| 2
wherein f is θ (z; c) representing a confrontation sample obtained by reversely reasoning a conditional GLOW model with a network parameter theta according to a hidden space representation z and a condition c, | | | | | | sweet wind 2 The two norms are found.
S505: updating model parameters:
and updating the parameters of the conditional standard flow model according to the loss obtained by current calculation. In this embodiment, an Adam optimization algorithm is used to update and optimize the model parameters.
S506: and judging whether a training end condition is reached, if so, ending the training, otherwise, returning to the step S501.
The training end conditions generally have two types, one is that the maximum iteration times are reached, or the condition standard flow model is converged, and the conditions are selected according to requirements in practical application.
S204: generating a distribution of the hidden spatial representation:
when the conditional standard flow model is trained, then the implicit spatial representation of each challenge sample in the training sample set is expected to follow the assumed gaussian distribution N (0, 1). In practice it has been found that these implicit spatial representations have changed the mean and standard deviation values. This may be because there is insufficient training data and image loss may bias the center of the gaussian distribution, but experiments have shown that even without image loss, a shift occurs. Based on this observation, sampling based on shifted mean and standard values to get a hidden spatial representation may lead to better performance than sampling from N (0, 1).
According to the analysis, after the training of the conditional standard flow model is completed, the training samples in the training sample set are sequentially input into the trained conditional standard flow model to obtain the implicit space representation corresponding to each training sample, the mean value mu and the variance sigma of the implicit space representations of all the training samples are obtained through calculation, and the distribution N (mu, sigma) of the implicit space representation is obtained 2 )。
S205: generating a new challenge sample:
when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image
Then extracting the input image by a convolution neural network in a conditional standard flow model
Condition variable of
According to the condition variable
For the distribution N (μ, σ) obtained in step S204 2 ) Sampling to obtain hidden space representation
Represented by conditional GLOW model in conditional standard flow model according to hidden space
And condition variable
Performing reverse reasoning to obtain input image
Initial confrontation sample of
Then using a clipping function to pair the initial confrontation samples
Processing to obtain final confrontation sample
The formula is as follows:
clip () represents a preset clipping function, and epsilon represents a preset disturbance parameter.
In order to better illustrate the technical effects of the invention, the invention is experimentally verified by using specific examples. In the experimental verification, a SimBA algorithm and an AdvFlow algorithm are used as comparison methods, and the comparison is carried out on the aspects of 1) attack success rate, 2) average query times and 3) mobility of the generated confrontation sample.
1) Attack success rate and average number of queries
The method of the invention aims to keep higher attack success rate when the number of times of inquiry is reduced. In the experimental verification, the maximum query number is respectively set to 100, 200, 300, 400, 500 and 1000, the attack is successful only in the predefined query number, otherwise, the attack fails. Table 1 is a comparison table of attack success rate and average query times of the CIFAR-10 data set by the present invention and the comparison method when the perturbation epsilon is 8.
TABLE 1
Table 2 is a comparison table of attack success rate and average query times of the CIFAR-10 data set by the present invention and the comparison method when the perturbation epsilon is 16.
TABLE 2
Table 3 is a comparison table of attack success rate and average query times of SVHN data set by the present invention and the comparison method when the perturbation epsilon is 8.
TABLE 3
Table 4 is a comparison table of attack success rates and average query times of the SVHN data set by the present invention and the comparison method when the perturbation epsilon is 16.
TABLE 4
From tables 1 to 4, it can be seen that, in most cases, the attack success rate of the present invention is higher than that of 2 comparison methods, and it can be seen that the generation model provided by the present invention can generate effective countermeasure samples. It should be noted that the average number of queries required by the present invention is much smaller than the average number of queries required by 2 comparison methods.
Experiments on the ImageNet dataset are more challenging because their data are much more complex than the CIFAR-10 dataset and the SVHN dataset. Table 5 is a table comparing the attack success rate and average query times of the ImageNet dataset for the present invention and the comparison method when the perturbation epsilon is 16.
TABLE 5
As shown in table 5, it can be seen that the attack success rate and the average query number of the ImageNet data set of the present invention have significant advantages.
2) Distribution of query times
To better observe the advantage of the present invention in terms of number of queries, this experiment verified that histograms were plotted for the number of queries that performed successful attacks on the CIFAR-10 dataset and the SVHN dataset, where ShuffleNet V2 and VGG-16 were used as the target attack models for CIFAR-10 and SVHN, respectively, with a maximum number of queries limited to 500. FIG. 6 is a comparison graph of the distribution of successful attack queries on a CIFAR-10 data set by the present invention and the comparison method. Fig. 7 is a comparison graph of the distribution of successful attack queries on SVHN datasets for the present invention and comparison method. As shown in fig. 6 and 7, the present invention can perform a successful attack based on most samples only once in all cases. At a perturbation ε of 16, the median number of queries of the present invention for the CIFAR-10 and SVHN datasets were only 19.41 and 23.67, respectively. It is noted that on the target attack model, ShuffleNetV2, the present invention has attack success rates of 88% and 90% within a few queries, respectively, when ∈ 8 and ∈ 16. In contrast, the AdvFlow algorithm and SimBA algorithm typically require hundreds of queries to attack success, while a small number of queries (e.g., ≦ 100) do not allow these methods to work properly. The results show that the number of bits in the present invention is 1 in all cases, which fully verifies the advancement of the present invention.
3) Transferability of generated challenge samples
Current black-box attacks rely heavily on the transferability assumption of the challenge samples, i.e., the challenge samples generated from a particular model can be used to attack other different models. To prove that this assumption is valid, the experimental validation explored the model-based migratability of the generated challenge samples on the CIFAR-10 dataset and SVHN dataset.
Specifically, the experiment verifies that 8 models are selected, including ResNet-56, VGG-16, VGG-19, ShuffleNet V2, MobileNet V2, InceptionV3, DenseNet-169 and GoogleLeNet. In each model case, the model is first trained until the best performance (typically over 90%) is obtained on the test set. 1000 images were then randomly selected from the test set, correctly classified by the model, and the corresponding challenge samples were misclassified. The generated challenge samples are used to attack his model. For fair comparison, let ε be 16 and the maximum number of queries be 500. The invention compares with the AdvFlow algorithm in a non-target black box attack. FIG. 8 is a graph comparing the confrontational sample transferability of the present invention and the AdvFlow algorithm on a CIFAR-10 dataset. Fig. 9 is a graph comparing the confrontational sample transferability of the present invention and the AdvFlow algorithm on SVHN datasets. As shown in fig. 8 and 9, where each row represents which model was targeted when the confrontation sample was generated, and each column represents which model was attacked by the generated sample. It can be seen that the migratability ASR of the present invention on the CIFAR-10 dataset is from 33.6% to 79.6%, while the AdvFlow algorithm is 3.4% to 13.0%. This means that the samples generated by the invention produce higher ASR on other models (approximately 30% -66% higher in most cases) than the AdvFlow algorithm, validating the excellent migratability of the invention. This is because the AdvFlow algorithm relies heavily on feedback from the target model during each query and cannot extract migratable features. In contrast, the present invention learns the distribution of antagonistic samples that will not fit to a particular model.
4) Image independence attacks
In order to evaluate the performance of the invention on samples with different semantics, an attack experiment was performed on other datasets than the training ImageNet dataset. Specifically, the test data set includes VOC 2007, VOC 2012, PlasceS565, Caltech101 and Caltech 256. Target attack models include VGG-19, Inception V3, ResNet-152, and WideResNet-50, all of which are implemented in PyTorch. Table 6 is a statistical table of attack results of different target attack models in different test data sets according to the present invention.
| VOC2007 | VOC2012 | Plasces365 | Caltech101 | Catech256 | |
| VGG-19 | 91.7 | 93.0 | 90.9 | 93.5 | 86.3 |
| Inception_v3 | 87.5 | 90.8 | 91.1 | 93.6 | 86.4 |
| ResNet-152 | 85.1 | 89.2 | 87.3 | 94.4 | 83.8 |
| WideResNet-50 | 86.1 | 89.7 | 84.1 | 93.4 | 84.1 |
TABLE 6
As shown in table 6, the invention trained on ImageNet data sets can be used to generate valid challenge samples on other data sets without retraining. In some cases, the attack success rate may exceed 90%, with a maximum query size limit of 100.
Although the illustrative embodiments of the present invention have been described in order to facilitate those skilled in the art to understand the present invention, it is to be understood that the present invention is not limited to the scope of the embodiments, and that various changes may be made apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined in the appended claims, and all matters of the invention using the inventive concepts are protected.
Claims (3)
1. A black box confrontation sample generation method based on a conditional standard flow model is characterized by comprising the following steps:
s1: acquiring a plurality of images and corresponding labels according to actual needs, and normalizing each image to a preset size to be used as an original sample, thereby obtaining an original sample set X;
training a target attack model by using the collected original sample set X, and then obtaining a countermeasure sample X 'of each sample image X in the original sample image set X by using a white-box attack method to obtain a countermeasure sample set X';
finally, taking the original sample set X and the confrontation sample set X' as training data sets of the conditional standard flow model;
s2: constructing a conditional standard flow model comprising a convolutional neural network and a conditional GLOW model, wherein:
the convolutional neural network is used for extracting image characteristics from an input image, and inputting the obtained characteristics serving as condition variables into a conditional GLOW model;
the conditional GLOW model is used for coding the countermeasure sample of the input image according to the condition variable of the convolutional neural network to obtain a corresponding hidden space representation; the conditional GLOW model is formed by stacking L-1 flow components, an squeeze layer and K flow blocks, wherein each flow component is formed by stacking an squeeze layer, K flow blocks and a split layer, and the values of L and K are determined according to requirements; the stream block is a conditional stream block and comprises an Actnorm layer, a 1 × 1 convolutional layer and an affine coupling layer, wherein:
the Actnorm layer is used for activating and standardizing input features and inputting the obtained features into the 1 × 1 convolutional layer;
the 1 × 1 convolution layer is used for performing 1 × 1 convolution processing on the input features and inputting the obtained features into the affine coupling layer;
the affine coupling layer is used for receiving the characteristics sent by the convolutional layer and the condition variables sent by the convolutional neural network, and outputting the characteristics and the condition variables after affine coupling processing;
s3: training the conditional standard flow model according to the training sample set obtained in the step S1;
s4: after the training of the conditional standard flow model is finished, sequentially inputting each training sample in the training sample set into the trained conditional standard flow model to obtain the implicit space representation corresponding to each training sample, calculating to obtain the mean value mu and the variance sigma of the implicit space representations of all the training samples to obtain the distribution N (mu, sigma) of the implicit space representations 2 )；
S5: when a countermeasure sample needs to be generated for a new input image, the input image is normalized to a preset size to obtain the input image
Then extracting the input image by a convolution neural network in a conditional standard flow model
Condition variable of
According to the condition variable
For the distribution obtained in step S4
Sampling to obtain hidden space representation
Represented by conditional GLOW model in conditional standard flow model according to hidden space
And condition variable
Carrying out reverse reasoning to obtain an input image x * Initial confrontation sample of
Then using a clipping function to pair the initial confrontation samples
Processing to obtain final confrontation sample
The formula is as follows:
wherein Clip () represents a preset clipping function, and epsilon represents a preset disturbance parameter.
2. The black-box countermeasure sample generation method of claim 1, wherein the convolutional neural network is a VGG-19 model, and features output by the last convolutional layer are taken as extracted features.
3. The black-box countermeasure sample generation method according to claim 1, wherein the training method of the conditional standard flow model in step S3 is as follows:
s3.1: randomly selecting B training samples from a training sample set as training samples of a current batch, wherein B represents the batch size, and the value of B is determined according to actual needs;
s3.2: judging whether the iteration round t meets a preset condition, if so, entering a step S3.3, otherwise, entering a step S3.4;
s3.3: calculating the implicit space loss L (theta; z, x ', c) of each training sample in the current batch, and then averaging to obtain the implicit space loss of the current batch, wherein the calculation formula of the implicit space loss L (theta; z, x', c) is as follows:
wherein, x and x' respectively represent the input original sample and the corresponding confrontation sample, c represents the condition variable extracted from the input image x by the convolution neural network, p z () The distribution is represented by a distribution of the,
the conditional GLOW model with the network parameter theta obtains a hidden space representation z according to the input confrontation sample x' and the condition variable c,
expressing a Jacobian, and | l represents the absolute value;
s3.4: randomly extracting a batch of training samples from the current batch, and calculating the image loss L of each extracted training sample MSE (theta; z, c) and then averaging to obtain the image loss of the current batchImage loss L MSE The calculation formula of (θ; z, c) is as follows:
L MSE (θ；z,c)＝||f θ (z；c)-x′|| 2
wherein f is θ (z; c) representing a confrontation sample obtained by carrying out reverse reasoning on a conditional GLOW model with a network parameter theta according to the implicit space representation z and the condition c;
s3.5: updating the parameters of the conditional standard flow model according to the loss obtained by current calculation;
s3.6: and judging whether the training end condition is reached, if so, ending the training, otherwise, returning to the step S3.1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210310612.5A CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210310612.5A CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114898168A true CN114898168A (en) | 2022-08-12 |
| CN114898168B CN114898168B (en) | 2024-05-17 |
Family
ID=82716155
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210310612.5A Active CN114898168B (en) | 2022-03-28 | 2022-03-28 | Black box countermeasure sample generation method based on conditional standard flow model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114898168B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801297A (en) * | 2021-01-20 | 2021-05-14 | 哈尔滨工业大学 | Machine learning model adversity sample generation method based on conditional variation self-encoder |
| CN113674140A (en) * | 2021-08-20 | 2021-11-19 | 燕山大学 | Physical countermeasure sample generation method and system |
| CN114066912A (en) * | 2021-11-23 | 2022-02-18 | 中国人民解放军战略支援部队信息工程大学 | Intelligent countermeasure sample generation method and system based on optimization algorithm and invariance |
| US20220092336A1 (en) * | 2020-03-26 | 2022-03-24 | Shenzhen Institutes Of Advanced Technology | Adversarial image generation method, computer device, and computer-readable storage medium |
-
2022
- 2022-03-28 CN CN202210310612.5A patent/CN114898168B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220092336A1 (en) * | 2020-03-26 | 2022-03-24 | Shenzhen Institutes Of Advanced Technology | Adversarial image generation method, computer device, and computer-readable storage medium |
| CN112801297A (en) * | 2021-01-20 | 2021-05-14 | 哈尔滨工业大学 | Machine learning model adversity sample generation method based on conditional variation self-encoder |
| CN113674140A (en) * | 2021-08-20 | 2021-11-19 | 燕山大学 | Physical countermeasure sample generation method and system |
| CN114066912A (en) * | 2021-11-23 | 2022-02-18 | 中国人民解放军战略支援部队信息工程大学 | Intelligent countermeasure sample generation method and system based on optimization algorithm and invariance |
Non-Patent Citations (2)
| Title |
|---|
| PENG, WENYU等: "EnsembleFool: A method to generate adversarial examples based on model fusion strategy", COMPUTERS & SECURITY, 7 July 2021 (2021-07-07) * |
| 刘西蒙;谢乐辉;王耀鹏;李旭如;: "深度学习中的对抗攻击与防御", 网络与信息安全学报, no. 05, 13 October 2020 (2020-10-13) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114898168B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021189364A1 (en) | Method and device for generating adversarial image, equipment, and readable storage medium | |
| CN111881935B (en) | Countermeasure sample generation method based on content-aware GAN | |
| CN111475797B (en) | Method, device and equipment for generating countermeasure image and readable storage medium | |
| CN110941794A (en) | Anti-attack defense method based on universal inverse disturbance defense matrix | |
| CN111598210B (en) | Anti-attack defense method for anti-attack based on artificial immune algorithm | |
| CN111507384B (en) | Method for generating confrontation sample of black box depth model | |
| CN112784929B (en) | Small sample image classification method and device based on double-element group expansion | |
| CN112200243B (en) | Black box countermeasure sample generation method based on low query image data | |
| CN112836798A (en) | Non-directional white-box attack resisting method aiming at scene character recognition | |
| CN113704758B (en) | Black box attack countermeasure sample generation method and system | |
| CN114758198A (en) | Black box attack method and system for resisting disturbance based on meta-learning | |
| CN113627543B (en) | Anti-attack detection method | |
| CN113033822A (en) | Antagonistic attack and defense method and system based on prediction correction and random step length optimization | |
| CN113935396A (en) | Manifold theory-based method and related device for resisting sample attack | |
| CN112861759B (en) | Method and device for generating confrontation sample | |
| CN111737688B (en) | Attack defense system based on user portrait | |
| Zhou et al. | Improving robustness of random forest under label noise | |
| CN113034332A (en) | Invisible watermark image and backdoor attack model construction and classification method and system | |
| CN114898168B (en) | Black box countermeasure sample generation method based on conditional standard flow model | |
| CN115510986A (en) | Countermeasure sample generation method based on AdvGAN | |
| CN113379593B (en) | Image generation method, system and related equipment | |
| CN113159317B (en) | Antagonistic sample generation method based on dynamic residual corrosion | |
| CN115270891A (en) | Method, device, equipment and storage medium for generating signal countermeasure sample | |
| CN113283520A (en) | Member reasoning attack-oriented depth model privacy protection method and device based on feature enhancement | |
| Yang et al. | Weight-based regularization for improving robustness in image classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |