CN115527084A - Intelligent system confrontation sample generation method and system based on diversified input strategy - Google Patents

Abstract

The invention belongs to the technical field of image model evaluation, and particularly relates to an intelligent system confrontation sample generation method and system based on diversified input strategies, wherein firstly, an image transformation method is collected, and a data enhancement method set is constructed according to the collected image transformation method; selecting a plurality of image transformation methods from the data enhancement method set to form a data enhancement strategy set; respectively performing image transformation by using an image transformation method in a data enhancement strategy set aiming at original image sample data to obtain image enhancement data after batch transformation; and acquiring gradient values of the image enhancement data corresponding to each image transformation method by using the image classification model, acquiring anti-disturbance noise by performing weighted average on the gradient values, and generating an anti-sample by using the anti-disturbance noise. According to the method, image data is enhanced by multiple random transformations of different image transformation methods, the disturbance noise in the resisting sample is calculated by using the model gradient value, the variety of data enhancement transformation types and the number of input images are improved, the phenomenon of overfitting is effectively relieved, and the black box attack capacity of the resisting sample is improved.

Description

Intelligent system confrontation sample generation method and system based on diversified input strategy

Technical Field

The invention belongs to the technical field of image model evaluation, and particularly relates to an intelligent system countermeasure sample generation method and system based on diversified input strategies.

Background

The neural network has wide application due to good performance, and brings great changes to the life of people in a plurality of scenes such as automatic driving, face recognition, natural language processing and the like. In the field of image classification, when people cannot extract effective information from images with disordered contents, the deep neural network can still finish classification tasks quickly and accurately. However, image classification models are often vulnerable to "sample-fighting" attacks due to their vulnerability. The challenge swatch is a challenge image formed by adding a subtle, subtle perturbation to the original image.

According to the degree of understanding of the model knowledge by an attacker, the attack of the countersample can be divided into white box attack and black box attack, and the attack success rate is low because the black box attack is an attack behavior developed under the condition that the structural parameters of the model are not solved. The challenge sample has the characteristic of mobility, namely the challenge sample generated aiming at the A model also has certain attack capability on the B model, so that a plurality of researchers utilize the property to improve the black box attack performance of the challenge sample. Since the introduction of FGSM (Fast Gradient signal Method), researchers have mainly boosted the ability to fight migratory attacks on samples from two directions: firstly, a better optimization algorithm is searched, and secondly, a training set is expanded by adopting a data enhancement mode: compared to BIM (Basic Iterative Method), the original version of FGSM has better mobility due to single step generation. Then, if momentum is introduced into the generation process of the countermeasure sample, the updating direction of the loss function in the generation process of the countermeasure sample is corrected, and the success rate of black box attack is improved; if data enhancement is introduced into the generation process of the confrontation sample, the input diversity is improved, overfitting in the generation process of the confrontation sample is effectively relieved, and the success rate of black box attack on the confrontation sample is improved; in another example, by using translation invariance, training samples are enriched by expanding the number of images, so that the resisting samples have better attack characteristics.

The data enhancement is widely applied, the specific methods are not limited to random scaling and translation transformation, and common operations include clipping, rotation, color gamut transformation and the like, so that the data enhancement technology is difficult to be utilized to the maximum extent to improve the mobility of the resisting sample only by using a specific method. Meanwhile, researches show that randomness caused by the random transformation process of the image is not always beneficial to improving the mobility of the resisting sample, and sometimes useless gradients can reduce the attack characteristics of the resisting sample.

Disclosure of Invention

Therefore, the invention provides an intelligent system countermeasure sample generation method and system based on a diversified input strategy, which can enhance image data by utilizing multiple random transformations of different image transformation methods, calculate disturbance noise in the countermeasure sample by utilizing a model gradient value, improve the diversity of data enhancement transformation types and input image quantity, effectively relieve the phenomenon of 'overfitting', and improve the black box attack capability of the countermeasure sample.

According to the design scheme provided by the invention, an intelligent system confrontation sample generation method based on a diversified input strategy is provided, and comprises the following contents:

collecting image transformation methods, and constructing a data enhancement method set according to the collected image transformation methods;

selecting a plurality of image transformation methods from the data enhancement method set to form a data enhancement strategy set according to the attack success rate of the pre-attack experiment;

respectively carrying out image transformation by using an image transformation method in a data enhancement strategy set aiming at original image sample data to obtain image enhancement data after batch transformation;

and acquiring gradient values of the image enhancement data corresponding to each image transformation method by using the image classification model, acquiring anti-disturbance noise by performing weighted average on the gradient values, and generating an anti-sample by using the anti-disturbance noise.

As the method for generating the confrontation sample of the intelligent system based on the diversified input strategy in the invention, further, the collected image transformation method includes but is not limited to: the image random scaling, the image translation transformation, the image clipping, the image rotation and the image color gamut transformation can be dynamically added and deleted according to the latest data enhancement method.

As the method for generating the confrontation sample of the intelligent system based on the diversified input strategy, further, in the method set for constructing the data enhancement, a transformation range for preventing the image transformation distortion is set, and each image transformation method corresponds to a respective transformation range.

As the method for generating the confrontation sample of the intelligent system based on the diversified input strategy, further, the data enhancement integration strategy in the data enhancement strategy set is expressed as follows:

wherein, T _i (x) Image transformation method denoted by reference numeral i, ω _i The weight coefficients corresponding to the image transformation methods in the strategy space, and n is the number of image transformation methods selected from the data enhancement method set.

As the intelligent system confrontation sample generation method based on the diversified input strategy, N rounds of loop iteration are further set and a back propagation process is utilized to generate a final confrontation sample, wherein N is a preset maximum iteration number.

The method comprises the steps of firstly obtaining sample data to be adjusted of current iteration in N rounds of loop iteration, if the current iteration is first iteration, the sample data to be adjusted is original image sample data, otherwise, the sample data is a countermeasure sample generated in the previous round, then respectively carrying out image transformation on the sample data to be adjusted by using an image transformation method in a data enhancement strategy set, inputting the enhanced image data after the image transformation into an image classification model for processing, obtaining a loss function gradient value of the corresponding enhanced image data according to the image classification model, then carrying out weighted average on the loss gradient value according to weight to generate countermeasure noise in the current iteration, and generating the countermeasure sample in the current iteration by adding the countermeasure noise to the sample data to be adjusted.

As the method for generating the confrontation sample of the intelligent system based on the diversified input strategy, further, the calculation process of the gradient value of the loss function is expressed as follows:

wherein L () is a cross entropy loss function,

image enhancement data, P, corresponding to the image transformation method denoted i _i The data corresponding to the image transformation method with the index i is used for enhancing the transformation probability, y is an image label, and theta is a model parameter.

Furthermore, the invention also provides an intelligent system confrontation sample generation system based on the diversified input strategy, which comprises the following steps: a strategy collection module, a strategy selection module, an enhancement transformation module and a sample generation module, wherein,

the strategy collection module is used for collecting the image transformation methods and constructing a data enhancement method set according to the collected image transformation methods;

the strategy selection module is used for selecting a plurality of image transformation methods from the data enhancement method set to form a data enhancement strategy set according to the attack success rate of the pre-attack experiment;

the enhancement transformation module is used for respectively carrying out image transformation by utilizing an image transformation method in the data enhancement strategy set aiming at original image sample data to obtain image enhancement data after batch transformation;

and the sample generation module is used for acquiring gradient values of the image enhancement data corresponding to the image transformation methods by using the image classification model, acquiring the anti-disturbance noise by performing weighted average on the gradient values, and generating the anti-sample by using the anti-disturbance noise.

The invention has the beneficial effects that:

the method generates the confrontation sample based on the diversified input strategy, enhances the variety of the image data, effectively balances the useless gradient and relieves the overfitting in the generation process of the confrontation sample. The generation reason of 'overfitting' is analyzed by comparing the generation process of the countermeasure sample with the training process of the neural network, a plurality of image transformation modes are introduced, a batch of transformed images are simultaneously input into the countermeasure sample generation system, the weighted average value of the gradient of the image loss function after the batch of transformed images is calculated, and then the noise is resisted based on the weighted average value, and then the noise is iteratively added to the original image to generate the countermeasure sample, so that the success rate of black box attack of the countermeasure sample is greatly improved. Through the super-parameter adjustment, the conversion and combination among different attack methods can be realized, and the attack mobility in the process of generating and resisting the sample is further improved. Experiments on ImageNet data sets show that compared with a benchmark method, the scheme can improve the attack success rate by 12.3%.

Description of the drawings:

FIG. 1 is a schematic diagram of a process for generating a challenge sample in an embodiment;

FIG. 2 is an example of confrontational perturbation and confrontational sample in an embodiment;

FIG. 3 is a flow chart of a countermeasure sample generation algorithm in an embodiment;

FIG. 4 is a diagram illustrating a conversion relationship between different countermeasure sample generation methods according to an embodiment;

FIG. 5 is a graph illustrating the effect of the diversified input strategy on mobility gain in the example;

FIG. 6 is an image comparison of each stage in the generation of the confrontation sample in the embodiment;

FIG. 7 is a schematic diagram showing the influence of the random transformation probability on the attack success rate in the embodiment;

fig. 8 is a schematic diagram illustrating the influence of the weight coefficient on the attack success rate in the embodiment.

The specific implementation mode is as follows:

in order to make the objects, technical solutions and advantages of the present invention clearer and more obvious, the present invention is further described in detail below with reference to the accompanying drawings and technical solutions.

The countersample is commonly used for evaluating and checking the safety and the robustness of the image classification model, the white-box attack of the countersample has already realized high attack success rate, but the black-box attack success rate is also low due to the existence of the overfitting. An embodiment of the present invention, as shown in fig. 1, provides a method for generating an intelligent system countermeasure sample based on a diversified input policy, including:

s101, collecting image transformation methods, and constructing a data enhancement method set according to the collected image transformation methods;

s102, selecting a plurality of image transformation methods from a data enhancement method set to form a data enhancement strategy set according to the attack success rate of the pre-attack experiment;

s103, respectively performing image transformation on the original image sample data by using an image transformation method in the data enhancement strategy set to obtain image enhancement data after batch transformation;

s104, obtaining gradient values of image enhancement data corresponding to each image transformation method by using the image classification model, obtaining anti-disturbance noise by performing weighted average on the gradient values, and generating an anti-sample by using the anti-disturbance noise.

As shown in fig. 2, after adding the antialiasing noise to the original image on the left side, the image classification model may misclassify the antialiasing sample image on the right side with high confidence. Although the inherent weakness of the deep neural network is exposed by the attack behavior of the countersample, the research on the attack behavior is beneficial to testing, evaluating and improving the robustness of the image classification model, and further promotes the landing and achievement transformation of the deep learning technology. The counterattack is an important research direction for counterchecking the samples, and is also an important security testing means before the deployment of the image classification model. In the white box attack and the black box attack, the black box attack is more consistent with the real world scene characteristics and has higher research value. At present, in the research aiming at enhancing the mobility of the resisting sample, strong black box attack means and better attack effect are achieved by introducing methods such as enhancing momentum, adjusting variance and the like. Furthermore, surrogate model-based and decision-based black box attack methods are also proposed. The attack based on the substitution model trains a substitution model according to the analyzed response result through multiple visits to the target model, thereby converting the black box attack into the white box attack. In a decision-based black-box attack, an attacker tends to generate countermeasure samples by evaluating the gradient and decision boundaries of the target model. However, these methods often require a large amount of access to the black box model, and are easy to monitor and capture, while the migration-based attack is naturally hidden, fast and low-cost, and therefore, the embodiment scheme is mainly developed around the migration-based black box attack.

Data plays an important driving role in the deep neural network training process, but sometimes an "overfitting" phenomenon usually occurs because the number of training samples in a training set is limited, that is, the classification accuracy on the training set is high, and the performance on a test set is poor. At this time, a data enhancement technology is often adopted to increase the number of samples of the training set and extract more feature information from a limited number of data, thereby increasing the generalization capability of the deep neural network and preventing overfitting. The phenomenon of over-fitting also exists in the generation process of the confrontation sample, so that more effective data are generated by using limited data through a data enhancement technology, and the richness of the input sample is improved.

The problem of generation of countersamples is an optimization problem with limited conditions, and therefore,firstly, the generation process is described by establishing a mathematical model: x is the original clean image in the data set, y is the real label corresponding to the image, x ^adv The generated challenge sample. Theta is used to represent the parameters of the image classification model, and L represents the cross entropy loss function during the forward propagation of the deep neural network.

In the process of generating the countermeasure sample, the purpose is to make the prediction labels Y different from Y, i.e. by increasing the loss function, so that Y is not equal to Y, but at the same time, the countermeasure disturbance should be kept within a certain range, i.e. | | x ^adv -x|| _∞ Epsilon is less than or equal to epsilon. The generation of the challenge sample is thus shown in equation (1).

In a typical FGSM countermeasure sample generation Method, fast Gradient Signal Method (FGSM) is an original version of the FGSM countermeasure Method, and the idea is along the Gradient direction of a loss function

The confrontation sample is searched, and the implementation process of the method is shown as an equation (2). The method generates the countermeasure sample in a single step mode, so that the method has the characteristics of simple algorithm and high generation speed.

The Iterative Fast Gradient Signal Method (I-FGSM), also known as BIM, is an improvement over the FGSM Method, as shown in equation (3). Wherein α = ε/T is the step size of each iteration, T is the number of iterations, and the size of the counterdisturbance is constrained by the Clip function in the ε neighborhood of the original image x. The method divides the single-step gradient operation in the FGSM into multi-step iteration, improves the success rate of white-box attack, but also causes overfitting, and reduces the success rate of black-box attack relative to the FGSM.

The M (M) Iterative Fast Gradient Signal Method (MI-FGSM) is shown in formula (4) (5), where μ is the attenuation factor of the Momentum term, g _t Is the gradient weighted accumulation of the previous t iterations. The method stabilizes the updating direction of the loss function by introducing momentum in a mode of accumulating gradient and improves the convergence process, thereby greatly improving the attack success rate. But the success rate of black box attacks is not very high due to the solidification of noise.

The reverse Input Method (DIM) considers the dependence of the confrontation sample on the original clean image, and performs random transformation on the original Input with a given probability before generating the confrontation sample in each iteration, and the process is shown in equations (6) and (7). The specific method is random scaling transformation and padding. As a module attack mode, the method can be combined with other attack methods based on FGSM classes to generate a more migratable countermeasure sample.

Similar to the neural network training process, the overfitting phenomenon also exists in the anti-sample generation process, namely the success rate of white box attack is high, and the success rate of black box attack is low. The main task of the research is to find a better method for generating the confrontation sample, improve the generalized attack capability of the confrontation sample, and effectively relieve overfitting, so that the success rate of the black box attack is improved.

The generation of the confrontation sample is closely related to the neural network training, the two processes are performed as shown in a table 1, common reasons generated by 'overfitting' are analyzed, and elicitation is obtained from an optimization method of the neural network training, so that the idea of improving the black box attack effect is obtained. By analyzing the two processes, the generation of the confrontation sample is different from the training of the neural network mainly in the output data, the parameters of the model are trained by the neural network, and the confrontation sample is generated into the confrontation image. The commonality of the two images is also clear, the input is the original training set image, and the generalization reduction phenomenon caused by over-training exists in the two processes. The generation process of the confrontation sample can be regarded as a black box process with input and output, when the input data is limited, the 'overfitting' of the confrontation sample can occur, which is mainly caused by the fact that unnecessary features of the input data are over-learned, and the effective method is to expand the input data amount of the generation process, improve the proportion of the necessary features learned by the confrontation sample, and accordingly relieve the overfitting. The generalization capability of the classification model is improved by a common data enhancement method in the neural network training process, the method can also be applied in the generation process of the countermeasure sample, and the success rate of black box attack of the countermeasure sample is improved by improving the diversification of the input images.

TABLE 1 contrast table for confrontation sample generation and neural network training process

Further, as a preferred embodiment, the collected image transformation methods include, but are not limited to: image random scaling, image translation transformation, image cropping, image rotation and image color gamut transformation. Further, in constructing the data enhancement method set, a transformation range for preventing image transformation distortion is set, and each image transformation method corresponds to a respective transformation range.

The single data enhancement mode is carried out by adopting a single transformation method, and the effect of relieving overfitting is limited. Therefore, in the embodiment of the scheme, through a diversified input strategy, a plurality of methods are introduced to carry out random transformation on the original clean image, and a batch of transformed images are generated at the same time, so that the number of input data is increased, the input diversity is improved, the overfitting is relieved more effectively, and the success rate of resisting the black box attack of the sample is improved.

First, a common image transformation method is generalized to a method set, and a data enhancement method set can be expressed as equation (8).

O＝{T ₁ (x),T ₂ (x),T ₃ (x),…,T _n (x)} (8)

Each of which methods T _i (x) Representing a data enhancement mode including, but not limited to, operations of translation, rotation, cropping, gamut transformation, and the like. In the method set, each transformation method has a respective transformation amplitude range, thereby ensuring that the image is not distorted, such as translation transformation needs to be within 0-10]Variations within the pixel range do not introduce changes in the values of the model classification labels.

One or more methods are randomly selected from the data enhancement method set, and the composition strategy integration image enhancement method is shown in formula (9).

Wherein, ω is _i The weight coefficients of the strategy are transformed for a certain image in the strategy space and act in the mode of a probability model. Omega _i The larger the probability that the transformation occurs in the entire policy space, the more feature information the generated countermeasure sample learns from the transformation.

Based on this, in the present embodiment, a confrontation sample generation Method (variant Input Strategies Method) based on a diversified Input strategy is shown in formula (10).

n represents the number of strategies whose objective function is to maximize the loss function. The random image transformation performed on each image is performed by the random image transformation with respective transformation probability P _i This is done as shown in equation (11).

The generation process of the challenge sample is shown in fig. 3. Firstly, inputting an original clean image into a countermeasure sample generation system, selecting a proper strategy from a data enhancement method set, and carrying out random image transformation. After which a batch of transformed images is formed. And inputting the images into a convolutional neural network to calculate gradient values of the images, performing weighted average summation on the gradients according to weights of the images, calculating the confrontation disturbance, adding the confrontation disturbance into the original clean image, and performing further iterative calculation until a generation process of the confrontation sample is completed. In N rounds of cyclic iterations, firstly obtaining sample data to be adjusted of current iteration, if the current iteration is first iteration, the sample data to be adjusted is original image sample data, otherwise, the sample data is a countermeasure sample generated in the previous round, then, image transformation is respectively carried out on the sample data to be adjusted by using an image transformation method in a data enhancement strategy set, the enhanced image data after image transformation is input into an image classification model for processing, a loss function gradient value of the corresponding enhanced image data is obtained according to the image classification model, then, weighted average is carried out on the loss gradient value according to weight to generate countermeasure noise in the current iteration, and the countermeasure sample in the current iteration is generated by adding the countermeasure noise to the sample data to be adjusted.

Aiming at the problem of single original data enhancement method, the diversity of the data enhancement method is improved by establishing a data enhancement method set and introducing various methods, the quantity of input data in the generation process of the countermeasure sample is increased by adopting a weighted summation mode, the fitting effect of the countermeasure sample on the input data is well relieved, and therefore the mobility of the countermeasure sample and the success rate of black box attack are effectively improved. In addition, for the problem of useless gradients, in the embodiment of the present application, cancellation of the useless gradients can be achieved to a certain extent by using the averaged gradient.

Meanwhile, the scheme also maintains the excellent property of strong expansibility, and the conversion relation of the scheme and the FGSM based method is shown in FIG. 4. Between the methods, the conversion of different methods can be realized by adjusting the hyper-parameters. When w =1 indicates that only one transformation mode is adopted, and the strategy is 'scaling transformation random filling', DISI-FGSM degenerates to DI ² -FGSM method, DISMI-FGSM degeneration to M-DI ² The FGSM method, in which only one specific data enhancement method in the transformation method set is used in the degraded method, embodies the downward compatibility of the method based on the diversified input strategy.

Therefore, according to the scheme of the embodiment of the scheme, aiming at the limitations and the defects existing in the existing method, the research introduces a diversified input strategy into the generation process of the countercheck sample, and the improvement of the countercheck sample black box attack capability is realized by utilizing various data enhancement technologies. The FGSM Method has good expansibility, and forms a stronger attack Method called as a diversified Input strategy Momentum iteration Method DISMI-FGSM (variant Input Strategies motion Iterative Method) by combining with MI-FGSM. The method performs pseudo-code description of the attack process on a single image classification model as shown in algorithm 1.

In the algorithm, the conversion between different countermeasure sample generation methods can be realized by adjusting the hyper-parameters, such as setting the data enhancement transformation probability Pi to 0, the algorithm becomes the algorithm of MI-FGSM, and if setting the iteration round number T to 0, the algorithm becomes the algorithm of FGSM.

Further, based on the above method, an embodiment of the present invention further provides an intelligent system countermeasure sample generating system based on a diversified input policy, including: a strategy collection module, a strategy selection module, an enhancement transformation module and a sample generation module, wherein,

the strategy collection module is used for collecting the image transformation methods and constructing a data enhancement method set according to the collected image transformation methods;

the strategy selection module is used for selecting a plurality of image transformation methods from the data enhancement method set to form a data enhancement strategy set according to the attack success rate of the pre-attack experiment;

the enhancement transformation module is used for respectively carrying out image transformation on the original image sample data by utilizing an image transformation method in the data enhancement strategy set to obtain image enhancement data after batch transformation;

and the sample generation module is used for acquiring gradient values of the image enhancement data corresponding to each image transformation method by using the image classification model, acquiring the anti-disturbance noise by performing weighted average on the gradient values, and generating the anti-disturbance sample by using the anti-disturbance noise.

In order to verify the validity of the scheme, the following further explanation is made by combining experimental data:

by performing extensive experiments on the ImageNet dataset, the experimental environment settings are shown in table 2, and the other settings are as follows:

data set: 1000 images were randomly selected from the ImageNet validation set, and the images were of different image categories. The purpose of the experiment is to successfully resist sample attack and show the effectiveness of the attack method through the attack success rate, so that the images can be correctly classified through testing. All images are RGB three-channel images, and the size of each image is 299 multiplied by 3.

Model: a total of 7 image classification models were studied in the experiment, including 4 normally trained models, namely inclusion-v 3 (Inc-v 3), inclusion-v 4 (Inc-v 4), inclusion-Resnet-v 2 (IncRes-v 2) and Resnet-v2-101 (Res-101), and three countertrained models, namely ens 3-adv-inclusion-v 3 (Inc-v 3) _ens3 ),ens4-adv-Inception-v3(Inc-v3 _ens4 ) And ens-adv-inclusion-ResNet-v 2 (IncRes-v 2) _ens )。

Setting the hyper-parameters: in the initial setting of the hyper-parameters of the experiment, in order to enhance the contrast effect, the parameter setting method of the existing research results is referred. The maximum disturbance added in the generation process of the confrontation sample is epsilon =16, the iteration number T =10 in the generation process, and the learning rate of each iteration is alpha =1.6. The momentum term attenuation coefficient was set to μ =1.0. For each method of the data enhancement method set, the randomness and the contrast of the effect of the transformation are reflected, and the M-DI is effectively compared with the M-DI ² FGSM is compared and the transition probability P is set to 0.5, i.e. each transformation occurs with a 50% probability. The weight coefficient omega in the method is set to be 1/n, so that the advantage effect among various transformation strategies is conveniently compared.

TABLE 2 Experimental Environment

1. Comparative analysis of single strategies and diverse strategies

Compared with a single data enhancement method of operating single transformation, the strategy combination method can effectively improve the sample migration resistance. To verify this, challenge samples were generated using inclusion-v 3 and white and black box tests were performed on the other models, with the experimental results shown in table 3, where the prime indicates the success rate of the white box attack and the non-prime indicates the success rate of the black box attack. To ensure equilibrium between strategies, ω will be _i Set to 1/5, the hybrid strategy uses a hybrid of 5 single-method strategies, i.e. diversification of the input transformation strategy is achieved by mixing multiple transformation strategies, i.e. 5 operations are performed using one transformation method. Can be seen from the reaction with M-DI ² Compared with FGSM, the single strategy generation method and the mixed strategy generation method both realize the great improvement of the success rate of the black box attack, and the success rate of the white box attack is also kept at a higher level. By comparison, it can be seen that in the single strategy attack method, the scaling transformation can greatly improve the success rate of the black box attack, andthe mobility of the challenge sample can be further improved by a combination of strategies between the methods, relative to M-DI ² FGSM, the average success rate of black box attacks is improved by 15.3% by the mixed strategy countersample generation method.

TABLE 3 success rate of single and hybrid strategies against attacks

The diversified input of the research is mainly embodied in strategy diversity and input number diversity, and the gain effect of strategy selection on sample mobility resistance is further researched below. As shown in fig. 5, in the inclusion v3, an experiment is performed by using a single strategy of image inversion, and compared with 5 single strategies in table 3, the attack test result of the generation method based on the inversion strategy has a lower success rate of black box attack and is lower than the success rate of mixed strategy attack. But when combined with the hybrid strategy (strategy 1 in the figure), with a new hybrid strategy 2 (integration of the flip strategy with the first 5 strategies, weighting system ω of hybrid strategy 2) _i Set to be 1/6) to generate the confrontation sample, the newly generated confrontation sample is not reduced by introducing a turnover strategy, but a better mobility attack gain is generated, so that the success rate of the black box attack is further improved. More feature learning spaces can be introduced into different image transformation strategies, different characteristics of high-dimensional features are reflected, more feature knowledge is learned in the generated countermeasure samples, and accordingly, the countermeasure samples generated based on the diversified input strategy method can be inferred to have better mobility and black box attack success rate.

2. Attack success rate comparison analysis of single image classification model

The validity of the protocol was verified by comparative experiments. Using DI ² -FGSM、DISI-FGSM、M-DI ² The experiment is carried out by four attack methods of FGSM and DISMI-FGSM, a confrontation sample image is generated on 4 normally trained image classification models respectively, and then attack test is carried out on 7 image classification models (comprising 3 confrontation trained image classification models) respectivelyThe measure index is the attack success rate, that is, the generated countermeasure sample is the percentage of the number of misclassifications of the image classification model to the total number of the images. The results of the experiment are shown in table 4.

As can be seen from table 4, for the method for generating the confrontation sample based on the diversified input strategy, the success rate of the attack is improved under both the white box condition and the black box condition. Wherein the DISI-FGSM method is performed with DI ² Comparison of the-FGSM method, DISMI-FGSM being M-DI ² Comparison of the FGSM method, in comparison of the 4 methods, only the diversified input data is replaced, so as to achieve the enhancement of the capability of resisting sample attack, which indicates the effectiveness of the method. As can be seen from the data in the table, when the image classification model with the defense mechanism is attacked against the sample and generated by applying the DISMI-FGSM method on the IncRes-v2, the image classification model is opposite to the M-DI ² The average success rate of black box attack is improved by 12.3% in FGSM, and meanwhile, when the anti-sample attack Inc-v3 generated on the Inc-v4 by using the DISMI-FGSM method is used, the success rate of black box attack reaches 87.2%.

TABLE 4 success rate (%) of DISMI-FGSM method for attacking single image classification model

The method has good expansibility and can be combined with other confrontation sample generation methods based on FGSM, so that the method is combined with the method provided by the scheme so as to verify the improvement effect of the method on the original method. Table 5 shows the comparison of the attack success rates after combining the method with TIM. As can be seen from the table, compared with the TIM method, the success rate of partial white-box attack of the TI-DISM is reduced, for example, the success rate of white-box attack of the TIM on the Inc-v4 is 99.8 percent, while the success rate of attack of the TI-DISM is 99.5 percentAnd the error of 3 attack failures in 1000 images is within an acceptable range. The success rate of the black box attack is greatly improved, for example, a countermeasure sample is generated on a Res-101 network by two methods, and a countermeasure training image classification model IncRes-v2 with a defense mechanism is attacked _ens In time, the success rate of the combined method TI-DISM is 43.8%, which is 22.4% higher than the 21.4% attack success rate of TIM, and thus, the new method has stronger capability of resisting the migration attack of the sample.

TABLE 5TIM vs. TI-DISM attack success ratio comparison

3. Integrated image classification model attack success rate comparison analysis

And simultaneously attacking the multiple models, and verifying attack performance under the environment of the multiple image classification models. Table 6 shows the attack success rate of each method under the integrated model. In the experiment, firstly, a confrontation sample is generated by using 4 common training models, and the integration process refers to a logic value integration method provided in a momentum method and adopts a mode of equal weight coefficients for integration. Compared with the multi-model integrated attack and the single-model attack, the integrated attack realizes further improvement of the attack success rate compared with the single-model attack. From the specific attack method, two comparative experiments were performed to compare I-FGSM and DI, respectively ² -FGSM, DISI-FGSM and MI-FGSM, M-DI ² In a first group of comparison experiments, the attack success rate of the I-FGSM and the DI-FGSM is not changed greatly, the average attack success rate is 59.7 percent and 59.9 percent, which shows that the DI-FGSM does not cause the improvement of the performance of a basic iteration method, the attack effect of the DISI-FGSM is better, the attack success rate reaches 73.9 percent, and particularly on an anti-training model, the attack success rate is improved by 25.2 percent compared with the I-FGSM, which shows that a diversified input strategy plays a role, and the migration attack capability of an anti-sample is effectively improved. The second set of comparative experiments is based on the introduction of momentum termsBased on developed, M-DI ² Compared with the basic momentum method, the white box attack success rate of the FGSM is reduced to some extent, but the black box attack success rate achieves performance improvement of more than 10%, while the DISMI-FGSM is relatively improved to a greater extent and is used in the Inc-v3 _ens4 The effect improvement of 29.5 percent is realized. Compared with two groups of data transversely, the improved method with momentum term can effectively improve the migration attack capability of the confrontation sample, so that the momentum method is introduced in the process of generating the confrontation sample, and the attack effect of the method is enhanced.

Table 6 integrated attack success ratio comparison table

In the generation process of the confrontational sample, the added confrontational disturbance is ensured to be as small as possible and invisible in visual perception. Thus, the study will show an image of the stages of the integrated attack process as shown in fig. 6. The left column of the figure is the original clean image, the middle is the transformed image example, and the right column is the image corresponding to the generated confrontational sample. It can be seen that the generated countermeasure sample is not greatly different from the original image, and the countermeasure disturbance is also small. In addition, in order to visually display the action principle of the confrontational sample, the research respectively calculates class activation maps of the original image and the generated confrontational sample by using a Grad-CAM method, as shown in a middle right side diagram of an original image and a generated confrontational sample in FIG. 6, and a framed area is an important identification area in the image classification process. By comparison, it can be seen that the key areas of the class activation maps of the generated confrontation sample and the original clean image are changed, and the positions and the numbers of the key areas are changed, which indicates that the image classification model focuses on different parts of the original image and the confrontation sample, and the focused image features are different. The countermeasure sample generated by the diversified input strategy enables the key area of the image classification to be shifted, and the image classification model is directly caused to show certain vulnerability when facing the attack of the countermeasure sample.

4. Hyper-parameter settings analysis

By setting certain hyper-parameters, some are general hyper-parameters of the generation process of the countermeasure sample, such as maximum disturbance, iteration times, learning rate and the like, and some are special hyper-parameters proposed for the research, such as transformation probability in a transformation method and weight coefficients of each transformation method. In the experiment, two methods, namely DISI-FGSM and DISMI-FGSM, are mainly used for comparative study, and corresponding image classification results are obtained by setting different special super-parameter values and attacking the image classification model, so that the attack success rate index is obtained. In general hyper-parametric setting, in order to enhance the effectiveness of contrast, according to the setting method studied before, the maximum perturbation is ∈ =16, the number of iterations T =10, the step length α =1.6, and the attenuation coefficient defaults to μ =1.0.

The random transformation probability P controls the transformation probability of each transformation in the data enhancement method set, and therefore, the influence of the transformation probability P on the capability of resisting sample attack is analyzed. In the research, two methods, namely DISMI-FGSM and DISI-FGSM, are used on Res-101 to generate a confrontation sample, white box attack is carried out on the model, black box attack is carried out on other classification models in the experimental part, and the experimental result is shown in figure 7. In the experimental process, the random transformation probability is increased from 0 in steps of 0.1, and the maximum transformation probability is 1, which means that each transformation strategy can play a role certainly, and when P =0, the method does not perform transformation processing on the image. Fig. 7 (a) shows a variation trend of the attack success rate along with the increase of the transformation probability in the DISMI-FGSM method, and it can be seen that the black box attack success rate shows a rising trend along with the increase of the random transformation probability P, while the white box attack success rate remains substantially unchanged. And (b) the graph shows that the curve situation of the DISI-FGSM is similar to that of the graph (a), but the promotion amplitude is larger, for example, when the Inc-v3 is attacked by a black box, the attack success rate of the DISMI-FGSM is increased by 35.2% and the attack success rate of the DISI-FGSM is increased by 49.6% in the process of changing the transformation probability from 0 to 1, which shows the effectiveness of the scheme and also shows the effectiveness promotion effect of the momentum term in the process of resisting the generation of the sample. Through the two graphs (a) and (b), the introduction of the random transformation method is shown to effectively improve the mobility of the random transformation method, so that when a high success rate of black box attack is to be realized, the random transformation probability needs to be adjusted to the maximum value.

The weight coefficient represents the action weight exerted by each conversion method, and the greater the weight, the greater the weight occupied in the overall image conversion. As can be seen from table 3, there is a difference in attack success rate when a single strategy is used to generate the confrontation sample. In order to research the influence of the weight value of each strategy method on the final attack success rate, the attack success rate s of a single strategy _i And taking different weight coefficient setting strategies as a basis to carry out comparative analysis, and taking the final attack success rate as an evaluation index to obtain the influence and setting suggestion of the weight system. Three strategies are mainly compared:

(1) And (3) uniformity strategy: i.e. using the coefficient setting method with equal weight, omega is set in the experiment _i ＝1/5；

(2) L1 norm strategy: taking the L1 norm of each attack success rate as a denominator to obtain

(3) softmax strategy:

however, it was noted in the study that when the success rate of the attack is greater, there is an exponential explosion phenomenon, so first on s _i Using L2 norm for normalization, i.e.

Thereby, the device

According to the three weight system setting strategies, the DISMI-FGSM is used for generating the countermeasure samples on the inclusion-v 3, the attack tests are carried out on 7 image classification models, and the result comparison shows that compared with the uniform strategy, the countermeasure samples generated on the common training model by the softmax strategy have better attack performance, but in general, the influence of the latter two strategies on the attack success rate is not large, which is mainly caused by the balance of small difference of the attack success rates of the strategies of the single methods. Therefore, in the generation process of the confrontation sample, when the attack success rates of the methods are not greatly different, the weight coefficient can be set by adopting a uniform strategy, and the high-quality confrontation sample can be generated efficiently and conveniently.

In conclusion, the scheme utilizes a diversified input strategy to introduce multiple transformation modes and multiple transformation images into the generation process of the countermeasure sample by analyzing the limitation of the existing generation method of the countermeasure sample and aiming at the phenomenon of 'overfitting' in the process of resisting the black box attack of the sample, so that the diversification of transformation types and transformation numbers is improved, and the mobility and the success rate of the black box attack of the countermeasure sample are improved. In addition, the method has the characteristic of good expansibility, and the attack success rate is further improved by combining with other FGSM methods. Single and multiple model attack experiments were performed on ImageNet datasets and the results demonstrate the effectiveness of the method. The diversified input strategy method is an important idea for constructing a high-quality attack test method, and can perform performance test on the image classification model before application and deployment, so that a safer and more reliable deep learning model is constructed.

Unless specifically stated otherwise, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The elements of the various examples and method steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and the components and steps of the examples have been described in a functional generic sense in the foregoing description for clarity of hardware and software interchangeability. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

Those skilled in the art will appreciate that all or part of the steps of the above methods may be implemented by instructing the relevant hardware through a program, which may be stored in a computer-readable storage medium, such as: read-only memory, magnetic or optical disk, and the like. Alternatively, all or part of the steps of the foregoing embodiments may also be implemented by using one or more integrated circuits, and accordingly, each module/unit in the foregoing embodiments may be implemented in the form of hardware, and may also be implemented in the form of a software functional module. The present invention is not limited to any specific form of combination of hardware and software.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. An intelligent system confrontation sample generation method based on a diversified input strategy is characterized by comprising the following contents:

collecting image transformation methods, and constructing a data enhancement method set according to the collected image transformation methods;

selecting a plurality of image transformation methods from the data enhancement method set according to the attack success rate of the pre-attack experiment and forming a data enhancement strategy set;

respectively carrying out image transformation by using an image transformation method in a data enhancement strategy set aiming at original image sample data to obtain image enhancement data after batch transformation;

and obtaining gradient values of the image enhancement data corresponding to each image transformation method by using the image classification model, obtaining anti-disturbance noise by performing weighted average on the gradient values, and generating an anti-disturbance sample by using the anti-disturbance noise.

2. The intelligent system confrontation sample generation method based on diversified input strategy as claimed in claim 1, wherein the collected image transformation method includes but not limited to: image random scaling, image translation transformation, image cropping, image rotation and image color gamut transformation.

3. The intelligent system countermeasure sample generation method based on diversified input strategies according to claim 1 or 2, wherein in constructing the data enhancement method set, a transformation amplitude range for preventing image transformation distortion is provided, and each image transformation method corresponds to a respective transformation amplitude range.

4. The intelligent system confrontation sample generation method based on diversified input strategy as claimed in claim 1, wherein the data enhancement strategy set is expressed as:

wherein, T _i (x) Image transformation method denoted by i, omega _i The weight coefficients corresponding to the image transformation methods in the strategy space, and n is the number of image transformation methods selected from the data enhancement method set.

5. The intelligent system countermeasure sample generation method based on a diversified input strategy as claimed in claim 1, wherein N rounds of loop iterations are set and a back propagation process is used to generate a final countermeasure sample, where N is a preset maximum number of iterations.

6. The method according to claim 5, wherein in N rounds of loop iteration, the sample data to be adjusted of the current iteration is first obtained, if the current iteration is the first iteration, the sample data to be adjusted is the original image sample data, otherwise, the sample data is the countermeasure sample generated in the previous round, then the image transformation method in the data enhancement strategy set is used to perform image transformation on the sample data to be adjusted, the enhanced image data after image transformation is input to the image classification model for processing, the loss function gradient value corresponding to the enhanced image data is obtained according to the image classification model, then the loss gradient value is weighted-averaged according to the weight to generate the countermeasure disturbance noise in the current iteration, and the countermeasure sample in the current iteration is generated by adding the countermeasure disturbance noise to the sample data to be adjusted.

7. The intelligent system confrontation sample generation method based on diversified input strategy as claimed in claim 1, wherein the loss function gradient value calculation process is expressed as:

wherein L () is a cross entropy loss function,

image enhancement data, P, corresponding to the image transformation method denoted i _i The data enhancement transformation probability corresponding to the image transformation method with the label i is shown, y is an image label, and theta is a model parameter.

8. An intelligent system confrontation sample generation system based on diversified input strategies, comprising: a strategy collection module, a strategy selection module, an enhancement transformation module and a sample generation module, wherein,

the strategy collection module is used for collecting the image transformation methods and constructing a data enhancement method set according to the collected image transformation methods;

the strategy selection module is used for selecting a plurality of image transformation methods from the data enhancement method set according to the attack success rate of the pre-attack experiment and forming a data enhancement strategy set;

the enhancement transformation module is used for respectively carrying out image transformation on the original image sample data by utilizing an image transformation method in the data enhancement strategy set to obtain image enhancement data after batch transformation;

and the sample generation module is used for acquiring gradient values of the image enhancement data corresponding to the image transformation methods by using the image classification model, acquiring the anti-disturbance noise by performing weighted average on the gradient values, and generating the anti-sample by using the anti-disturbance noise.

9. An electronic device comprising a memory and a processor, wherein the memory has stored therein executable code and an original image and generated countermeasure samples, the processor implementing the method of any one of claims 1 to 7 when executing the executable code.

10. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed in a computer, causes the computer to perform the method of any one of claims 1 to 7.

Images