CN112949678A - Method, system, equipment and storage medium for generating confrontation sample of deep learning model - Google Patents

Abstract

The invention belongs to the field of deep learning models, and discloses a method, a system, equipment and a storage medium for generating a confrontation sample of a deep learning model. By acquiring the sensitive matrix, the disturbance is realized based on the sensitive matrix, the distribution of disturbed pixel points becomes sparse, the disturbance is less easily perceived from the angle of human eye observation, and the two norms of the confrontation sample are greatly reduced from the angle of quantization.

Description

Deep learning model adversarial sample generation method, system, device and storage medium

technical field

The invention belongs to the field of deep learning models, and relates to a method, system, equipment and storage medium for generating a deep learning model confrontation sample.

Background technique

Deep learning has outperformed traditional machine learning in many tasks, such as image classification, object detection, speech recognition, natural language processing, etc. With the extensive application and development of deep neural networks in various fields, its security issues have also attracted more and more attention. Adversarial attack means that the attacker constructs targeted deep learning model input to make the deep learning model make a misjudgment, that is, the result is inconsistent with human judgment. The perturbation causes the deep learning model to make misjudgments and obtain results that are inconsistent with the input benign samples. Such "malignant" samples are called adversarial samples.

In the field of image classification, the perturbation of an adversarial attack is to slightly change the value of each pixel in the image, so that the deep learning model makes an error in classifying the image. Adversarial attacks are divided into two types, black box and white box, according to how much information the attacker can obtain. Among them, the white box means that the attacker can obtain the internal parameters of the deep learning model, including the structure and weight of the deep learning model, while the black box attack is closer to the actual application situation, and the attacker can only obtain the output vector of the deep learning model. In the black-box adversarial attack, because the gradient of the deep learning model cannot be obtained directly, some optimization methods using evolutionary strategies can have relatively good performance. Evolution strategy is a method of solving parameter optimization problems that imitates the principle of biological evolution. It generates new individuals (individual) continuously, and eliminates them by comparing the fitness between individuals, and finally obtains a higher fitness. degree individual. Since the optimization process of the evolutionary algorithm does not require gradient information, it is not constrained by the black box condition. In image classification, the adversarial attack is also divided into targeted adversarial attack (targeted) and non-targeted adversarial attack (non-targeted) according to the different situations that make the deep learning model misjudgment. If the classification result of the model is different from the original class , it is an untargeted attack, and if the model misjudges the image as an arbitrarily selected target class, the attack is a targeted attack.

When measuring the generation efficiency of adversarial samples in the case of black boxes, the number of queries of the deep learning model is often used as an indicator, and when measuring the "indiscernibility" of an adversarial sample, the zero norm (number of disturbed pixels) with added disturbance is often used. , the second norm (the square and root of the perturbation value of each pixel) and the infinite norm (the largest perturbation value in each pixel) as indicators, in the case of finite infinite norm, evolutionary algorithms often have relatively high efficiency However, due to the existence of some redundant perturbations, that is, some pixels do not need to be perturbed, as a whole, the second norm of the adversarial samples generated by the evolutionary algorithm is generally high, and the distribution of perturbation points is relatively wide, so it is easy to be The human eye perceives. In response to this problem, if the size of the second norm is simply added to the fitness, the balance between the original fitness and the second norm will be difficult to control, resulting in a decrease in the optimization speed of the deep learning model, or the loss of the second norm. The size has almost no binding force, so that the second norm of the generated adversarial samples is still high, and the higher second norm of the adversarial samples leads to the low stability and security of the deep learning model after training with this type of adversarial samples. .

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art that the two-norm of the adversarial samples generated by the existing black-box attack methods is generally high or the size of the two-norm has almost no binding force, and provides a deep learning model adversarial sample. Generation method, system, device and storage medium.

To achieve the above object, the present invention adopts the following technical solutions to realize:

In a first aspect of the present invention, a method for generating an adversarial sample for a deep learning model includes the following steps:

S1: Obtain the sensitivity matrix of the original image based on the target deep learning model;

S2: Construct several norm groups according to a number of preset zero norms and a number of infinite norms for perturbation; obtain a perturbation map corresponding to each norm group according to the sensitivity matrix and several norm groups;

S3: Query the target deep learning model for both the original image and the perturbation map corresponding to each norm group, and obtain and obtain the anti-zero norm and the anti-zero norm according to the drop value of the predicted probability of the original class of the perturbation map corresponding to each norm group relative to the original image. infinity norm;

S4: Construct a preset number of adversarial disturbance matrices according to the adversarial infinity norm, take the prediction probability of the attack target class as the fitness, and take the maximum fitness as the optimization goal, based on the original image, the sensitive matrix and the adversarial zero norm, iteratively optimize through the evolutionary algorithm For each adversarial perturbation matrix, S5 is performed after each iteration;

S5: When there is at least one target adversarial disturbance matrix in the adversarial disturbance matrix after the current iteration optimization, the original image is perturbed by the target adversarial disturbance matrix to obtain adversarial samples and output, and the iteration ends; otherwise, return to S4.

The further improvement of the adversarial sample generation method of the deep learning model of the present invention is:

The specific method of the S1 is:

Obtain a deep learning model with the same classification target as the target deep learning model as a reference model;

Select each pixel point in the original image one by one and perform the following steps: perturb the current pixel point with a preset size to obtain a sensitive image, input the sensitive image into the reference model, and obtain the original class output of the current pixel point after perturbation relative to the reference model before perturbation The change value of the predicted probability is used as the sensitive value of the current pixel point;

Arrange the sensitivity values of each pixel in the original image according to the position of each pixel in the original image to obtain the sensitivity matrix of the original image based on the target deep learning model.

The specific method of the S2 is:

According to a number of zero norms and a number of infinite norms preset for perturbation, select the zero norm of the number of zero norms one by one to traverse the number of zero norms, and compare the selected zero norm and the number of infinite norms. Infinite norms are combined one by one to obtain several norm arrays;

The initial perturbation matrix is preset. The number of parameters in the initial perturbation matrix is the same as the number of pixels in the original image, and is arranged according to the position of each pixel in the original image. The absolute value of each parameter is 1, and the initial perturbation matrix is the same as the sensitivity matrix. The symbols of the parameters of the position are the same; select the norm groups one by one, and traverse several norm groups, multiply the half of the infinite norm in the current norm group with each parameter in the initial perturbation matrix, and obtain the perturbation matrix corresponding to each norm group, get For the zero norm in the current norm array, according to the order of sensitivity values from large to small, set the first zero norm sensitive values in the sensitive matrix to 1, and set the remaining sensitive values to 0 to obtain the mask matrix corresponding to each disturbance matrix. ;

The perturbation matrix corresponding to each norm group is superimposed on each pixel in the original image, and then multiplied by the mask matrix corresponding to each perturbation matrix to obtain the perturbation map corresponding to each norm group.

The specific method of the S3 is:

The original image and the perturbation map corresponding to each norm group are input into the target deep learning model, and the original class prediction probability of the original image and the original class prediction probability of the perturbation map corresponding to each norm group are obtained, and the perturbation map corresponding to each norm group is obtained. The drop value of the original class prediction probability of the original image;

In all norm groups, the perturbation map corresponding to the norm group of the same zero norm is superimposed relative to the drop value of the original class prediction probability of the original image to obtain several first superimposed values. The zero norm corresponding to the first superposition value is the anti-zero norm;

In all norm groups, the perturbation map corresponding to the norm group of the same infinite norm is superimposed relative to the drop value of the original class prediction probability of the original image to obtain a number of second superposition values, and select the number of the second superposition values with the largest slope change. The infinity norm corresponding to the two superposition values is the anti-infinity norm.

The evolution algorithm in S4 is: an adaptive differential evolution strategy or a linearly stretched covariance matrix adaptive evolution strategy.

When the evolutionary algorithm is an adaptive differential evolution strategy, the specific method of S4 is:

S401: Randomly select values in [- against infinity norm, against infinity norm] as parameters in the adversarial disturbance matrix, and construct a preset number of adversarial disturbance matrices; each parameter in the adversarial disturbance matrix and each pixel of the original image one-to-one correspondence;

S402: According to the adaptive scale factor and the mutation formula, mutate a preset number of anti-disturbance matrices to obtain a preset number of mutant anti-disturbance matrices, and use both the anti-disturbance matrix and the mutant anti-disturbance matrix as individuals; according to the anti-zero norm and the sensitivity matrix to obtain the mask matrix corresponding to each individual; each pixel in the original image is superimposed on each individual, and then multiplied with the mask matrix corresponding to each individual to obtain the perturbation map corresponding to each individual; each individual corresponds to The perturbation map is input into the target deep learning model, and the fitness of the perturbation map corresponding to each individual is obtained. According to the order of fitness from large to small, a preset number of each individual is selected as the iteratively optimized confrontation perturbation matrix;

S403: Update the adversarial disturbance matrix with the iteratively optimized adversarial disturbance matrix, perform S402 iteratively, and perform S5 after each iteration.

When the evolutionary algorithm is a linearly stretched covariance matrix adaptive evolution strategy, the specific method of S4 is:

S411: Randomly generate k n-dimensional vectors as the basis of the stretched space; randomly generate a preset number of k-dimensional vectors through a preset k-dimensional Gaussian distribution, multiply the basis by the k-dimensional vector as a weight, and obtain a preset number of The n-dimensional sample vector, the anti-perturbation matrix is obtained by limiting the n-dimensional sample vector with the anti-infinite norm;

S412: According to the anti-zero norm and the sensitive matrix, obtain a mask matrix corresponding to each anti-disturbance matrix; superimpose each pixel point in the original image with each anti-disturbance matrix, and then multiply the mask matrix corresponding to each anti-disturbance matrix , obtain the disturbance graph corresponding to each anti-disturbance matrix; input the disturbance graph corresponding to each anti-disturbance matrix into the target deep learning model to obtain the fitness of the disturbance graph corresponding to each anti-disturbance matrix. Set a number of anti-disturbance matrices, update the k-dimensional Gaussian distribution according to the set number of anti-disturbance matrices, randomly generate a preset number of optimized k-dimensional vectors according to the updated k-dimensional Gaussian distribution, and multiply the optimized k-dimensional vector by the weight base, obtain a preset number of optimized n-dimensional sample vectors, and optimize the n-dimensional sample vectors to resist infinite norm limiting to obtain an iteratively optimized anti-disturbance matrix;

S413: Update the adversarial disturbance matrix with the iteratively optimized adversarial disturbance matrix, perform S412 iteratively, and perform S5 after each iteration.

In a second aspect of the present invention, a deep learning model adversarial sample generation system includes a sensitivity matrix acquisition module, a disturbance map acquisition module, an optimization module, a parameter determination module, and an output module;

The sensitivity matrix acquisition module is used to acquire the sensitivity matrix of the original image based on the target deep learning model;

The perturbation map acquisition module is used to construct several norm groups according to a number of zero norms and a number of infinite norms preset for perturbation; according to the sensitivity matrix and several norm groups, obtain a perturbation map corresponding to each norm group;

The parameter determination module is used to query the target deep learning model for both the original image and the perturbation map corresponding to each norm group, and obtain and predict the drop value of the original class prediction probability of the perturbation map corresponding to each norm group relative to the original image to obtain the anti-zero norm numbers and against the infinity norm;

The optimization module is used to construct a preset number of adversarial disturbance matrices according to the adversarial infinity norm, take the prediction probability of the attack target class as the fitness, and take the maximum fitness as the optimization goal. Iteratively optimize each anti-disturbance matrix, and trigger the output module after each iteration;

The output module is used to perturb the original image through the target adversarial perturbation matrix when there is at least one target adversarial perturbation matrix in the adversarial perturbation matrix after the current iterative optimization, to obtain adversarial samples and output them, and the iteration ends; otherwise, the optimization module is triggered.

In a third aspect of the present invention, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above-mentioned deep learning when executing the computer program The steps of the model adversarial example generation method.

A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the steps of the above-mentioned deep learning model adversarial sample generation method are implemented .

Compared with the prior art, the present invention has the following beneficial effects:

The deep learning model adversarial sample generation method of the present invention limits the disturbance area of the original image based on the sensitive matrix of the target deep learning model, so that the distribution of disturbance points becomes sparse, thereby making the adversarial disturbance more difficult to detect. From the perspective of quantification , the two-norm against perturbation has been greatly reduced. At the same time, by combining the sensitive matrix to conduct reasonable "trials", trying different combinations of zero norm and infinite norm, the zero norm and infinite norm are automatically and dynamically adjusted for each adversarial attack, which improves the efficiency of adversarial attacks and the The success rate when the number of queries is limited. The whole process is plug-in, in which the iterative optimization strategy part can be replaced by different evolutionary algorithms for optimization in combination with the actual application situation, which is convenient and fast. Then, training the target deep learning model through the generated adversarial samples with low two-norm can further improve the prediction accuracy of the target deep learning model and improve the robustness of the target deep learning model.

Further, by virtue of the highly similar sensitivity matrix between different deep learning models, the target deep learning model is not directly queried, but the sensitivity matrix is obtained through the reference model, which reduces the number of queries.

Description of drawings

Fig. 1 is a flow chart of a method for generating an adversarial sample of a deep learning model of the present invention;

2 is a schematic diagram of a method for generating an adversarial sample of a deep learning model of the present invention;

FIG. 3 is a schematic block diagram of the evolutionary algorithm of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

Below in conjunction with accompanying drawing, the present invention is described in further detail:

Referring to Figures 1 and 2, the present invention provides a method for generating adversarial samples for deep learning models. Based on the sensitivity matrix, a low-disturbance evolutionary algorithm black-box adversarial attack plug-in framework is designed to realize the generation of adversarial samples for deep learning models. , including the following steps.

S1: Obtain the sensitivity matrix of the original image based on the target deep learning model.

Specifically, obtaining a deep learning model with the same classification target as the target deep learning model is the reference model M _ref ; each pixel in the original image is selected one by one to perform the following steps:

The current pixel is perturbed by a preset size to obtain a sensitive image, and the sensitive image is input into the reference model to obtain the change value of the original class prediction probability output by the current pixel after perturbation relative to the reference model before perturbation, as the sensitivity of the current pixel. value; arrange the sensitivity values of each pixel in the original image according to the position of each pixel in the original image to obtain the sensitivity matrix of the original image based on the target deep learning model, and the sensitivity matrix is composed of the sensitivity value of each pixel.

Specifically, the sensitivity of the reference model M _ref to each pixel in the original image is defined as the change in the predicted probability of the original category in the model output vector caused by the output of the reference model M _ref caused by a certain fixed-size perturbation to the pixel. value, this process is used to estimate the dimension, that is, the value of the partial derivative of the pixel point, as the change value of the original class prediction probability output by the reference model before the disturbance of the current pixel point after the disturbance, the absolute value of the partial derivative The bigger it is, the more sensitive it is.

S2: Construct several norm groups according to a number of zero norms and a number of infinite norms preset for perturbation; obtain a perturbation map corresponding to each norm group according to the sensitivity matrix and several norm groups.

及若干无穷范数

逐一选取若干零范数中的零范数至遍历若干零范数，将选取的零范数与若干无穷范数中的各无穷范数一一组合，得到若干范数组

The sensitivity matrix generated by S1 is used to attack the target deep learning model. Since the sensitivity matrices of different deep learning models are highly similar, the sensitivity matrix of the reference model can be directly used to attack the target deep learning model. Specifically, according to preset zero-norms for perturbation

and some infinity norm

Select the zero-norms of the zero-norms one by one to traverse the zero-norms, and combine the selected zero-norms with the infinite-norms of the infinite-norms one by one to obtain a number of norm groups

的一半即

与初始扰动矩阵中的各参数相乘，得到各范数组对应的扰动矩阵，获取当前范数组中的零范数，按照敏感值从大到小的顺序，将敏感矩阵中前零范数

个敏感值置为1，其余敏感值置为0，得到各扰动矩阵对应的掩膜矩阵；将原始图像中的各像素点分别叠加各范数组对应的扰动矩阵，然后与各扰动矩阵对应的掩膜矩阵相乘，得到各范数组对应的扰动图。The initial perturbation matrix is preset. The number of parameters in the initial perturbation matrix is the same as the number of pixels in the original image, and is arranged according to the position of each pixel in the original image. The absolute value of each parameter is 1, and the initial perturbation matrix is the same as the sensitivity matrix. The symbols of the positional parameters are the same; select the norm arrays one by one, until traversing several norm arrays, the infinite norm in the current norm array

half of

Multiply each parameter in the initial perturbation matrix to obtain the perturbation matrix corresponding to each norm group, obtain the zero norm in the current norm group, and put the first zero norm in the sensitivity matrix according to the order of sensitivity values from large to small.

Each sensitive value is set to 1, and the remaining sensitive values are set to 0, and the mask matrix corresponding to each perturbation matrix is obtained; the perturbation matrix corresponding to each norm group is superimposed on each pixel in the original image, and then the mask matrix corresponding to each perturbation matrix is superimposed on each pixel in the original image. The membrane matrix is multiplied to obtain the perturbation map corresponding to each norm array.

S3: Query the target deep learning model for both the original image and the perturbation map corresponding to each norm group, and obtain the anti-zero norm l ₀ according to the drop value of the predicted probability of the original class of the perturbation map corresponding to each norm group relative to the original image. and against the infinity norm l _∞ .

运用肘部法则，选取若干第一叠加值中斜率变化最大的第一叠加值对应的零范数为对抗零范数；将所有范数组中，同一无穷范数的范数组对应的扰动图相对于原始图像的原始类预测概率的下降值叠加，得到若干第二叠加值

运用肘部法则，选取若干第二叠加值中斜率变化最大的第二叠加值对应的无穷范数为对抗无穷范数。Specifically, the original image and the perturbation map corresponding to each norm group are input into the target deep learning model, the original class prediction probability of the original image and the original class prediction probability of the perturbation map corresponding to each norm group are obtained, and the perturbation map corresponding to each norm group is obtained. The drop value p _i,j of the original class prediction probability of the graph relative to the original image; in all norm groups, the perturbation graph corresponding to the norm group of the same zero norm is superimposed relative to the drop value of the original class prediction probability of the original image, get several first superposition values

Using the elbow rule, select the zero norm corresponding to the first superposition value with the largest slope change among the first superposition values as the anti-zero norm; in all norm groups, the perturbation map corresponding to the norm group of the same infinite norm is relative to The descending values of the original class prediction probability of the original image are superimposed to obtain several second superimposed values

Using the elbow rule, the infinity norm corresponding to the second superposition value with the largest slope change among several second superposition values is selected as the anti-infinity norm.

S4: Construct a preset number of adversarial disturbance matrices according to the adversarial infinity norm, take the prediction probability of the attack target class as the fitness, and take the maximum fitness as the optimization goal, based on the original image, the sensitive matrix and the adversarial zero norm, iteratively optimize through the evolutionary algorithm For each adversarial perturbation matrix, S5 is performed after each iteration.

Among them, when there is no target attack, the fitness is the negative value of the probability of the original class in the output vector of the target deep learning model, and when there is a target attack, the fitness is the probability of the target class in the output vector of the target deep learning model. See Figure 3, the principle of the evolutionary algorithm, initialize the population according to human experience, then mutate and cross, select with the maximum fitness as the optimization goal, and judge whether the attack target is reached according to the result of the selection knot. If the attack target is reached, output the corresponding Adversarial examples, otherwise, continue to iterate within the iteration limit. The evolutionary algorithm is an adaptive differential evolution strategy or a linearly stretched covariance matrix adaptive evolution strategy.

When the evolutionary algorithm is an adaptive differential evolution strategy, the specific method of S4 is:

S401: In [- against infinity norm, against infinity norm], random values are taken as parameters in the adversarial disturbance matrix, and a preset number of adversarial disturbance matrices are constructed, generally 20; the parameters in the adversarial disturbance matrix are the same as the original Each pixel of the image corresponds one-to-one.

S402: According to the adaptive scale factor and the mutation formula, mutate a preset number of anti-disturbance matrices to obtain a preset number of mutant anti-disturbance matrices, and use both the anti-disturbance matrix and the mutant anti-disturbance matrix as individuals; according to the anti-zero norm and the sensitivity matrix to obtain the mask matrix corresponding to each individual; each pixel in the original image is superimposed on each individual, and then multiplied with the mask matrix corresponding to each individual to obtain the perturbation map corresponding to each individual; each individual corresponds to Input the perturbation map of the target deep learning model to obtain the fitness of the perturbation map corresponding to each individual. According to the order of fitness from large to small, a preset number of each individual is selected as the iteratively optimized confrontation perturbation matrix.

Specifically, the variation part of the original differential evolution strategy is:

表示第i代群体中的某个个体，

与

均为随机选取的索引值，F为固定的比例因子。本实施例中，在原本的变异的基础形式上，考虑到在进化(优化)初期可以适当增加比例因子F(变异步长)，将F改为随i变化的形式：in,

represents an individual in the i-th generation population,

and

Both are randomly selected index values, and F is a fixed scale factor. In this embodiment, on the basis of the original variation, considering that the scaling factor F (variable asynchronous length) can be appropriately increased in the early stage of evolution (optimization), F is changed to a form that varies with i:

Among them, AF stands for adaptive factor (adaptive factor), F ₁ and F ₂ are the lower limit and initial value of AF, respectively, α is a factor for adjusting the rate of change of AF, and s is a scaling factor for the iteration number i. Using the improved scale factor for the differential evolution strategy can speed up its convergence.

S403: Update the adversarial disturbance matrix with the iteratively optimized adversarial disturbance matrix, perform S402 iteratively, and perform S5 after each iteration.

When the evolutionary algorithm is the Linear Spanned Covariance Matrix Adaptation Evolutionary Strategies, in order to solve the covariance matrix adaptive evolutionary strategy in solving extremely high-dimensional optimization problems, due to the frequent calculation of the covariance matrix, The problem of high time complexity is caused by reducing the dimension of the n-dimensional (n=w×h) optimization space to a linearly stretched space of k (k is much smaller than n) n-dimensional vectors, that is, setting the output disturbance to k n The weighted sum of the dimensional vectors is optimized by the adaptive evolution strategy of the covariance matrix. Since k is much smaller than n, the time complexity of calculating the covariance matrix is reduced from O(n ² ) to O(k ² ), which greatly reduces the time complexity of the covariance matrix. decline. On this basis, using the covariance matrix adaptive evolution strategy can also achieve better results. The specific method of S4 is:

S411: Randomly generate k n-dimensional vectors as the basis of the stretched space; randomly generate a preset number of k-dimensional vectors through a preset k-dimensional Gaussian distribution, multiply the basis by the k-dimensional vector as a weight, and obtain a preset number of The n-dimensional sample vector is sliced with the anti-infinity norm to obtain the adversarial perturbation matrix.

S412: According to the anti-zero norm and the sensitive matrix, obtain a mask matrix corresponding to each anti-disturbance matrix; superimpose each pixel point in the original image with each anti-disturbance matrix, and then multiply the mask matrix corresponding to each anti-disturbance matrix , obtain the disturbance graph corresponding to each anti-disturbance matrix; input the disturbance graph corresponding to each anti-disturbance matrix into the target deep learning model to obtain the fitness of the disturbance graph corresponding to each anti-disturbance matrix. Set a number of anti-disturbance matrices, update the k-dimensional Gaussian distribution according to the set number of anti-disturbance matrices, randomly generate a preset number of optimized k-dimensional vectors according to the updated k-dimensional Gaussian distribution, and multiply the optimized k-dimensional vector by the weight A preset number of optimized n-dimensional sample vectors are obtained, and the n-dimensional sample vectors are optimized to resist infinite norm limiting to obtain an iteratively optimized adversarial perturbation matrix.

S413: Update the adversarial disturbance matrix with the iteratively optimized adversarial disturbance matrix, perform S412 iteratively, and perform S5 after each iteration.

S5: When there is at least one target adversarial disturbance matrix in the adversarial disturbance matrix after the current iteration optimization, the original image is perturbed by the target adversarial disturbance matrix to obtain adversarial samples and output, and the iteration ends; otherwise, return to S4.

The target adversarial perturbation matrix is the original image perturbed by the target adversarial perturbation matrix, which can make the output of the target deep learning model reach the attack target, for example: make the perturbed original image input into the target deep learning model output and the original image input The output of the target deep learning model is different, or the output of the perturbed original image input to the target deep learning model is a preset output result.

To sum up, the method for generating an adversarial sample of the deep learning model of the present invention, by migrating the sensitivity map of the reference model, imposes a threshold to limit the disturbed area to pixels with high sensitivity, so that the distribution of disturbed pixels becomes sparse, Therefore, the adversarial disturbance is more difficult to be perceived. From the perspective of quantization, the two-norm of the adversarial disturbance (the square and the root of the disturbance value of each pixel) can thus be greatly reduced. At the same time, by combining the sensitive graphs for reasonable "trials", trying different combinations of infinite norm l _∞ and zero norm l ₀ , automatically and dynamically adjusting l _∞ and l ₀ for each adversarial attack, improving the efficiency of adversarial attacks The success rate when the number of queries is limited. The whole process is plug-in, in which the evolution strategy part can be replaced by different evolution strategies (such as adaptive differential evolution algorithm and linear stretched covariance matrix adaptive evolution strategy) for optimization, which is convenient and fast. specialty. Because the evolutionary algorithm is parallelizable, the computation of different individuals in the same iteration can be processed in parallel, so it can have higher efficiency in the iterative process. Training the target deep learning model through the generated adversarial samples with low two-norm can further improve the prediction accuracy of the target deep learning model and improve the robustness of the target deep learning model.

After experimental demonstration, in the deep learning model against the sample generation method, even using different reference models, such as VGG16 or DenseNet121, can obtain a smaller two-norm disturbance than the original algorithm, and only use less than 20% of the pixels point, indicating that the method for generating the adversarial samples of the deep learning model of the present invention, the two-norm of the adversarial samples generated by the existing methods has been greatly reduced.

The following are apparatus embodiments of the present invention, which can be used to execute method embodiments of the present invention. For details that are not omitted in the device embodiments, please refer to the method embodiments of the present invention.

In yet another embodiment of the present invention, a system for generating adversarial samples for deep learning models is provided, which can be used to implement the above-mentioned method for generating adversarial samples for deep learning models. Specifically, the system for generating adversarial samples for deep learning models includes a sensitivity matrix acquisition module , a perturbation graph acquisition module, an optimization module, a parameter determination module and an output module.

Among them, the sensitivity matrix acquisition module is used to acquire the sensitivity matrix of the original image based on the target deep learning model; the perturbation map acquisition module is used to construct several norm groups according to preset zero norm and several infinite norm for perturbation; The sensitivity matrix and several norm groups are used to obtain the perturbation map corresponding to each norm group; the parameter determination module is used to query the target deep learning model for both the original image and the perturbation map corresponding to each norm group, and obtain and compare the perturbation map corresponding to each norm group. The anti-zero norm and the anti-infinity norm are obtained from the drop value of the original class prediction probability of the original image; the optimization module is used to construct a preset number of adversarial disturbance matrices according to the anti-infinity norm, and the predicted probability of the attack target class is the fitness, Taking the maximum fitness as the optimization goal, according to the original image, the sensitive matrix and the confrontation zero norm, each confrontation disturbance matrix is iteratively optimized through the evolutionary algorithm, and the output module is triggered after each iteration; the output module is used for the confrontation after the current iteration optimization. In the perturbation matrix, when there is at least one target confrontation perturbation matrix, the original image is perturbed by the target confrontation perturbation matrix, and the confrontation sample is obtained and output, and the iteration ends; otherwise, the optimization module is triggered.

In yet another embodiment of the present invention, a computer device is provided, the computer device includes a processor and a memory, the memory is used for storing a computer program, the computer program includes program instructions, and the processor is used for executing the computer Program instructions stored in the storage medium. The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (ASICs), and off-the-shelf programmable gate arrays. (Field-Programmable GateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, specifically suitable for One or more instructions in the computer storage medium are loaded and executed to implement the corresponding method process or corresponding function; the processor according to the embodiment of the present invention can be used for the operation of the deep learning model adversarial sample generation method.

In yet another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), where the computer-readable storage medium is a memory device in a computer device for storing programs and data . It can be understood that, the computer-readable storage medium here may include both a built-in storage medium in a computer device, and certainly also an extended storage medium supported by the computer device. The computer-readable storage medium provides storage space in which the operating system of the terminal is stored. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor, so as to implement the corresponding steps of the method for generating an adversarial sample of a deep learning model in the foregoing embodiment.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be Modifications or equivalent replacements are made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (10)

1. A method for generating confrontation samples of a deep learning model is characterized by comprising the following steps:

s1: acquiring a sensitive matrix of an original image based on a target deep learning model;

s2: constructing a plurality of norm groups according to a plurality of preset zero norms and a plurality of infinite norms for disturbance; obtaining a disturbance diagram corresponding to each norm group according to the sensitive matrix and the plurality of norm groups;

s3: inquiring a target deep learning model for the original image and the disturbance graphs corresponding to the range groups to obtain and obtain a zero norm countermeasure and an infinite norm countermeasure according to a reduction value of the original class prediction probability of the disturbance graphs corresponding to the range groups relative to the original image;

s4: constructing a preset number of confrontation disturbance matrixes according to the confrontation infinite norm, taking the prediction probability of the attack target class as the fitness, taking the fitness as the maximum optimization target, iteratively optimizing each pair of confrontation disturbance matrixes through an evolutionary algorithm according to the original image, the sensitivity matrix and the confrontation zero norm, and performing S5 after each iteration;

s5: when at least one target anti-disturbance matrix exists in the anti-disturbance matrix after current iterative optimization, disturbing the original image through the target anti-disturbance matrix to obtain an anti-sample and output the anti-sample, and finishing iteration; otherwise, return to S4.

2. The method for generating confrontation samples of deep learning model according to claim 1, wherein the specific method of S1 is:

acquiring a deep learning model with the same classification target as the target deep learning model as a reference model;

selecting pixel points in the original image one by one to carry out the following steps: disturbing the current pixel point by a preset size to obtain a sensitive image, inputting the sensitive image into a reference model to obtain a change value of the original class prediction probability output relative to the reference model before disturbance after the current pixel point is disturbed, and taking the change value as the sensitive value of the current pixel point;

and arranging the sensitive values of all pixel points in the original image according to the positions of all the pixel points in the original image to obtain a sensitive matrix of the original image based on the target deep learning model.

3. The method for generating confrontation samples of deep learning model according to claim 2, wherein the specific method of S2 is:

selecting zero norms in the plurality of zero norms one by one to traverse the plurality of zero norms according to a plurality of preset zero norms and a plurality of infinite norms for disturbance, and combining the selected zero norms and the infinite norms in the plurality of infinite norms one by one to obtain a plurality of norm groups;

presetting an initial disturbance matrix, wherein the number of parameters in the initial disturbance matrix is the same as the number of pixel points in the original image, the parameters are arranged according to the positions of the pixel points in the original image, the absolute value of each parameter is 1, and the symbols of the parameters at the same position in the initial disturbance matrix and the sensitive matrix are consistent; selecting a norm group one by one, traversing a plurality of norm groups, multiplying half of infinite norms in the current norm group by each parameter in the initial disturbance matrix to obtain disturbance matrixes corresponding to the norm groups, obtaining zero norms in the current norm group, setting the front zero norm sensitive values in the sensitive matrixes to be 1 and setting the other sensitive values to be 0 according to the descending order of the sensitive values, and obtaining mask matrixes corresponding to the disturbance matrixes;

and superposing the pixel points in the original image with the disturbance matrixes corresponding to the norm groups respectively, and multiplying the pixel points by the mask matrixes corresponding to the disturbance matrixes to obtain the disturbance graphs corresponding to the norm groups.

4. The method for generating confrontation samples of deep learning model according to claim 1, wherein the specific method of S3 is:

inputting the original image and the disturbance images corresponding to the norm groups into a target deep learning model to obtain the original class prediction probability of the original image and the original class prediction probability of the disturbance images corresponding to the norm groups, and obtaining a reduction value of the original class prediction probability of the disturbance images corresponding to the norm groups relative to the original image;

superposing disturbance graphs corresponding to norm groups of the same zero norm relative to the descending values of the original type prediction probability of the original image in all the norm groups to obtain a plurality of first superposed values, and selecting the zero norm corresponding to the first superposed value with the largest slope change in the plurality of first superposed values as the anti-zero norm;

and superposing disturbance graphs corresponding to norm groups of the same infinite norm with respect to the descending values of the original type prediction probability of the original image in all the norm groups to obtain a plurality of second superposed values, and selecting the infinite norm corresponding to the second superposed value with the largest slope change in the plurality of second superposed values as the confrontation infinite norm.

5. The method for generating the confrontation sample of the deep learning model as claimed in claim 1, wherein the evolutionary algorithm in S4 is: an adaptive differential evolution strategy or a linear-stretched covariance matrix adaptive evolution strategy.

6. The method for generating the confrontation sample of the deep learning model as claimed in claim 5, wherein when the evolutionary algorithm is the adaptive differential evolution strategy, the specific method of S4 is as follows:

s401: randomly taking values in [ -confrontation infinite norm, confrontation infinite norm ] as parameters in the confrontation disturbance matrix, and constructing a preset number of confrontation disturbance matrices; each parameter in the anti-disturbance matrix corresponds to each pixel point of the original image one by one;

s402: according to the adaptive scale factor and the variation formula, a preset number of the antagonistic disturbance matrixes are varied to obtain a preset number of the variable antagonistic disturbance matrixes, and the antagonistic disturbance matrixes and the variable antagonistic disturbance matrixes are used as individuals; acquiring a mask matrix corresponding to each body according to the anti-zero norm and the sensitive matrix; respectively superposing each individual on each pixel point in the original image, and multiplying the individual with the mask matrix corresponding to each individual to obtain a disturbance diagram corresponding to each individual; inputting the disturbance graphs corresponding to the individuals into a target deep learning model to obtain the fitness of the disturbance graphs corresponding to the individuals, and selecting a preset number of the individuals as an anti-disturbance matrix after iterative optimization according to the sequence of the fitness from large to small;

s403: and updating the anti-disturbance matrix with the iteratively optimized anti-disturbance matrix, and iterating S402 and after each iteration, performing S5.

7. The method for generating the confrontation sample of the deep learning model according to claim 5, wherein when the evolutionary algorithm is a linear-stretched covariance matrix adaptive evolution strategy, the specific method of S4 is as follows:

s411: randomly generating k n-dimensional vectors as a basis of a span space; randomly generating a preset number of k-dimensional vectors through preset k-dimensional Gaussian distribution, multiplying the k-dimensional vectors by a substrate by taking the k-dimensional vectors as weights to obtain a preset number of n-dimensional sample vectors, and limiting the n-dimensional sample vectors by an anti-infinite norm to obtain an anti-disturbance matrix;

s412: acquiring a mask matrix corresponding to each anti-disturbance matrix according to the anti-zero norm and the sensitive matrix; superposing each anti-disturbance matrix on each pixel point in the original image respectively, and multiplying the anti-disturbance matrixes by the corresponding mask matrixes of the anti-disturbance matrixes to obtain disturbance graphs corresponding to the anti-disturbance matrixes; inputting a disturbance graph corresponding to each anti-disturbance matrix into a target deep learning model to obtain the fitness of the disturbance graph corresponding to each anti-disturbance matrix, selecting a set number of anti-disturbance matrices according to the sequence of the fitness from large to small, updating k-dimensional Gaussian distribution according to the set number of anti-disturbance matrices, randomly generating a preset number of optimized k-dimensional vectors according to the updated k-dimensional Gaussian distribution, multiplying the optimized k-dimensional vectors by a weight and a substrate to obtain a preset number of optimized n-dimensional sample vectors, and limiting the optimized n-dimensional sample vectors by an infinite norm to obtain an iteratively optimized anti-disturbance matrix;

s413: and updating the anti-disturbance matrix with the iteratively optimized anti-disturbance matrix, and iterating S412, and performing S5 after each iteration.

8. A system for generating confrontation samples of a deep learning model is characterized by comprising a sensitive matrix acquisition module, a disturbance map acquisition module, an optimization module, a parameter determination module and an output module;

the sensitive matrix acquisition module is used for acquiring a sensitive matrix of the original image based on the target deep learning model;

the disturbance diagram acquisition module is used for constructing a plurality of norm groups according to a plurality of preset zero norms and a plurality of infinite norms for disturbance; obtaining a disturbance diagram corresponding to each norm group according to the sensitive matrix and the plurality of norm groups;

the parameter determination module is used for inquiring the target deep learning model from the original image and the disturbance graphs corresponding to the range groups to obtain and obtain a zero-norm countermeasure and an infinite norm countermeasure according to a reduction value of the original class prediction probability of the disturbance graphs corresponding to the range groups relative to the original image;

the optimization module is used for constructing a preset number of resistance disturbance matrixes according to resistance infinite norms, taking the prediction probability of an attack target class as the fitness, taking the fitness as the maximum optimization target, iteratively optimizing each pair of resistance disturbance matrixes through an evolutionary algorithm according to the original image, the sensitivity matrix and the resistance zero norm, and triggering the output module after each iteration;

the output module is used for disturbing the original image through the target disturbance resisting matrix when at least one target disturbance resisting matrix exists in the disturbance resisting matrix after current iteration optimization to obtain a resisting sample and output the resisting sample, and the iteration is finished; otherwise, triggering the optimization module.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of the deep learning model confrontation sample generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for generating confrontation samples of a deep learning model according to any one of claims 1 to 7.

Images