CN114330652A - Target detection attack method and device - Google Patents

Abstract

The invention provides a target detection attack method and device. By converting the countermeasure sample from the traditional optimization mechanism into the generation mechanism, the time required for generating the countermeasure sample is greatly shortened, and meanwhile, the interior of the model does not need to be accessed after the GAN network is trained, so that black box attack can be effectively carried out. The method can effectively guide the training of the network by effectively utilizing the classification and position regression loss output by the target model, and meanwhile, the loss of the feature layer is introduced, so that the characteristics of the image, which are relatively sensitive to the network, in a high-dimensional space can be effectively captured, and the attack success rate can be further improved by disturbing the image. In addition, the added Gaussian filtering module can remove the high-dimensional disturbance of the confrontation sample, leave the low-dimensional disturbance, improve the quality of the picture of the generated confrontation sample and further enhance the attack success rate.

Description

Target detection attack method and device

Technical Field

The invention belongs to the field of deep learning counterattack, and particularly relates to a target detection attack method and device.

Background

With the development of software and hardware technologies, deep learning technologies represented by convolutional neural networks are widely applied to many computer vision tasks, such as image classification, target detection, semantic segmentation, scene text recognition, and the like. Despite the great success of deep learning on these tasks, recent studies have shown that neural networks are vulnerable to challenge samples. Szegydy et al for the first time found that by adding small perturbations to the original sample that are hard to perceive by humans, the resulting pictures made this way would make the neural network unable to correctly classify the pictures, and he called "confrontational samples" the pictures added with a certain perturbation. The countermeasure sample relates to the problem of deep learning security, and is concerned by many scholars at home and abroad, and many counterattack methods aiming at image classification are proposed, such as FGSM, Deepfol, C & W, MI-FGSM, and the like. Nowadays, the countersample is not only presented to the image classification task, but also other visual tasks begin to be attacked by the countersample, and the target detection is one of them.

The target detection is one of the most core tasks in the computer vision field, and is closely related to many other tasks, such as target tracking, semantic segmentation and the like, and the tasks are realized based on the target detection technology. Meanwhile, target detection is widely applied to safety-critical scenes such as industrial control and aerospace, and therefore, the method is particularly important for researching the safety of target detection. In 2017, Lu et al added disturbances to the stop flag and face pictures for the first time to mislead the Faster R-CNN detector and successfully deceive the detector, and then developed a series of studies on the target detection field. Xie et al propose DAG methods to attack Faster R-CNN, which assign an error label to all detector extracted proposed regions and then let the detector classify the regions as errors by a gradient optimization strategy. Although the number of attacks for target detection is infinite, most methods inevitably suffer from several problems: 1) generating challenge samples requires a significant amount of time. For example, DPatch generation requires tens or even hundreds of thousands of iterations to generate a valid Patch, and the method proposed by Darren [ herein incorporated ] requires an inference time of hundreds of seconds to generate a challenge sample. 2) Most are white-box attacks, which require knowledge of the parameters of the model. The existing attack method is usually based on a gradient optimization strategy, when a countermeasure sample is generated, parameters of an actual model need to be obtained, and noise is optimized through back propagation to obtain the countermeasure sample. However, in actual attack, an attacker is often faced with a black box model of unknown type, and no internal parameters are known, so that it is particularly important to develop a method for black box attack.

Disclosure of Invention

An object of the present invention is to provide a target detection attack method, which addresses the deficiencies of the prior art.

The method comprises the following specific steps:

firstly, preprocessing a clean original picture x;

inputting the preprocessed picture into a generator G to obtain antagonistic noise G (x); smoothing the antagonistic noise G (x) by two-dimensional convolution kernel Gaussian filtering, and smoothing the smoothed antagonistic noise x_advAdding the picture to the original picture x before preprocessing to form a confrontation picture

Step three, the confrontation picture

Inputting the loss into a discriminator D, and calculating the GAN loss;

the GAN loss is as in equation (1):

wherein D (x) represents the result of discrimination of the original picture x;

representing a Pair contrast Picture

The result of the discrimination of (1); e_xRepresents the mathematical distribution expectation of x.

Step four, the confrontation picture

Inputting the data into a target network to be attacked, and calculating the resistance loss L_adv；

The countermeasure loss comprises a confidence loss and a regression loss;

the confidence loss is used for classifying the region of interest in the picture as a background; the confidence loss function is as follows (2):

where M represents the total number of proposed frames, l, of the target network to be attacked_BCERepresenting a binary cross entropy loss, C_iA confidence score representing the ith suggestion box;

the regression loss is used for interfering the detected object to make the position of the detection frame deviate from the true value, and the specific method is to assign a fixed position coefficient (delta x) to the suggestion frame detected by the target network to be attacked_j，Δy_j，Δw_j，Δh_j) Then, calculating the distance between the position coefficient of the proposed frame output by the target network and the artificially set target position coefficient, wherein the regression loss function is as the following formula (3):

wherein z is_jE {0,1}, represents the existence of the object in the jth suggestion box,

a position coefficient threshold value representing an artificially set target; (Δ x)_j，Δy_j，Δw_j，Δh_j) Position coefficient representing the jth suggestion box, where (Δ x)_j，Δy_j) Coordinates representing the centre point of the jth suggestion box, Δ w_j，Δh_jWidth and height of the jth suggestion box;

therefore, the resistance loss is as in formula (4):

L_adv＝L_confidence+μ·L_regression (4)

where μ is a hyperparameter balancing the confidence loss and the positional regression loss.

Step five, calculating the Euclidean distance between the generated countermeasure picture and the original picture to obtain the disturbance loss L_perturb(ii) a Simultaneously inputting the confrontation picture into a feature extraction network in a target network to be attacked, and calculating the loss L of a feature layer_feature。

Step six: calculating total loss according to the loss obtained in the third, fourth and fifth steps, then minimizing the total loss through back propagation, and updating parameters of a generator and a discriminator to obtain a training weight;

L_all＝α/L_feature+L_GAN+β·L_adv+γ·L_perturb (5)

wherein, alpha, beta and gamma are L_feature、L_adv、L_perturbThe weight coefficient of (2).

And step seven, loading the training weight obtained in the step six by the generator, and generating a corresponding countermeasure picture from the input original picture.

Preferably, the disturbance loss L is_perturbThe method is used for limiting the noise generated by the generator, and is calculated according to the formula (6):

L_perturb＝E_x(||G(x)||₂) (6)

in the formula | · | non-conducting phosphor₂Is L₂Norm, which is used to limit the magnitude of the generated noise.

Preferably, the characteristic layer loss L is_featureThe method comprises the following steps of sending a generated countermeasure picture and an original picture into a feature extractor, extracting features of each layer, calculating the distance between the two layers, maximizing a loss function during training, and enabling the features of the generated countermeasure picture to be far away from the features of the original picture so as to further attack, wherein the feature layer loss function is expressed as a formula (7):

wherein F represents the total number of feature layers in the feature extractor, F represents the F-th feature layer of the set, and T (L (x, F)) represents the strong normalization of the F-th feature map extracted from the original picture x.

Preferably, the generator G comprises seven convolutional blocks, a convolutional layer and a Tanh activation function in sequence, wherein each convolutional block comprises a convolutional layer and a LeakyReLU activation function in sequence.

Preferably, the discriminator comprises a convolutional layer, a LeakyReLU activation function, three convolutional blocks, a convolutional layer and a Sigmoid layer in sequence. Wherein each convolutional block comprises a convolutional layer, a BN layer, and a leakage relu activation function in turn.

Preferably, in the training process, the generator and the arbiter adopt an alternate training mode: firstly, training the discriminator once, updating the parameters of the discriminator, then training the generator for multiple times, and finally realizing the parameter updating of the generator.

Preferably, the two-dimensional convolution kernel gaussian filtering operation is as shown in formula (9):

x_adv＝τ_k*G(x) (9)

in the formula, τ_kA gaussian kernel representing a kernel size k; x is the number of_advThe noise generated by the gaussian kernel of k × k is convolved to be smoothed.

Preferably, the feature extraction network is a backbone network of a target network to be attacked.

It is another object of the present invention to provide a computing device comprising a memory having stored therein executable code and a processor that, when executing the executable code, implements the method described above.

The invention has the beneficial effects that:

1. the invention provides a target detection counterattack based on GAN, which converts the traditional iterative optimization attack method based on gradient into a generation mechanism so as to improve the generation speed of countersamples and greatly save the calculation cost.

2. The method provided by the invention introduces confidence loss and position loss of target detection to train a generator, so that the generated countermeasure sample can effectively deceive a target detection model; meanwhile, in order to solve the defect that noise generated by the GAN network training is too obvious, Gaussian filtering is connected to the output end of the generator, and the noise with too large disturbance is filtered, so that the generated noise is smoother and is not easy to be perceived by human eyes.

3. The method provided by the invention is not only greatly higher than the traditional target detection attack method based on gradient in the generation speed, but also better than the traditional methods in the attack effect on the detector.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

Fig. 2 is a graph of the effect of the attack against the sample generated in the present invention.

Fig. 3 is a graph comparing the results of the method proposed by the present invention with other methods.

Detailed Description

The invention is further analyzed with reference to the following specific examples.

As shown in fig. 1, a target detection attack method includes the following steps:

step one, the picture x used for training in the present invention is all pictures of train + val of the public data set VOC 2007. Before the pictures are input to the generator, the pictures are first resize to 300 x 300 while the corresponding label positions and sizes are modified. In addition, the target network FasterR-CNN is trained to a better weight for detecting objects, participates in later training and testing, and takes the feature extraction network VGG16 of FasterR-CNN as a feature extractor.

Inputting the preprocessed picture into a generator G to obtain antagonistic noise G (x); smoothing the antagonistic noise G (x) by two-dimensional convolution kernel Gaussian filtering, and smoothing the smoothed antagonistic noise x_advAdding the picture to the original picture x before preprocessing to form a confrontation picture

Step three, the confrontation picture

Inputting the loss into a discriminator D, and calculating the GAN loss;

the GAN loss is as in equation (10):

wherein D (x) represents the result of discrimination of the original picture x;

representing a Pair contrast Picture

The result of the discrimination of (1); e_xRepresents the mathematical distribution expectation of x. Judging a real sample (namely an original picture x), wherein the judgment result of a discriminator is expected to be better as being closer to 1, so that the loss function is logD (x); for the generated confrontation picture, the discrimination result of the discriminator is desired

The closer to 0, the better, so its loss function is

Step four, the confrontation picture

Inputting the result into a target network FasterR-CNN to be attacked, and calculating the resistance loss L_adv；

The countermeasure loss comprises a confidence loss and a regression loss;

the confidence loss is used for classifying the region of interest in the picture as a background; the confidence loss function is as follows (11):

where M represents the total number of proposed frames detected by the RPN network, these filtered proposed frames are often considered to contain objects, l_BCERepresenting a binary cross entropy loss, C_iA confidence score representing the ith suggestion box;

the regression loss is used for interfering the detected object and enabling the position of the detection frame to deviate from a true value, and the specific method is that a fixed position coefficient is appointed for the suggestion frame detected by the target network to be attacked, then the distance between the position coefficient of the suggestion frame output by the target network and the fixed position coefficient is calculated, and the regression loss function is as shown in formula (12):

wherein z is_j∈{0,1}，z_j1 represents that the jth suggestion box contains objects, otherwise no objects,

a position coefficient threshold value representing an artificially set target; (Δ x)_j，Δy_j，Δw_j，Δh_j) Position coefficient representing the jth suggestion box, where (Δ x)_j，Δy_j) Indicates the jth suggestion box centerCoordinates of points,. DELTA.w_j，Δh_jWidth and height of the jth suggestion box;

therefore, the overall formula for the penalty loss contains two losses, the formula is given by formula (13):

L_adv＝L_confidence+μ·L_regression (13)

where μ is a hyperparameter balancing the confidence loss and the positional regression loss and is empirically set to 0.4.

Step five, calculating the Euclidean distance between the generated countermeasure picture and the original picture to obtain the disturbance loss L_perturb(ii) a Simultaneously inputting the confrontation picture into a feature extraction network in a target network to be attacked, and calculating the loss L of a feature layer_feature。

Step six: and (4) solving the total loss of the losses obtained in the third step, the fourth step and the fifth step according to an equation (14), minimizing the total loss through back propagation, and updating parameters of a generator and a discriminator to obtain the training weight.

L_all＝α/L_feature+L_GAN+β·L_adv+γ·L_perturb (14)

Wherein, alpha, beta and gamma are L_feature、L_adv、L_perturbThe weight coefficient of (2).

And step seven, loading training weights by the generator, and generating corresponding countermeasure pictures from the input original pictures.

Said disturbance loss L_perturbUsed to limit the amount of noise generated by the generator is L for the noise₂Distance calculation, the calculation mode is as follows (15):

L_perturb＝E_x(||G(x)||₂) (15)

in the formula | · | non-conducting phosphor₂Is L₂Norm, which is used to limit the size of the generated noise;

the characteristic layer loss L_featureThe method comprises the steps of sending generated countermeasure pictures and original pictures into a feature extractor, extracting features of each layer, calculating the distance between the features, maximizing the loss function during training, and enabling the features of the generated countermeasure pictures to be far away from the features of the original picturesAnd thus further aggressive, the feature layer loss function is expressed as equation (16):

in the formula, F represents the total number of feature layers in the feature extractor, in this example, all layers of target feature extraction are selected, F represents the F-th feature layer of the set, and T (L (x, F)) represents strong normalization of the F-th feature map extracted from the original picture x.

The generator G comprises seven convolution blocks, a convolution layer and a Tanh activation function in sequence, wherein each convolution block comprises a convolution layer and an LeakyReLU activation function in sequence.

The discriminator comprises a convolution layer, a LeakyReLU activation function, three convolution blocks, a convolution layer and a Sigmoid layer in sequence. Wherein each convolutional block comprises a convolutional layer, a BN layer, and a leakage relu activation function in turn.

In the training process, the generator and the discriminator adopt an alternate training mode: firstly, training the discriminator once, updating the parameters of the discriminator, then training the generator for multiple times, and finally realizing the parameter updating of the generator.

The two-dimensional convolution kernel Gaussian filtering is to utilize a Gaussian kernel with a certain size to convolute generated noise to remove high-frequency information. The benefits of this are two: 1) the generated noise can be smoothed by the gaussian filtering operation, and some protruding noise can be filtered out. The processed noise is added to the original image x to form a confrontation picture

The perception is much better than that of the untreated; 2) and the attack performance can be more effectively improved by removing the high-frequency disturbance and leaving the low-frequency disturbance. The gaussian filtering operation is shown in equation (17):

x_adv＝τ_k*G(x) (17)

in the formula, τ_kIndicates a kernel size of k highA kernel, k in this example is selected to be 5; x is the number of_advThe noise generated by the gaussian kernel of k × k is convolved to be smoothed.

The feature extraction network is shared by a backbone network VGG16 of the Faster R-CNN, and the weight used by VGG16 and the backbone network of the Faster R-CNN. Fig. 2 is a graph of the effect of the attack against the sample generated in the present invention.

In order to verify the effectiveness of the invention, the existing various anti-attack methods aiming at the target detection model are as follows: DAG, RAP, UEA, etc. were subjected to comparative experiments. The data set used for the experiment was paschaloc 2007. The paschaloc 2007 dataset is divided into four major classes: vehicle, household, animal, person, on this basis, can be subdivided into 20 subclasses, 9963 pictures in total. Each picture is associated with a corresponding XML file containing the location and type of the picture object. The whole data set consists of three parts of train/val/test, two parts of train + val are selected as a GAN training set in the experiment, 5011 pictures are used in total, and a test part of VOC2007 is selected as a test set.

In order to verify the effectiveness of the Attack method provided by the invention, an Attack Success Rate (ASR) is selected as an evaluation index which represents the change condition of mAP (mean average precision) before and after the Attack, and the mAP represents the average value of the average precision Rate of all categories, which is the most important index for measuring the detection effect of the target detector. ASR is calculated as formula (18):

in the formula mAP_cleanmAP, mAP representing target detector before attack_attackFor the mAP of the target detector after attack, the higher the ASR, the stronger the attack.

In order to measure the time cost of the attack method for generating the countermeasure sample. And selecting the whole test set to generate a countermeasure sample, and taking an average value as a time cost evaluation index for measuring the attack method.

To evaluate the attack method proposed by the present inventionGenerating a noise magnitude of the challenge sample using L₂The distance was used as an evaluation index.

The results of the experiment are shown in table 1. The method has the highest attack success rate which reaches 97.8 percent, is 10 percent higher than that of a UEA model and is 5.75 percent higher than that of a DAG model which is the second highest. In terms of generation speed, the method provided by the invention and the UEA method based on GAN at the same time are both 0.06s, which is far faster than other methods. On the generated disturbance, the method provided by the invention is higher than DAG and RAP on the L2 distance index of noise, because the method is not generated in a back propagation 'special' way for each picture in actual generation, but the countermeasure sample generated by the method is lower than UEA on the L2 distance index, and the method can generate a picture with better quality than UEA as shown in combination with FIG. 3. Generally, the method provided by the invention is superior to the existing methods, and can effectively attack the target detection task.

TABLE 1 results of attacks on FasterR-CNN by different methods

Claims (10)

1. A target detection attack method is characterized by comprising the following steps:

firstly, preprocessing a clean original picture x;

inputting the preprocessed picture into a generator G to obtain antagonistic noise G (x); smoothing the antagonistic noise G (x) by two-dimensional convolution kernel Gaussian filtering, and smoothing the smoothed antagonistic noise x_advAdding the picture to the original picture x before preprocessing to form a confrontation picture

Step three, the pairsAnti-picture

Inputting the loss into a discriminator D, and calculating the GAN loss;

the GAN loss is as in equation (1):

wherein D (x) represents the discrimination result of the original picture x;

representing a Pair contrast Picture

The result of the discrimination of (1); e_xRepresents a mathematical distribution expectation of x;

step four, the confrontation picture

Inputting the data into a target network to be attacked, and calculating the resistance loss L_adv；

The countermeasure loss comprises a confidence loss and a regression loss;

the confidence loss is used for classifying the region of interest in the picture as a background; the confidence loss function is as follows (2):

where M represents the total number of proposed frames for the target network to be attacked, l_BCERepresenting a binary cross entropy loss, C_iA confidence score representing the ith suggestion box;

the regression loss is used for interfering the detected object, so that the position of the detection frame deviates from the true value; the regression loss function is as follows (3):

wherein z is_jE 0,1 represents the presence of an object in the jth proposed box,

a position coefficient threshold representing a target; (Δ x)_j，Δy_j，Δw_j，Δh_j) Position coefficient representing the jth suggestion box, where (Δ x)_j，Δy_j) Coordinates representing the centre point of the jth suggestion box, Δ w_j，Δh_jWidth and height of the jth suggestion box;

the countermeasure loss is as follows:

L_adv＝L_confidence+μ·L_regression (4)

wherein μ is a hyperparameter balancing confidence loss and position regression loss;

step five, calculating the Euclidean distance between the generated countermeasure picture and the original picture to obtain the disturbance loss L_perturb(ii) a Simultaneously inputting the confrontation picture into a feature extraction network in a target network to be attacked, and calculating the loss L of a feature layer_feature；

Step six: calculating total loss according to the loss obtained in the third, fourth and fifth steps, then minimizing the total loss through back propagation, and updating parameters of a generator and a discriminator to obtain a training weight;

L_all＝α/L_feature+L_GAN+β·L_adv+γ·L_perturb (5)

wherein alpha, beta and gamma are respectively L_feature、L_adv、L_perturbThe weight coefficient of (a);

and step seven, loading the training weight obtained in the step six by the generator, and generating a corresponding confrontation picture from the original picture.

2. A method of target detection attack according to claim 1,the method is characterized in that: said disturbance loss L_perturbUsed to limit the amount of noise generated by the generator is L for the noise₂And (3) distance calculation:

L_perturb＝E_x(||G(x)||₂) (6)

in the formula | · | non-conducting phosphor₂Is L₂Norm, which is used to limit the magnitude of the generated noise.

3. The method of claim 1, wherein the target detection attack comprises: the characteristic layer loss L_featureIs calculated as follows:

wherein F represents the total number of feature layers in the feature extractor, F represents the F-th feature layer of the set, and T (L (x, F)) represents the strong normalization of the F-th feature map extracted from the original picture x.

4. The method of claim 1, wherein the target detection attack comprises: the generator G comprises seven convolutional blocks, a convolutional layer and a Tanh activation function in sequence, wherein each convolutional block comprises a convolutional layer and an leakyreu activation function.

5. The method of claim 1 or 4, wherein: the discriminator comprises a convolution layer, a LeakyReLU activation function, three convolution blocks, a convolution layer and a Sigmoid layer in sequence; wherein each convolutional block comprises a convolutional layer, a BN layer, and a leakage relu activation function in turn.

6. The method of claim 1, wherein the target detection attack comprises: in the training process, the generator and the discriminator adopt an alternate training mode: firstly, training the discriminator once, updating the parameters of the discriminator, then training the generator for multiple times, and finally realizing the parameter updating of the generator.

7. The method of claim 1, wherein the target detection attack comprises: the two-dimensional convolution kernel Gaussian filtering operation in the second step is shown as the formula (9):

x_adv＝τ_k*G(x) (9)

wherein tau is_kA gaussian kernel representing a kernel size k; x is the number of_advThe noise generated by the gaussian kernel of k × k is convolved to be smoothed.

8. The method of claim 1, wherein the target detection attack comprises: the feature extraction network is a backbone network of a target network to be attacked.

9. The method of claim 8, wherein the target detection attack comprises: the target network to be attacked is FasterR-CNN, and the feature extraction network is VGG 16.

10. A computing device comprising a memory having stored therein executable code and a processor that, when executing the executable code, implements the method of any of claims 1-9.

Images