CN113609482A - Back door detection and restoration method and system for image classification model - Google Patents

Abstract

The invention discloses a backdoor detection and repair method and system for an image classification model, belonging to the technical field of software technology and information security. The methods of model pruning, migration learning and shallow model training are adopted to obtain the same task as the backdoor model but A series of comparison models without backdoor; using the comparison model to reverse each category of the backdoor model by optimizing the objective function to obtain a series of potential triggers; use the contribution heat map to refine the potential triggers, and only retain the classification results that affect the model The key features of ; based on the difference in the transferability of backdoor triggers and adversarial patches on the control model, distinguish the backdoor triggers and adversarial patches of the refined potential triggers; add the distinguished backdoor triggers to the clean dataset, Remove backdoors in backdoor models by adversarial training. The invention can detect and repair the backdoor of the image classification model by only using a small amount of clean data to generate a normal model.

Description

A backdoor detection and repair method and system for an image classification model

technical field

The invention belongs to the technical fields of software technology and information security, relates to artificial intelligence-oriented security technology, and in particular relates to a backdoor detection and repair method and system for a deep neural network image classification model.

Background technique

In recent years, deep neural network (DNN) has been widely used in computer vision, speech recognition, natural language processing and other fields because of its accurate prediction results. Deep neural networks are even used in some important security fields, such as access control systems, autonomous driving, and medical diagnosis, because their accuracy is comparable to, and sometimes more reliable than, human experts.

However, while deep neural networks are widely used, they also face serious security problems, such as data poisoning attacks, adversarial attacks, backdoor attacks, and so on. In particular, attackers can inject backdoors into deep neural networks during model training to control the behavior of the models. The DNN model injected into the backdoor is basically the same as the model without the backdoor on normal input data, but when it encounters a special "trigger (that is, a special pattern overlaid on the original image)" input, the model will be abnormal. behavior that brings about the desired result of the attacker. The existence of backdoor attacks brings security risks to deep neural networks. For example, by injecting a backdoor into a DNN model, it can be made to misidentify a stop sign with a special sticker (trigger) as a speed limit sign. If a self-driving car were equipped with such a rear door model, a fatal traffic accident could occur.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a backdoor detection and repair method for a deep neural network image classification model. The present invention can detect backdoors that may exist in the model without knowing the backdoor trigger and the backdoor attack target, only use a small amount of clean data, and repair the detected backdoor to generate a normal model.

To achieve the above object, the present invention adopts the following technical solutions:

A backdoor detection and repair method for an image classification model, comprising the following steps:

Based on the clean data set, adopt the methods of model pruning, transfer learning and shallow model training to obtain a series of comparison models with the same task as the backdoor model but without the backdoor;

Reverse each category of the backdoor model by optimizing the objective function with the help of the control model and the clean dataset, and obtain a series of potential triggers, which include backdoor triggers and adversarial patches;

Calculate the contribution heat map based on the clean data set and potential triggers, use the contribution heat map to refine the potential triggers, and only retain the key features that affect the classification results of the model;

Based on the difference in transferability between backdoor triggers and adversarial patches on the control model, distinguish the backdoor triggers and adversarial patches of the refined potential triggers;

The differentiated backdoor triggers are added to the clean dataset, and the backdoor in the backdoor model is removed by adversarial training.

Further, the clean data set is a contaminated training set from a backdoor attack or a data set with a similar data distribution to the contaminated training set. Similar data distribution means that the similarity of the data distribution is higher than a preset index; the data volume of the clean data set 10%-20% of the contaminated training set.

Further, the model pruning method is: removing the backdoor by pruning neurons with low activation rates in the backdoor model, and at the same time restoring the classification accuracy of the model by fine-tuning training;

The transfer learning method is: based on a neural network model similar to the classification task of the backdoor model, a comparison model is obtained through transfer learning training;

The shallow model training method is: simplify the structure of the backdoor model, and train on the simplified model structure to obtain a comparison model.

Further, the objective function is optimized by adjusting the weight of the loss function of the objective function, and the formula is as follows:

Among them, the loss functions L _backdoor and L _clean represent the influence of the backdoor trigger on the classification results of the backdoor model and the control model, respectively, the loss function L _noise is the noise reduction function applied to m; α, β and γ are the weight coefficients of the loss function; Δ and m are the two variables of the objective function optimization and are three-dimensional matrices of the same size as the clean dataset, where _Δ is the pattern that holds the potential triggers; m is the transparency matrix that controls the position of the potential triggers; Randomly selected images in the dataset; J is an all-one matrix with the same dimension as Δ; Δ*m+ _xi *(Jm) means overlaying the trigger on the image _xi ; f _b and f _c are the backdoor model and The prediction function of the control model; CE is the cross-entropy loss function; n is the total number of images in the clean dataset; i is the number of the current image; on the backdoor model, images with triggers are classified to target class y _t , in The control model is classified to the correct category y _i ; j and k represent the rows and columns of the matrix m, respectively, and a and b are the subscripts of the summation symbols.

Further, the steps of calculating the contribution heatmap based on the clean dataset and potential triggers include:

A set of images is randomly selected from a clean dataset and overlaid with potential triggers;

Calculate the heat map representing the contribution of the classification results for all images, which is the contribution heat map.

Further, the steps of using the contribution heatmap to refine potential triggers include:

Average all contribution heatmaps to get the average heatmap;

According to the average heatmap, remove the area with the lowest current contribution among potential triggers;

Calculate the current attack success rate of the potential trigger, if it is lower than a threshold, end, otherwise continue to remove the area with the lowest current contribution in the potential trigger.

Further, the steps of distinguishing backdoor triggers and adversarial patches of the refined potential triggers include:

A set of images is randomly selected from a clean dataset and overlaid with potential triggers;

Calculate the attack success rate of the potential trigger on the backdoor model, if the attack success rate is lower than a threshold, it is determined as an adversarial patch, and the end;

If the attack success rate is not lower than the above threshold, calculate the attack success rate of the potential trigger on all comparison models. If the attack success rate on one comparison model is higher than the other threshold, it is judged as an adversarial patch, otherwise it is judged as Backdoor trigger.

Further, a certain proportion of images are randomly selected from the clean dataset and covered with backdoor triggers; then the discriminated backdoor triggers are added to the clean dataset.

Further, the steps of removing the backdoor in the backdoor model through adversarial training include: adding the differentiated backdoor triggers to the clean data set, and keeping the category information of the images unchanged to obtain the adversarial training data set; fine-tuning with the adversarial training data set. Train the backdoor model and remove the backdoor in the backdoor model.

A backdoor detection and repair system for an image classification model includes a memory and a processor, where a computer program is stored on the memory, and the processor implements the steps of the above method when executing the program.

A computer-readable storage medium stores a computer program, which implements the steps of the above method when the program is executed by a processor.

Compared with the prior art, the positive effects of the present invention are:

The invention has stronger backdoor detection capability, wider backdoor detection range for different types of triggers, less influence by trigger area ratio, position, shape, pattern and other factors, and lower false alarm rate and false alarm rate. Compared with the existing backdoor detection methods (such as NeuralCleanse, ABS, TABOR), all of them put forward assumptions on the area ratio of triggers, so that attackers can sacrifice the concealment of triggers and adopt a larger area ratio. The disadvantage of large triggers (greater than 10%) evading detection, the present invention can still maintain the detection capability when the proportion of the trigger area reaches 25%, and is more difficult to be adaptively attacked.

Description of drawings

FIG. 1 is an overall flowchart of a backdoor detection and repair method for an image classification model according to the present invention.

Figure 2 is a flow chart for the refinement of potential triggers.

Figure 3 is a flowchart of backdoor trigger identification.

Detailed ways

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be further described below through specific embodiments and accompanying drawings.

This embodiment discloses a backdoor detection and repair method for an image classification model, as shown in FIG. 1 , the steps are as follows:

1. The present invention includes the following points:

1.1. Comparison model generation: Simultaneously adopt the methods of model pruning, transfer learning and shallow model training to obtain a series of comparison models with the same tasks as the backdoor model but without the backdoor.

1.2. Potential trigger reversal: With the help of the control model and the clean dataset, an objective function is designed to reverse each category of the backdoor model to obtain a series of potential triggers (consisting of backdoor triggers and adversarial patches).

1.3. Refinement of potential triggers: With the help of the contribution heat map technology, the potential triggers are refined, the redundant features of the potential triggers are removed, and the refined potential triggers are obtained.

1.4. Backdoor Trigger Recognition: Based on the difference in transferability between backdoor triggers and adversarial patches on the control model, the refined potential triggers are divided into two categories: backdoor triggers and adversarial patches.

1.5. Repair of the backdoor model: Add the backdoor trigger to the clean data set, remove the backdoor in the backdoor model through adversarial training, and obtain a normal model without backdoor.

2. The generation of the control model includes the following three methods and adopts them at the same time:

2.1. Model pruning: The backdoor is removed by pruning neurons with low activation rates in the model, and the classification accuracy of the model is restored by fine-tuning training.

2.2. Transfer learning: Based on a model similar to the backdoor model classification task, the control model is trained through transfer learning.

2.3. Shallow model training: Simplify the structure of the backdoor model, and train the control model on the simplified model structure.

3. The potential trigger reverse is done by optimizing the objective function:

Δ and m are the two variables of the objective function optimization, which are three-dimensional matrices of the same size as the clean dataset. where Δ is the pattern holding the potential triggers; m is the transparency matrix that controls the position of the potential triggers. The objective function consists of three loss functions, which are adjusted by three weights α, β and γ.

x _i are randomly selected images from the clean dataset. J is an all-1 matrix with the same dimension as Δ. Δ*m+ _xi *(Jm) means overlaying the trigger on the image _xi . f _b and f _c are the prediction functions of the backdoor model and the control model, respectively. CE is the cross-entropy loss function. n is the total number of images in the clean dataset and i is the number of the current image. L _backdoor and L _clean represent the influence of backdoor triggers on the classification results of the backdoor model and the control model, respectively. On the backdoor model, images with triggers should be classified to the target class y _t , and on the control model should be classified to the correct class _yi . Only one control model is used here. L _noise is the noise reduction function applied to m. j and k represent the row and column of matrix m, respectively, and a and b are the subscripts of the summation symbol. L _noise achieves the purpose of noise reduction by adding the adjacent pixels of m, taking the absolute value, and then summing up.

4. The process of potential trigger refinement is shown in Figure 2, which includes the following steps:

4.1. A set of raw images is randomly selected from the clean dataset and overlaid with latent triggers.

4.2. Calculate the heat map representing the contribution of the classification result for all images (a two-dimensional matrix with the same size as the original image, the larger the value of the point in the matrix, the greater the contribution of the pixel in the same position of the original image to the classification result. large), and average all heatmaps to get the average heatmap.

4.3. Based on the average heatmap, remove the regions with the lowest contribution among potential triggers.

4.4. Calculate the current attack success rate of the potential trigger, if it is lower than the threshold (95% of the attack success rate of the unrefined original potential trigger), then end, otherwise go to step 4.3

5. The process of backdoor trigger recognition is shown in Figure 3, including the following steps:

5.1. Randomly select a set of images from a clean dataset and overlay with potential triggers.

5.2. Calculate the attack success rate of potential triggers on the backdoor model.

5.3. If the attack success rate is lower than a threshold (pre-set hyperparameter, the value is 60%), it is determined as an adversarial patch, and it ends, otherwise, jump to 5.4.

5.4. Calculate the attack success rate of potential triggers on all control models.

5.5. If the attack success rate on a control model is higher than another threshold (pre-set hyperparameters, related to the number of classification categories, 40% on MNIST and GTSRB datasets with a small number of categories, in Youtube-Face and VGG-Face categories are 20%), it is determined as an adversarial patch, otherwise it is determined as a backdoor trigger.

6. The repair of the backdoor model includes the following steps:

6.1. Randomly select a certain proportion of images from the clean dataset and overlay with backdoor triggers.

6.2. Add the image to the clean dataset and keep the category information of the image unchanged to get the adversarial training dataset.

6.3. Fine-tune the training backdoor model with the adversarial training dataset to remove the backdoor in the backdoor model.

This embodiment first imitates the perspective of backdoor attackers, in the three application fields of handwritten digit classification (MNIST data set), traffic sign classification (GTSRB data set) and face classification (Youtube-Face and VGG-Face data sets). On each dataset, 60 backdoor models were generated by two mainstream backdoor attack methods: polluted training set (Badnets) and modified pre-trained model (TrojanNN); at the same time, 30 normal (without backdoor) models were generated by using normal model training method . The "trigger" of the backdoor model is some kind of special pattern covering the original image, occupying 2%-25% of the area, with different positions, shapes and patterns. The present invention achieves that the false alarm rate (the number of normal models with falsely detected backdoors/the total number of normal models) and the false alarm rate (the number of backdoor models without backdoors/the total number of backdoor models) on the above 90 models are both less than 10 % of test results.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Those skilled in the art can modify or equivalently replace the technical solutions of the present invention. The protection scope of the present invention is subject to the claims.

Claims (10)

1. a backdoor detection and repair method for image classification model, is characterized in that, comprises the following steps: Based on the clean data set, adopt the methods of model pruning, transfer learning and shallow model training to obtain a series of comparison models with the same task as the backdoor model but without the backdoor; Reverse each category of the backdoor model by optimizing the objective function with the help of the control model and the clean dataset, and obtain a series of potential triggers, which include backdoor triggers and adversarial patches; Calculate the contribution heat map based on the clean data set and potential triggers, use the contribution heat map to refine the potential triggers, and only retain the key features that affect the classification results of the model; Based on the difference in transferability between backdoor triggers and adversarial patches on the control model, distinguish the backdoor triggers and adversarial patches of the refined potential triggers; The differentiated backdoor triggers are added to the clean dataset, and the backdoor in the backdoor model is removed by adversarial training. 2. The method of claim 1, wherein the model pruning method is: remove the backdoor by pruning the neuron with low activation rate in the backdoor model, and restore the classification accuracy of the model by fine-tuning training simultaneously; The transfer learning method is: based on a neural network model similar to the classification task of the backdoor model, a comparison model is obtained through transfer learning training; The shallow model training method is: simplify the structure of the backdoor model, and train on the simplified model structure to obtain a comparison model. 3. The method of claim 1, wherein the objective function is optimized by adjusting the loss function weight of the objective function, and the formula is as follows:

Among them, the loss functions L _backdoor and L _clean represent the influence of the backdoor trigger on the classification results of the backdoor model and the control model, respectively, the loss function L _noise is the noise reduction function applied to m; α, β and γ are the weight coefficients of the loss function; Δ and m are the two variables of the objective function optimization and are three-dimensional matrices of the same size as the clean dataset, where _Δ is the pattern that holds the potential triggers; m is the transparency matrix that controls the position of the potential triggers; Randomly selected images in the dataset; J is an all-one matrix with the same dimension as Δ; Δ*m+ _xi *(Jm) means overlaying the trigger on the image _xi ; f _b and f _c are the backdoor model and prediction function of the control model; CE is the cross-entropy loss function; n is the total number of images in the clean dataset; i is the number of the current image; on the backdoor model, images with triggers are classified to the target class y _t , in The control model is classified to the correct category y _i ; j and k represent the rows and columns of the matrix m, respectively, and a and b are the subscripts of the summation symbols. 4. The method of claim 1, wherein the step of calculating the contribution degree heatmap according to the clean data set and potential triggers comprises: A set of images is randomly selected from a clean dataset and overlaid with potential triggers; Calculate the heat map representing the contribution of the classification results for all images, which is the contribution heat map. 5. The method of claim 1 or 4, wherein the step of refining the potential triggers using the contribution heat map comprises: Average all contribution heatmaps to get the average heatmap; According to the average heatmap, remove the area with the lowest current contribution among potential triggers; Calculate the current attack success rate of the potential trigger, if it is lower than a threshold, end, otherwise continue to remove the area with the lowest current contribution in the potential trigger. 6. The method of claim 1, wherein the step of distinguishing between the backdoor triggers of the refined potential triggers and the adversarial patches comprises: A set of images is randomly selected from a clean dataset and overlaid with potential triggers; Calculate the attack success rate of the potential trigger on the backdoor model, if the attack success rate is lower than a threshold, it is determined as an adversarial patch, and the end; If the attack success rate is not lower than the above threshold, calculate the attack success rate of the potential trigger on all comparison models. If the attack success rate on one comparison model is higher than the other threshold, it is judged as an adversarial patch, otherwise it is judged as Backdoor trigger. 7. The method of claim 1, wherein a certain proportion of images are randomly selected from the clean data set and covered with backdoor triggers; then the differentiated backdoor triggers are added to the clean data set. 8. The method of claim 1, wherein the step of removing the backdoor in the backdoor model by adversarial training comprises: adding the differentiated backdoor trigger into the clean data set, and keeping the category information of the image unchanged, Obtain the adversarial training dataset; fine-tune the training backdoor model with the adversarial training dataset, and remove the backdoor in the backdoor model. 9. A backdoor detection and repair system for an image classification model, characterized in that it comprises a memory and a processor, and a computer program is stored on the memory, and when the processor executes the program, any one of claims 1-8 is realized the steps of the method. 10. A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method of any one of claims 1-8.

Images