WO2021042665A1 - Dnn-based method for protecting passport against fuzzy attack - Google Patents

Abstract

The present invention provides a DNN-based method for protecting a passport against the fuzzy attack. A DNN model is comprised. The ownership verification scheme of the DNN model comprises an embedding process, a fidelity evaluation process, a signature verification process, and a reversible process. During ownership verification, a private passport and a trigger set are embedded but not distributed, which comprises: embedding the passport and embedding a group of trigger images; detecting and declaring the ownership of a suspicious DNN model by remotely calling a service API; first, declaring the ownership in a black box mode, and then declaring the ownership of a trigger set image again in a white box mode by means of the passport verification, alternately minimizing the original task loss, and reducing a joint loss function containing passport constraints. The range of the network performance significantly changes from 3% to 80% for AlexNet and ResNet trained for classification tasks CIFAR10 and CIFAR100. The similarity between the accuracy of a DNN model providing a valid passport, and the accuracy of an original network exceeds 90%, and a classification accuracy rate of about 10% is achieved for the same DNN model using a fake passport.

Description

A method based on DNN for passport to resist ambiguity attack Technical field

The present disclosure relates to the field of passport security, and specifically is a method based on Deep Neural Networks (DNN) for passports to resist ambiguity attacks.

Background technique

At present, the embedded watermarking methods used by related technologies in the field of machine learning can generally be divided into the following two types:

Method 1: Feature-based method, which embeds the specified watermark into the network weight by adding additional optimization target constraints;

Method 2: The method based on the trigger set relies on the adversarial training samples with specific labels, that is, the backdoor trigger set.

The watermarks embedded in the above two schemes have been successfully proven to be robust against removal attacks, which mainly consist in modifying network weights, such as fine-tuning or pruning.

Summary of the invention

At least some of the embodiments of the present disclosure provide a method for passports to resist ambiguity attacks based on DNN. When using real passports, the performance remains unchanged. Once a modified or forged passport is used, the network performance will be severely degraded. Robust, while being able to resist fuzzing attacks. The technical solutions adopted in the present disclosure are as follows:

A method based on DNN for passports to resist fuzzing attacks, including a DNN model. The ownership verification scheme of the DNN model includes an embedding process E, a fidelity evaluation process F, a signature verification process V, and a reversible process I. The processing steps are as follows :

S11: The embedding process E is a DNN learning process, which takes training data D as input, the training data includes trigger set data T or signature s, and optimizes the model N by minimizing the given loss function L;

S12: Fidelity evaluation process F={False, True}, used to evaluate whether the performance difference is less than the threshold, that is (MM _t )≤ε, where M is the DNN performance tested against a set of test data D, and M _t is Target performance, ε is the threshold, F is the fidelity evaluation result;

S13: The signature verification process V={False, True} is used to check whether the predetermined signature s or trigger set data T is successfully verified for the given neural network N;

S14: When the following conditions are met, there is a reversible process I(N)=N', and a successful ambiguity attack A _{a is} caused:

a) For a given DNN model, infer a new set of trigger set data T'and/or signature s'through reverse engineering;

b) The fake T', s'can be successfully verified with respect to the given DNN weight W, that is, V=True;

c) The fidelity evaluation result F is still True;

S15: The DNN verification scheme V with a reversible process is defined as a reversible scheme V ^l , otherwise it is defined as an irreversible scheme

Optionally, the feature-based and trigger-set-based methods use the combined loss function as follows:

Among them, λ ^t , λ ^r are the weights of related hyperparameters, f(W, X_) is _{the prediction function with input X r} or X _t and output prediction value, L _c is the prediction value and target label y _r or y _T Loss function of cross entropy. Signature s={P,B}, consisting of passport P and signature string B, the constraint item is R=L _c (σ(W,P), B), or R=MSE(B-PW); MSE is both Square error function.

Optionally, the DNN model based on the watermarking method of the trigger set also embeds a private passport and a trigger set, but does not distribute it. The trigger set is a set of trigger images, and the ownership of the suspicious DNN model is detected and declared by remotely calling the service API; Declare ownership in black box mode, and then declare ownership again through passport verification in white box mode. Trigger set images alternately minimize the original mission loss and reduce the joint loss function containing passport constraints. The original mission loss does not include the passport layer, and GroupNormalisation is used. algorithm.

Optionally, the passport is a passport generated after random shuffling. The specific method is: sending a set of N selected images into a training DNN model with the same structure, and collecting N corresponding feature maps in each layer ; Among the N options, only one of each layer is randomly selected as a passport. In particular, for a group of N elementary image having DNN model layer L, N ^L may be generated a total of possible combinations of the passport.

Optionally, the DNN model based on the watermarking method of the trigger set uses the following steps to add the trainable noise component to the randomly selected basic image:

S31: randomly select a group of N basic images T _b ;

S32: Generate a random noise pattern with the same size T _{n as the trainable parameter;}

S33: Use the summation X _T =T _b +ηT _n as the trigger set image, where η=0.04 to make the noise component invisible;

S34: randomly assign a trigger set label y _T ;

S35: Minimize the cross-entropy loss Lc related to the _{trainable parameter T n.}

Optionally, the DNN structure is pre-determined by the optimization model N, and after the DNN weight W is learned, the trigger set T or signature s will be embedded in the model; the signature verification process V first calls the DNN prediction process, and sets the trigger set samples T _{x is taken} as an input, and then it is checked whether the prediction function f generates the designated label _Ty under the condition of the false detection rate less than the threshold value.

Optionally, the DNN model further includes a passport layer and a convolution layer. The scale factor γ and the offset β of the _{passport layer depend on the convolution kernel W p} and the designated passport layer P. The formula is as follows:

Among them, * represents the convolution operation, l is the number of layers, X _p is the input of the passport layer, X _c is the input of the convolution layer; O() is the corresponding linear transformation output, and P _γ ^l and P _β ^l are respectively Derive the passport layer of the scale factor γ and the bias term β; each convolution layer in the convolution layer is composed of several convolution units, and the parameters of each convolution unit are optimized through the backpropagation algorithm, The different features of the fuzzy attack are extracted by convolution operation.

Optionally, use the passport layer s _e = {P _γ ^l , P _β ^l } ^l for the trained DNN model, whose prediction performance M depends on the digital passport provided when using the network, namely:

If the real digital passport s _t ≠s _e cannot be provided, the network performance will deteriorate significantly.

Optional, irreversible solution

The mid-fidelity evaluation result F depends on the presented signature s or trigger set T. If the forged passport s _t ≠s _e , the performance M deteriorates sharply, and the performance difference is greater than the threshold, that is

∈ _f is the threshold.

Optionally, the signature is an embedded binary signature. In the DNN weight learning process, the following symbol loss constraints are added to the combined loss function to force the scale factor to adopt a specified positive or negative sign:

Where B={b ₁ ,···,b _C }∈{-1,1} ^C , consisting of the specified binary bits of the C convolution kernel, γ ₀ is a positive control parameter, the default is 0.1, and the value of the excitation scale factor is greater than γ ₀ .

Optionally, the parameters of the DNN model are divided into a public convolutional layer parameter W and a hidden passport layer scale factor γ and a bias term β. After the passport information is embedded in the weight W, the following constraints must be enforced: Avg(W _p ^l *P _γ ^l )=c _γ ^l , Avg(W _p ^l *P _y ^l )=c _β ^l ; the weight distribution of the convolutional layer is the same as that of the original DNN without a passport layer; c _γ ^l and c _β ^l is the constant value to which the parameters γ ^l and β ^l converge, and the scale factor can only take a positive or negative value away from zero.

The deep neural networks targeted by the embodiments of the present disclosure include all the various forms mentioned above, different input signals, different types, different network structures, different application functions, and deep neural networks on different computing carriers. Any neural network with the same principle, regardless of its operating environment. Optionally, neural networks can run on computer central processing units (CPU), graphics accelerators (GPU), tensor processors (TPU), dedicated artificial intelligence chips, and cloud computing centers, mobile devices, wearable devices, and smart videos Terminals, in-vehicle equipment and other vehicles, IoT devices (IoT devices) and other equipment.

Optionally, the DNN-based method for passports to resist obfuscation attacks can be applied to the above-mentioned terminal devices to generate passports resisting obscuration attacks. The terminal equipment includes an embedded module, a fidelity evaluation module, a signature verification module, a reversible module, and a passport. Generate modules.

Optionally, the embedding module inputs training data D, including trigger set data T or signature s, and optimizes model N by minimizing a given loss function L;

Optionally, the fidelity evaluation module F evaluates whether the performance difference is less than a threshold, that is, (MM _t )≤ε, where M is the DNN performance tested against a set of test data D, M _t is the target performance, and ε is Threshold, F is the fidelity evaluation result;

Optionally, the signature verification module V checks whether the predetermined signature s or trigger set data T is successfully verified for the given neural network N;

Optionally, the reversible module I(N)=N' satisfies the following conditions to exist, and causes a successful obfuscation attack A _a :

d) For a given DNN model, infer a new set of trigger set data T'and/or signature s'through reverse engineering;

e) The fake T', s'can be successfully verified with respect to the given DNN weight W, that is, V=True;

f) The fidelity evaluation result F is still True;

Optionally, the DNN verification scheme V of the reversible module is defined as a reversible scheme, otherwise it is defined as an irreversible scheme;

Optionally, the passport generation module sends a set of N selected images into the training DNN model with the same structure, and collects N corresponding feature maps in each layer; among the N options, there is only one in each layer Was randomly selected as a passport. In particular, for a group of N elementary image having DNN model layer L, N ^L may be generated a total of possible combinations of the passport.

As can be seen from the above technical solutions, the beneficial effects of the present disclosure: using a real passport, the performance remains unchanged, but once a modified or forged passport is used, the network performance will be severely degraded, and it is robust to removal attacks, while being able to resist ambiguity. Attack: The DNN model embeds a private passport and trigger set but does not distribute it, alternately minimizes the original mission loss and reduces the joint loss function containing passport constraints, minimizes the original mission loss such as CIFAR10 classification, but does not include the passport layer.

Description of the drawings

Figure 1 is a structural diagram of the disclosed digital passport layer;

Figure 2 is a diagram showing the performance of the DNN model of different passports of the present disclosure;

Figure 3 is a diagram of an ownership verification scheme embedded with a private passport and trigger set but not distributed;

Figure 4 is a performance diagram of the disclosed CIFAR10 classification against attacks;

Figure 5 is a diagram showing the performance of the disclosed CIFAR100 classification against attacks;

Figure 6 is a performance diagram of the defensive power of the present disclosure;

Figure 7 is a diagram of the passport and the trained DNN model distributed together with the ownership verification scheme;

Figure 8 is a diagram of the ownership verification scheme in which the private passport is embedded in the DNN model but not distributed;

1 is a fake passport, 2 is a passport obtained by reverse engineering, 3 is a valid passport, 4 is the original network DNN, 5 is Signature, 6 is CIFAR10, 7 is CIFAR100, 8 is fake1, 9 is fake2, 10 is valid, and 11 is orig.

detailed description:

The overall technical idea of the present disclosure is to reveal the existence and effectiveness of the obfuscation attack. The purpose of this attack is to question and shake the uniqueness of model ownership verification by forging the watermark of the DNN model. Furthermore, even if the original training data set is not needed, it is possible to use a small computational cost and reverse engineering to forge the watermark to implement the blur attack.

In one of the embodiments of the present disclosure, a method for passports to resist ambiguity attacks based on DNN is provided. When a real passport is used, the performance remains unchanged. Once a modified or forged passport is used, the network performance will be severely degraded. The removal attack is robust, and at the same time it can resist the ambiguity attack.

This DNN-based method for passports to resist fuzzing attacks includes a DNN model. The ownership verification scheme of the DNN model includes an embedding process E, a fidelity evaluation process F, a signature verification process V, and a reversible process I. The processing steps are as follows:

S11: The embedding process E is a DNN learning process, which takes training data D as input, the training data includes trigger set data T or signature s, and optimizes the model N by minimizing the given loss function L;

S12: Fidelity evaluation process F={False, True}, used to evaluate whether the performance difference is less than the threshold, that is (MM _t )≤ε, where M is the DNN performance tested against a set of test data D, and M _t is Target performance, ε is the threshold, F is the fidelity evaluation result;

S13: The signature verification process V={False, True} is used to check whether the predetermined signature s or trigger set data T is successfully verified for the given neural network N;

S14: When the following conditions are met, there is a reversible process I(N)=N', and a successful ambiguity attack A _{a is} caused:

a) For a given DNN model, infer a new set of trigger set data T'and/or signature s'through reverse engineering;

b) The fake T', s'can be successfully verified with respect to the given DNN weight W, that is, V=True;

c) The fidelity evaluation result F is still True;

S15: The DNN verification scheme V with a reversible process is defined as a reversible scheme V ^l , otherwise it is defined as an irreversible scheme

Optionally, the feature-based and trigger-set-based methods use the combined loss function as follows:

Among them, λ ^t , λ ^r are the weights of related hyperparameters, f(W, X_) is _{the prediction function with input X r} or X _T and output prediction value, L _c is the prediction value and target label y _r or y _T Loss function of cross entropy. Signature s={P,B}, consisting of passport P and signature string B, the constraint item is R=L _c (σ(W,P), B), or R=MSE(B-PW); MSE is both Square error function.

The following table shows the effect of the combined loss function used by the feature-based and trigger-set-based watermarking methods, as shown in Table 1:

Table 1

In Table 1, the accuracy of the watermark is detected before and after the fine-tuning of the transfer learning task. Trans.L1 represents the network trained with CIFAR10 and fine-tune the weight of CIFAR100 (top row); Trans.L2 represents the fine-tuning of Caltech-101 (bottom row). The accuracy of the transfer task is outside the brackets, and the original task is inside the brackets. WMDet. represents the detection accuracy of the watermark, where the accuracy outside the brackets correspond to after fine-tuning, and the accuracy inside the brackets correspond to before the fine-tuning.

For the DNN model that performs classification tasks, the network performance M = L _{c obtained by using the test data set D t} ={X _T ,y _T } is independent of the embedded signature s or the trigger set T. It is this independence that leads to The existing watermark-based methods are all reversible.

As shown in Figure 3, the DNN model based on the watermarking method of the trigger set also embeds a private passport and a trigger set, but does not distribute it. The trigger set is a set of trigger images, and the ownership of the suspicious DNN model is detected and declared by remotely calling the service API. ; First declare the ownership in the black box mode, and then declare the ownership again through passport verification in the white box mode. The trigger set image alternately minimizes the original mission loss and reduces the joint loss function containing passport constraints. The original mission loss does not include the passport layer. Using GroupNormalisation algorithm.

The aforementioned passport is a passport generated after random shuffling. In an alternative embodiment, a set of N selected maps are sent to the training DNN model with the same structure, and N corresponding feature maps are collected in each layer; among the N options, only one is selected in each layer. Randomly selected as a passport. In particular, for a group of N elementary image having DNN model layer L, N ^L may be generated a total of possible combinations of the passport.

The DNN model of the watermarking method based on the trigger set uses the following steps to add the trainable noise component to the randomly selected basic image:

S31: randomly select a group of N basic images T _b ;

S32: Generate a random noise pattern with the same size T _{n as the trainable parameter;}

S33: Use the summation X _T =T _b +ηT _n as the trigger set image, where η=0.04 to make the noise component invisible;

S34: randomly assign a trigger set label y _T ;

S35: Minimize the cross-entropy loss Lc related to the _{trainable parameter T n.}

The DNN architecture is pre-determined by the optimization model N, and after the DNN weight W is learned, the trigger set T or signature s will be embedded in the model; the signature verification process V first calls the DNN prediction process and takes the trigger set sample T _x as input , And then check whether the prediction function f generates the specified label _Ty under the condition of the false detection rate less than the threshold.

The DNN model also includes a passport layer and a convolution layer. The scale factor γ and offset β of the _{passport layer depend on the convolution kernel W p} and the designated passport layer P. The formula is as follows:

Among them, * represents the convolution operation, l is the number of layers, X _p is the input of the passport layer, X _c is the input of the convolution layer; O() is the corresponding linear transformation output, and P _γ ^l and P _β ^l are respectively Derive the passport layer of the scale factor γ and the bias term β; each convolution layer in the convolution layer is composed of several convolution units, and the parameters of each convolution unit are optimized through the backpropagation algorithm, The different features of the fuzzy attack are extracted by convolution operation.

Figure 1 depicts the architecture of the digital passport layer used in the ResNet layer. This is an example of the ResNet layer, including two convolutional layers and two passport layers. P ^l ={P _γ ^l , P _β ^l } is a digital passport. F=Avg(W _p ^l *P _{γ, β} ^l ) is a passport function for calculating hidden parameters (ie, γ and β), and it has been given in formula (2).

Figure 2 shows the performance of the DNN model of different passports. Compare the CIFAR10 classification accuracy of the original network DNN4, DNN and valid passport 3, DNN and fake passport 1, and passport 2 obtained by DNN through reverse engineering. % Means) distribution.

Using the passport layer s _e = {P _γ ^l , P _β ^l } ^l for the trained DNN model, its prediction performance M depends on the digital passport provided when using the network, the formula is as follows:

If the real digital passport s _t ≠s _e cannot be provided, the network performance will deteriorate significantly. Because the corresponding scale factor γ and bias term β are calculated based on the wrong passport. For example, as shown in Figure 2, the DNN model that provides a valid passport 3 shows almost the same accuracy as the original network DNN4, while the same DNN model that uses the fake passport 1 only achieves a classification rate of about 10%. The key to the passport layer is to ensure the dependency between the scale factor, the bias term, and the network weight.

Irreversible scheme

The mid-fidelity evaluation result F depends on the presented signature s or trigger set data T. If the forged passport s _t ≠s _e , the performance M deteriorates sharply, and the performance difference is greater than the threshold, that is

∈ _f is the threshold.

The signature is an embedded binary signature. In the DNN weight learning process, the following symbol loss constraints are added to the combined loss function to force the scale factor to adopt a specified positive or negative sign:

Among them, B={b ₁ ,···,b _C }∈{-1,1} ^C , composed of the specified binary bits of the C convolution kernel, γ ₀ is a positive control parameter, and the default is 0.1 to the value of the excitation scale factor Greater than γ ₀ .

The parameters of the DNN model are divided into the public convolutional layer parameter W and the scale factor γ and the bias term β of the hidden passport layer. After the passport information is embedded in the weight W, the following constraints must be enforced: Avg(W _p ^l *P _γ ^l )=c _γ ^l , Avg(W _p ^l *P _y ^l )=c _β ^l ; the weight distribution of the convolutional layer is the same as that of the original DNN without a passport layer; c _γ ^l and c _β ^l are the parameters γ ^l and β ^l converge to a constant value, and the scale factor can only take a positive or negative value away from zero.

Next, we conduct experimental tests on the robustness of fine-tuning, pruning, and various fuzzing attacks.

Regarding the robustness of fine-tuning, the following Table 2 shows the performance (%) of the passport network and the robustness to fine-tuning, where BN=batch standardization GN=group standardization. (Left: Use CIFAR10 to train and transfer to CIFAR100/Caltech-101 task; Right: Use CIFAR100 to train and transfer to CIFAR10/Caltech-101).

In this experiment, for each DNN model, we embed the specified scale factor symbol and repeat the training five times. For the three ownership verification schemes, passport signatures are detected at a detection rate of 100%. As shown in Table 2 below, even after fine-tuning the network for other classification tasks (for example, from CIFAR10 to Caltech-101), the embedded passport still maintains a 100% detection rate. Note that the detected passport signature will only be declared when all binary digits match exactly. We attribute this superior robustness to the unique control properties of the scale factor-if the scale factor value is reduced to close to zero, the channel output will be almost zero, its gradient will disappear and lose power, so it cannot continue to the opposite If you move in the direction of the value, the sign cannot be changed. According to experimental experience, no counter-examples of this explanation have been observed, as shown in Table 2:

Table 2

Regarding the robustness of pruning, as shown in Figures 4 and 5, the DNN performance and passport signature detection rate corresponding to the pruning weight ratio are shown. In this experiment, the performance of the embedded passport model against attacks is tested when a certain proportion of the DNN weight is pruned. This weight pruning strategy has been used in network compression. For the CIFAR10 classification, the passport signature detection accuracy is close to 100% when the trimming percentage is maintained at about 60%. Even if 90% of the DNN weights are pruned, the detection rate still reaches 70%. We attribute the robustness against modification attacks to the superior persistence presented by the embedded features in the example symbols.

The defense against obfuscation attacks, as shown in Figure 6, shows the performance of DNN, valid passports and two different types of fake passports, namely random attack fake18 and fuzzy attack fake29. For the AlexNet and ResNet trained on the CIFAR10 classification task, the network performance is very different, which depends on the authenticity of the passport-the DNN model that provides a valid passport shows almost the same accuracy as the original DNN model. At the same time, fake passports use the same DNN model (fake18=random attack in this case) to achieve a classification rate of about 10%, which is only equivalent to random guessing. In the case of fake29, it is assumed that the attacker has obtained the original training data set and attempts to reversely infer the scale factor and bias terms by freezing the trained DNN weights. As shown in Figure 6, AlexNet can only reach 84% at most, and ResNet can only reach 70% at most. In the CIFAR100 classification task, for fake18 cases, the attack success rate of AlexNet and ResNet is about 1%; for fake29, the attack success rate of AlexNet is 44%, and the attack success rate of ResNet is 35%. Based on these experimental studies, the threshold εf in Definition 1 can be set to 3% and 20% of AlexNet and ResNet, respectively. This fidelity evaluation process can effectively resist any potential ambiguity attacks. In short, a large number of experimental studies have shown that it is impossible for opponents to maintain the performance of the original DNN model by using fake passports, regardless of whether the fake passports are randomly generated or using the original training data set for reverse inference. This passport-related performance plays an indispensable role in the design of secure ownership verification schemes.

In addition, we also studied the two ownership verification methods of Passport and the trained DNN model to distribute this scheme V2, and private passport embedding the DNN model but not distribute this scheme V3.

First, as shown in Figure 7, during the distribution process of the passport and the trained DNN model, the learning process aims to minimize the combined loss function (Equation 1), where λ _t =0, because the trigger set is not used in this scheme Image and add the symbol loss (Equation 5) as a constraint term. Distribute the trained DNN model together with the passport to legitimate users, and the legitimate users use the given passport as the passport layer input for network prediction. Network ownership is automatically verified by distributed passport. This ownership verification is robust to the fine-tuning and pruning of DNN weights. In addition, obfuscation attacks cannot successfully forge a set of passports and signatures that can maintain network performance. The disadvantage of this scheme is that the passport needs to be used in the prediction stage, which leads to an additional calculation cost, about 10%. We show the experimental results in Table 5 of Appendix E. In addition, the distribution of passports to end users will interfere with the user experience and bear the additional responsibility of ensuring that digital passports are not leaked.

Next, as shown in Figure 8, we talk about embedding the private passport into the DNN model but not distributing it. This learning process aims to achieve two goals at the same time. The first goal is to minimize the loss of the original task (such as CIFAR10 classification). Including the passport layer; the second is to minimize the joint loss function that includes passport constraints (Equation 1). Algorithmically, this multi-task learning is achieved by alternately minimizing these two goals. The successfully trained DNN model is then distributed to end users, who can perform network predictions without the need for a passport. Note that this is achievable because the passport layer is not included in the distributed network. Ownership verification is only carried out at the request of law enforcement agencies, by adding the passport layer to the relevant network and verifying the embedded sign signature using undegraded network performance.

Compared with solution V2, this solution is easy to use for end users because it does not require a passport and does not incur additional calculation costs. At the same time, ownership verification is very effective for removing attacks and obfuscation attacks. However, its disadvantage is that it needs to access the DNN weights and attach a passport layer for ownership verification, which is the disadvantage of the white box protection mode.

The deep neural networks targeted by the embodiments of the present disclosure include all the various forms mentioned above, different input signals, different types, different network structures, different application functions, and deep neural networks on different computing carriers. Any neural network with the same principle, regardless of its operating environment. Optionally, neural networks can run on computer central processing units (CPU), graphics accelerators (GPU), tensor processors (TPU), dedicated artificial intelligence chips, and cloud computing centers, mobile devices, wearable devices, and smart videos Terminals, in-vehicle equipment and other vehicles, IoT devices (IoT devices) and other equipment.

Optionally, the DNN-based method for passports to resist obfuscation attacks can be applied to the above-mentioned terminal devices to generate passports resisting obscuration attacks. The terminal devices include: an embedded module, a fidelity evaluation module, a signature verification module, a reversible module, Passport generation module.

The embedding module inputs training data D, including trigger set data T or signature s, and optimizes the model N by minimizing a given loss function L; the fidelity evaluation module F evaluates whether the performance difference is less than a threshold, that is ( MM _t ) ≤ ε, where M is the performance of the DNN tested against a set of test data D, M _t is the target performance, ε is the threshold, and F is the fidelity evaluation result; the signature verification module V checks for a given nerve Whether the network N successfully verifies the predetermined signature s or the trigger set data T; the reversible module I(N)=N' satisfies the following conditions to exist, and causes a successful obfuscation attack A _a :

d) For a given DNN model, infer a new set of trigger set data T'and/or signature s'through reverse engineering;

e) The fake T', s'can be successfully verified with respect to the given DNN weight W, that is, V=True;

f) The fidelity evaluation result F is still True;

The DNN verification scheme V of the reversible module is defined as a reversible scheme V ^l , otherwise it is defined as an irreversible scheme

The passport generation module sends a set of N selected images into the training DNN model with the same structure, and collects N corresponding feature maps in each layer; among the N options, only one of each layer is randomly selected as passport. In particular, for a group of N elementary image having DNN model layer L, N ^L may be generated a total of possible combinations of the passport.

Claims (14)

1. A method for passports to resist ambiguity attacks based on a deep neural network DNN, including a DNN model and a passport. The ownership verification scheme of the DNN model includes an embedding process, a fidelity evaluation process, a signature verification process, and a reversible process. The method include:

   The embedding process is a DNN learning process, which takes training data as input, and the training data includes trigger set data or signatures, and optimizes the model by minimizing a given loss function;

   The fidelity evaluation process uses the following formula to evaluate whether the performance difference is less than the threshold, and obtain the fidelity evaluation result:

   (MM _t )≤ε,

   Among them, M is the DNN performance tested against a set of test data, M _t is the target performance, and ε is the threshold;

   The signature verification process is used to check whether the predetermined signature or trigger set data is successfully verified for a given neural network N;

   When the following conditions are met, there is a reversible process and a successful fuzzing attack: For a given DNN model, a new set of trigger set data and/or signatures are inferred through reverse engineering; a forged set of new trigger set data and / Or the signature is successfully verified relative to the given DNN weight W, the signature verification result is true; the fidelity evaluation result F is still True;

   A DNN verification scheme with a reversible process is defined as a reversible scheme, otherwise it is defined as an irreversible scheme.
2. The method according to claim 1, wherein the feature-based and trigger-set-based methods adopt a combined loss function as follows:

   L = L _c (f(W, X _r ), y _r )+λ ^t L _c (f(W, X _T ), y _T )+λ ^r R(W, s), (1)

   Among them, λ ^t , λ ^r are the weights of related hyperparameters, f(W, X_) is _{the prediction function with input X r} or X _T and output prediction value, L _c is the prediction value and target label y _r or y _T The loss function of cross entropy, signature s={P,B}, composed of passport P and signature string B, the constraint item is R=L _c (σ(W,P), B), or R=MSE(B- PW); MSE is the mean square error function.
3. The method according to claim 1, wherein the DNN model of the watermarking method based on the trigger set also embeds a private passport and a trigger set, but does not distribute, the trigger set is a set of trigger images, which are detected and declared by remotely calling the service API Suspicious ownership of the DNN model; first declare the ownership in the black box mode, and then declare the ownership again through passport verification in the white box mode. Trigger set images alternately minimize the original mission loss and reduce the joint loss function containing passport constraints, the original mission loss Excluding the passport layer, the GroupNormalisation algorithm is used.
4. The method according to claim 1, wherein the passport is a passport generated after random shuffling, and the method comprises: sending a set of N selected images into a training DNN model with the same structure, and in each The layer collects N corresponding feature maps; among the N options, only one of each layer is randomly selected as a passport, where, for a set of N basic images of the DNN model with L layers, a total of N ^L possible passports are generated combination.
5. The method according to claim 1, wherein the DNN model of the watermarking method based on the trigger set uses the following steps to add a trainable noise component to the randomly selected basic image:

   Randomly select a group of N basic images T _b ;

   Generate a random noise pattern with the same size T _{n as the trainable parameter;}

   Use the summation X _T =T _b +ηT _n as the trigger set image, where η=0.04 to make the noise component invisible;

   Randomly assign the trigger set label y _T ;

   Minimize the cross-entropy loss L _c associated with the trainable parameter T _n .
6. The method according to claim 1, wherein the DNN architecture is predetermined by the optimization model, and after the DNN weights are learned, the trigger set data or signature will be embedded in the model; the signature verification process first calls the DNN prediction process, which will trigger Set samples as input, and then check whether the prediction function generates the specified label under the condition of the false detection rate less than the threshold.
7. The method according to claim 1, wherein the DNN model further includes a passport layer, a convolutional layer, the passport layer is embedded with a digital signature, and the scale factor γ and the offset β of the passport layer depend on the convolution kernel _Wp and the specified Passport layer P, the formula is as follows:

   

   

   Among them, * represents the convolution operation, l is the number of layers, X _p is the input of the passport layer, X _c is the input of the convolution layer; O() is the corresponding linear transformation output, and P _γ ^l and P _β ^l are respectively Derive the passport layer of the scale factor γ and the bias term β; each convolution layer in the convolution layer is composed of several convolution units, and the parameters of each convolution unit are optimized through the backpropagation algorithm, The different features of the fuzzy attack are extracted by convolution operation.
8. The method according to claim 6, wherein _{the DNN model trained using the passport layer s e} = {P _γ ^l , P _β ^l } ^l , its prediction performance M depends on the digital passport provided when using the network, the formula as follows:

   

   If the real digital passport s _t ≠s _e cannot be provided, the network performance will deteriorate significantly.
9. The method according to claim 1, wherein the fidelity evaluation result in the irreversible scheme depends on the presented signature or trigger set data; if the forged passport s _t ≠s _e , the performance M deteriorates sharply, and the performance difference is greater than the threshold.
10. The method according to claim 1, wherein the signature is an embedded binary signature. During the learning process of the DNN weights, the following symbol loss constraints are added to the combined loss function to force the scale factor to adopt a specified positive sign Or minus sign:

    

    Among them, B={b ₁ ,···,b _C }∈{-1,1} ^C , composed of the specified binary bits of the C convolution kernel, γ ₀ is a positive control parameter, and the default is 0.1 to the value of the excitation scale factor Greater than γ ₀ .
11. The method according to claim 1, wherein the parameters of the DNN model are divided into the public convolutional layer parameter W and the scale factor γ and the bias term β of the hidden passport layer. The passport information is embedded in the weight W and the learning is compulsory. The following constraints are implemented: Avg(W _p ^l *P _γ ^l )=c _γ ^l , Avg(W _p ^l *P _y ^l )=c _β ^l ; the weight distribution of the convolutional layer is the same as that of the original DNN without the passport layer ; C _γ ^l and c _β ^l are constant values to which the parameters γ ^l and β ^l converge, and the scale factor can only take a positive or negative value away from zero.
12. The method according to claim 1, wherein the method is applied to a terminal device, the terminal device comprising: an embedded module, a fidelity evaluation module, a signature verification module, a reversible module, and a passport generation module;

    The embedding module inputs training data, including trigger set data or signatures, and optimizes the model by minimizing a given loss function;

    The fidelity evaluation module uses the following formula to evaluate whether the performance difference is less than the threshold, and obtain the fidelity evaluation result:

    (MM _t )≤ε,

    Among them, M is the DNN performance tested against a set of test data D, M _t is the target performance, and ε is the threshold;

    The signature verification module checks whether the predetermined signature or trigger set data is successfully verified for a given neural network;

    The reversible module satisfies the following conditions and causes a successful fuzzy attack: for a given DNN model, a new set of trigger set data and/or signatures are inferred through reverse engineering; a forged set of new trigger set data and / Or the signature is successfully verified relative to the given DNN weight, the signature verification result is true; the fidelity evaluation result is still true;

    The DNN verification scheme of the reversible module is defined as a reversible scheme, otherwise it is defined as an irreversible scheme;

    The passport generation module sends a set of N selected images into the training DNN model with the same structure, and collects N corresponding feature maps in each layer; among the N options, only one of each layer is randomly selected as passport, wherein, for a group of N elementary image having DNN model layer L, ^L N generated a total possible combinations passport.
13. The method according to claim 1, wherein: the deep neural network targeted by the DNN model includes different input signals, different types, different network structures, different application functions, and deep neural networks on different computing carriers, which are also included in the principle Any neural network on the same.
14. The method according to claim 1, wherein the method runs on computer central processing units, graphics accelerators, tensor processors, dedicated artificial intelligence chips, and cloud computing centers, mobile devices, wearable devices, smart video terminals, In-vehicle equipment, other vehicles, and Internet of Things equipment.

Images