CN112132733B

CN112132733B - Chaos-based automatic labeling algorithm for black box watermark trigger set of intelligent model

Info

Publication number: CN112132733B
Application number: CN202010999918.7A
Authority: CN
Inventors: 张盈谦; 贾贻然; 闻芊芊
Original assignee: Xiamen University Tan Kah Kee College
Current assignee: Xiamen University Tan Kah Kee College
Priority date: 2020-09-22
Filing date: 2020-09-22
Publication date: 2022-06-03
Anticipated expiration: 2040-09-22
Also published as: CN112132733A

Abstract

The invention discloses an intelligent model black box watermark triggering set automatic labeling algorithm based on chaos. The technology solves the problem of automatic labeling of the trigger set label by using chaos, and can effectively solve the problem of automatic labeling of the existing black box watermark trigger set, so that the watermark can be extracted by using a small amount of inquiry through a remote API (application program interface), and the identity of an owner of the deep learning intelligent model can be further determined. The method can effectively solve the problem that the white-box watermark scheme is difficult to extract the watermark without downloading a model of a server or reading a source program. And when judicial evidence is obtained, the method is simple and only needs to compare whether the output classification result is consistent with an expected result.

Description

Chaos-based automatic labeling algorithm for black box watermark trigger set of intelligent model

Technical Field

The invention relates to the technical field of image processing, in particular to an automatic labeling algorithm of a black box watermark triggering set of an intelligent model based on chaos.

Background

At present, in the development of the new generation of artificial intelligence in China, the science and technology department starts to build a national new generation artificial intelligence open innovation platform, and the typical established method is as follows: "automatic driving (Baidu), intelligent vision (Shangtui group), medical image (Tengchong), intelligent voice (Korea communication Fei Co.), etc. The creation of such high-performance deep learning intelligent models requires a lot of manpower and material resources, and they have been deeply integrated with human society. A deep learning intelligent model can be seen as an agent that works in place of a human. Therefore, they need to have identity recognition like human, and provide the owner and user of deep learning intelligent model with the basis for identity authentication. At present, many users do not know which artificial intelligence applications acquire their own information, and also do not know which are legal deep learning intelligent models and which are illegal deep learning intelligent models. Thus, the artificial intelligence industry is in need of a specialized technique to identify and prove the identity or owner identity of deep learning intelligent models.

Some solutions for proving the identity of the model have appeared, such as white-box watermarks and black-box watermarks. However, these techniques that are currently available have certain drawbacks. For example, white-box watermarks are vulnerable to statistical attacks; when the white box watermark is extracted, the whole model at the server end needs to be obtained, and the watermark is difficult to extract; the information quantity borne by the black box watermark is small; the black box watermark trigger set is characterized by being generally understandable by human beings, having great generalization and being incapable of resisting fraudulent claim of ownership attacks and coverage attacks; the black box watermark does not realize the separation of the trigger set and the secret key, and the problem of secret key leakage caused by the leakage of the trigger set cannot be solved. And the white-box watermark and black-box watermark schemes only consider single identity authentication and limited use scenes, but do not consider the problems of commercialization of the model and the ground of actual production. However, the rapid development of artificial intelligence inevitably prompts commercialization of the deep learning intelligent model, and the distribution of different keys to different users purchasing the deep learning intelligent model can effectively solve the problems of model landing and source tracing when the model is revealed.

In view of this, we propose a chaos-based deep learning intelligent model authentication scheme: and automatically labeling the black box watermark trigger set by using chaos, putting the automatically labeled trigger set into a model for training, and further verifying the ownership of the model in the watermark extraction stage. Due to the characteristics of sensitivity, non-periodicity, irregularity, unpredictability for a long time and the like of the chaotic initial value. The chaotic characteristic cannot be predicted by machine learning, and an attacker cannot find other trigger sets which accord with the watermark characteristic. Thus, the watermark is not generalizable. Such non-generalizability can resist fraudulent claim of ownership attacks and coverage attacks, among others. In addition, as the chaos is sensitive to the initial value, a large number of irrelevant chaos sequences can be generated to uniquely identify a plurality of intelligent products, and the method is favorable for commercialization of the model and source tracing when the model is leaked. In addition, the scheme can realize the separation of the trigger set and the key, even if an attacker already masters the watermark trigger set and the chaotic equation for generating the label, the key is unknown according to the kerchhoff criterion, namely the coefficient parameter of the chaotic equation and the initial value of the chaotic sequence are unknown, so that the security of the key is ensured.

Disclosure of Invention

The technical scheme adopted by the invention is as follows:

assuming there is a classification problem, the label of the data is l_iRepresents where 1 ≦ i ≦ m, the set of labels is

The automatic labeling scheme of the trigger set watermark can be carried out according to the following steps:

firstly, selecting N trigger set pictures N_k

Selecting a certain number of N trigger set pictures N_k1. ltoreq. k. ltoreq.n, their corresponding original labels are each L_kAnd k is more than or equal to 1 and less than or equal to n. Wherein

That is, the union of all the trigger set tags is a subset of m tag sets, and the value range of n in the first step is an integer.

II, setting picture N as trigger set_kAssigning a chaotic value X_Nk，

Iteration is carried out on the Logistic chaotic mapping for N 'times, and N results after the iteration for N' times are respectively distributed to N trigger sets selected in the step one, namely, each trigger set is N_kAll correspond to a Logistic mapping value X_NkWherein X is_Nk∈[0,1]，

Thirdly, dividing the Logistic mapping value y into m intervals

Dividing Logistic mapping value y into m intervals [ y_i-1,y_i) I is more than or equal to 1 and less than or equal to m, and each interval corresponds to a specific class l_iSo as to trigger a certain trigger picture N_kDivision into a particular class l_iWherein k is more than or equal to 1 and less than or equal to n, i is more than or equal to 1 and less than or equal to m,

fourthly, triggering set N by utilizing chaos label_k，1≤k≤n

For each trigger set picture N in step two_kCorresponding chaos value X_NkLooking up the section it belongs to in step three and classifying it into a specific category, assuming X_Nk∈[y_i-1,y_i)，N_kThe corresponding label is L'_k＝l_iWherein k is more than or equal to 1 and less than or equal to n, i is more than or equal to 1 and less than or equal to m,

fifthly, marking the trigger set N again_k，1≤k≤n，

Comparing automatic chaos labeled post-trigger set N_kLabel L'_kWhether or not to match its original label L_kSame, if L'_k＝L_k＝l_iWhen k is not less than 1 and not more than n-1, then use L'_k＝l_i+1To mark data N_k(ii) a When k is n, use L'_k＝l₁To mark data N_n。

k takes other values meaningless.

Compared with the prior art, the invention has the following beneficial effects:

1. provides an effective black box watermark scheme for proving the identity of an intelligent model for deep learning

The technology solves the problem of automatic labeling of the triggering set label by utilizing chaos, and can effectively solve the problem of automatic labeling of the existing black box watermark triggering set, so that the watermark can be extracted by utilizing a small amount of inquiry through a remote Application Program Interface (API), and further the identity of an owner of the deep learning intelligent model can be determined. The method can effectively solve the problem that the white-box watermark scheme is difficult to extract the watermark without downloading a model of a server or reading a source program. And when judicial evidence is obtained, the method is simple and only needs to compare whether the output classification result is consistent with an expected result.

2. Ensuring the non-generalization of the watermark

Since the trigger set pictures designed at present often overlap the pictures with human comprehensible disturbance, the disclosure of one trigger set may cause disclosure of the whole trigger set key, so that the trigger set has great generalization and is likely to be subjected to fraudulent ownership attack and coverage attack. The generalization of the watermark can be effectively avoided by automatically labeling the trigger set by using the logistic mapping in the chaos. The chaos has the characteristics of sensitivity, non-periodicity, irregularity, long-term unpredictability and the like, and the behavior of the chaos cannot be predicted by machine learning based on statistics, so that an attacker cannot find other trigger sets conforming to the watermark characteristics of the attacker.

3. Save time and manpower, be favorable to the commercialization of deep learning intelligent model and the problem of tracing to the source when the model is revealed

The proposal can effectively save manpower and material resources and is beneficial to the commercialization process of the deep learning intelligent model and the source tracing when the model is leaked. Chaos is very sensitive to the initial value and can generate a large number of uncorrelated and pseudorandom chaos sequences. Given that with our automatic labeling scheme, each user is provided with a unique watermark, this can efficiently solve the commercialization problem of deep-learning intelligent model watermarks. In addition, if the deep learning intelligent model is stolen or a model leakage problem occurs, the unique performance of the watermark can help us to quickly find the source of the model leakage, so that the legitimate rights and interests of a model owner are maintained.

4. Realizes the separation of the secret key and the watermark and ensures the security of the watermark

The scheme realizes the separation of the key and the watermark, and even if an attacker already masters our chaotic equation for triggering the data set and generating the label, the key is unknown according to the kerchhoff criterion, namely the coefficient parameters of the chaotic equation and the initial value of the chaotic sequence are unknown, so that the security of the key is ensured.

Drawings

Fig. 1 is a flow chart of trigger set tagging and watermark embedding.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Taking the watermark of the intelligent driving-direction control model as an example:

taking the intelligent driving-direction intelligent control model as an example, the classification result includes four categories of forward, left, right and stop. In the construction of the model, firstly, a running track of the intelligent trolley is built, and then images and labels in all directions are collected to be used as training data of the model. The trigger set labeling and watermark embedding flow chart is shown in fig. 1. To embed the watermark in the model, it is first necessary to pick the trigger picture, and we pick 32 images from the training data and add salt-pepper noise to them, and these 32 images are the trigger pictures. Next, we need to perform chaotic automatic labeling on the selected 32 triggered pictures, that is, putting them into the chaotic automatic labeling device shown in fig. 1 for automatic labeling. The chaos labeling scheme of the chaos automatic labeling device is as follows:

the automatic labeling scheme may be performed as follows:

1) selecting N trigger set pictures N_k，1≤k≤n

2) Iteration is carried out on the Logistic chaotic mapping for N 'times, and N results after the iteration for N' times are respectively distributed to N trigger sets selected in the step 1), namely, each trigger set N_kAll correspond to a Logistic mapping value X_Nk。

3) Dividing the Logistic mapping value y into m intervals, wherein each interval corresponds to a specific category l_iWherein i is more than or equal to 1 and less than or equal to m.

4) For step 2) per trigger set N_kCorresponding chaos value X_NkLooking up the interval to which it belongs in step 3) and classifying it into a specific category l_iI.e. L'_k＝l_i。

5) Compare auto-annotated post-trigger set N_kLabel L'_kWhether or not to match its original label L_kSame, if L'_k＝L_k＝l_iWhen k is not less than 1 and not more than n-1, then use L'_k＝l_i+1To mark data N_k(ii) a When k is n, use L'_k＝l₁To mark data N_n。

The 32 images labeled by the chaotic automatic labeling device can be regarded as a trigger set, and the training set is all the training data collected before except for the 32 trigger images. The data set is a collection of training and trigger sets. We put the data set into the model for training, and finally 4 classification results are generated: forward, left, right, stop. Finally, in the watermark extraction stage, the complete intelligent model, including the code and the model file, does not need to be acquired. The watermark need only be extracted by placing the trigger set into the model through the API. By comparing the output result with the chaotic automatic labeling label, the ownership of the model can be simply verified. At present, the accuracy of the model is 0.9951, and the watermark extraction success rate is 1.0.

Experiments and simulation results show that the automatic labeling algorithm can resist attacks such as fine tuning, compression, coverage and the like.

(1) Robustness to fine-tuning attacks

Micro-toning and transfer learning are the most common attacks because training a model from scratch is a very labor intensive task. We expand fine tuning attacks on the original model, and retrain the original training data by using the traditional cross entropy loss function. After the fine tuning attack, the accuracy of the model is 99.61%. Compared with the accuracy of 99.51% of the original model, the accuracy is slightly improved by 0.10%. After fine adjustment, the watermark extraction rate is still 1.0. Therefore, experimental results show that the scheme does not influence the accuracy of the model and the extraction rate of the watermark.

(2) Robustness to compression attacks

Compression attacks are also a common attack method. We use the TensorFlow model optimization tool to prune our model so that we can compress it. Pruning means eliminating unnecessary values in the weight tensor. The pruning step in the model optimization tool is amplitude-based weight pruning. It gradually zeroes the model weights from the weights closest to zero to achieve model sparsity. The value of sparsity represents the number of weights to prune. As shown in table 1, when the pruning rate is less than 0.60, the accuracy of the model and the watermark extraction rate are high, which indicates that our model can resist the pruning attack in this case. When the pruning rate is greater than 0.70, although the pruning attack makes watermark extraction difficult, the accuracy of the attacked model is already subject to a significant loss and the model therefore becomes unusable. Therefore, our scheme is able to resist compression attacks.

TABLE 1 robustness to pruning-based compression attacks

(3) Robustness to overlay attacks

The overlay attack assumes that the attacker knows our watermarking mechanism. The adversary then attacks our model according to our watermark generation algorithm to achieve the goal of covering/eliminating our previous watermark. In this process, the attacker cannot know our trigger data set. However, we assume that the worst case is: the attacker gets our trigger set and chaotic annotator. According to the Kerchhoff principle, the adversary does not know the key of the parameters and initial values of our chaotic annotators.

By changing the parameters and initial values of the Logistic mapping function, we obtain four new trigger sets. After putting these trigger sets into the training data set for individual training, the following results are obtained in table 2. The four groups of simulation results all reduce the precision of the model. On average performance, although the extraction rate of the new watermark after the overlay attack is 0.8047, and the extraction rate of the original watermark is reduced to 0.429, the accuracy of the model is reduced to 0.6926, and the model function can not be realized. The accuracy of the model obtained by the overlay attack is greatly affected. Therefore, our chaotic scheme is robust to coverage attacks.

TABLE 2 robustness to overlay attacks

Although embodiments of the present invention have been shown and described in the present disclosure, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims

1. An intelligent model black box watermark triggering set automatic labeling algorithm based on chaos is characterized in that: the method comprises the following steps:

data label l_iRepresents where 1 ≦ i ≦ m, the set of labels is

firstly, selecting N trigger set pictures N_k

Selecting a certain number of N trigger set pictures N_k1. ltoreq. k. ltoreq.n, their corresponding original labels are each L_kK is more than or equal to 1 and less than or equal to n, and the value range of n is an integer, wherein

I.e., the union of all trigger set tags is a subset of the m sets of tags,

II, setting picture N as trigger set_kThe value of the chaos is assigned and,

iteration is carried out on the Logistic chaotic mapping for N 'times, and N results after the iteration for N' times are respectively distributed to N trigger sets selected in the step one, namely, N is carried out on each trigger set_kAll correspond to a Logistic mapping value X_NkWherein X is_Nk∈[0,1]，

Thirdly, dividing the Logistic mapping value y into m intervals

fourthly, triggering set N by utilizing chaos label_k，1≤k≤n

fifthly, marking the trigger set N again_k，1≤k≤n，

Comparing automatic chaos labeled post-trigger set N_kL 'of'_kWhether or not to match its original label L_kSame, if L'_k＝L_k＝l_iWhen k is not less than 1 and not more than n-1, then use L'_k＝l_i+1To mark data N_k(ii) a When k is n, L'_k＝l₁To mark data N_n(ii) a k takes other values meaningless.