CN110941855A - Stealing and defending method for neural network model under AIoT scene - Google Patents
Stealing and defending method for neural network model under AIoT scene Download PDFInfo
- Publication number
- CN110941855A CN110941855A CN201911173524.XA CN201911173524A CN110941855A CN 110941855 A CN110941855 A CN 110941855A CN 201911173524 A CN201911173524 A CN 201911173524A CN 110941855 A CN110941855 A CN 110941855A
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- matrix
- network model
- encryption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Biophysics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a neural network model stealing defense method in an AIoT scene, which is applied to the field of network security and aims at the problem that data leakage is likely to occur when edge end equipment transmits data to a cloud server in the AIoT scene so as to cause model stealing; the method mainly comprises a training stage and a deployment stage, wherein the training stage is carried out on the server, the deployment stage is carried out after the training stage is completed, and data in the training stage is deployed to the edge terminal equipment and the cloud server.
Description
Technical Field
The invention belongs to the field of network defense, and particularly relates to a stealing defense technology of a neural network model.
Background
Model stealing is a serious threat to Artificial Intelligence (AI) applications based on Neural Networks (NN). As shown in FIG. 1, the principle of model stealing is to send a large number of inferred requests to open machine learning APIs such as Amazon AWS, Microsoft Azure, Google Cloud, BigML, etc. And the output and input of the API can constitute a training data set that steals the model, then trains and finds the best results using various common machine learning models. The purpose of stealing models is to bypass the original model and use the surrogate model to obtain future predictions for the commercial interest. And/or steal model knowledge and generate transferable antagonistic samples and can be used to perturb the original model to make erroneous predictions. The model stealing technology is used for stealing models or recovering training data members through black box detection, such as stealing stock market prediction models and spam filtering models, and an attacker can effectively optimize the attack models in a targeted manner by using the technology.
Model stealing attacks refer to information that attempts to recover the data used in the model or training. Such an attack is a significant concern because the model is a very valuable intellectual property asset that is trained with the most valuable data in the company, such as financial transactions, medical information, user transaction information, and so forth. It is important to ensure the security of models trained using user privacy data, as these models may be abused to cause leakage of user sensitive information.
Model stealing attacks have two main forms: model reconstruction and member leakage.
And (4) creating a model. The key to model reconstruction is that an attacker can reconstruct a model by exploring the public API and restricting his own model. The adaptation of such attacks is demonstrated in the paper steeling Machine Learning Models via Prediction APIs to be effective for most AI algorithms including SVM, random forest, deep neural networks.
The member leaks. A hacker may decide which records to train the model with by building a shadow model. Such attacks, while not requiring a recovery model, may leak sensitive information.
In the aspect of model stealing detection, m.juuti et al propose a model stealing detection method based on the difference between the query distribution of an attacker and the query distribution of a normal client; in the aspect of model stealing detection and defense, n.papernot et al proposes a student-teacher network architecture based on differential privacy, and through the network architecture, direct contact between a training set and a final deployment model is isolated, so that an attacker cannot directly obtain privacy information in an original training set through a public API.
However, in the AIoT scenario, data leakage may occur when the edge device transmits data to the cloud server, which may result in model theft.
Disclosure of Invention
In order to solve the technical problem, the invention provides a method for defending stealing of a neural network model in an AIoT scene, which is used for defending model stealing based on a mode of encrypting an acquired image and an output prediction result.
The technical scheme adopted by the invention is as follows: a stealing defense method of a neural network model under an AIoT scene is based on a defense system and comprises the following steps: the system comprises a cloud server and edge-end equipment, wherein training of an incomplete self-encoder is completed on the cloud server, an encoder part of the self-encoder which is completed through training is deployed to the edge-end equipment, and a decoding part of the self-encoder which is completed through training is deployed to the cloud server; the edge end device includes: the device comprises a neural network model inference module used for executing AI tasks of edge end equipment, an image encryption area positioning module based on activation mapping of a neural network and used for positioning a minimum encryption area, and an encryption module used for encrypting images and outputting inference data by the neural network model; the remote server at least comprises a decryption module used for decrypting the encrypted data transmitted back by the edge terminal equipment.
Further, the cloud server completes training of the under-complete self-encoder, and specifically comprises the following steps:
A. determining an encryption and decryption key and generating an encryption and decryption rule; the training set is image data and corresponding label data that are collected in advance according to a specific recognition scene.
B. Training according to a training set to obtain a neural network model, calculating an image encryption minimum area coefficient according to the neural network model, and storing the calculated image encryption minimum area coefficient to edge-end equipment;
C. an under-complete autocoder is trained.
Further, the step A comprises the following substeps:
and A1, acquiring all training data in the training set, and calculating the image mean value of each recognition class in the training set. The identification class includes: cars, buildings, etc. identify objects.
A2, calculating an average confidence coefficient vector of each recognition class;
a3, sequentially clustering all recognition class average confidence coefficient vectors;
and A4, determining an encryption and decryption key according to the clustering result, generating an encryption and decryption rule, and obtaining an average image, which is obtained by calculating the identification class in the same group in A1, as a symmetric encryption and decryption key.
The encryption and decryption rules are as follows: and sequencing the identification classes in the same group, forming a closed loop by the image mean values corresponding to the identification classes, and taking the image mean value of the previous identification class as an image encryption key of the next identification class.
Further, the step B comprises the following substeps:
b1, training a neural network model (such as VGG16, ResNet50 and the like) obtained according to pre-collected training set data, acquiring an encryption key matrix according to the inference of the neural network model, and generating a mask matrix which consists of keys and has the same dimension as the input keys;
the matrix is used for encrypting the image, the initial value of the matrix is an encryption key matrix, and then the effect of encrypting the image is controlled by continuously modifying the value in the mask.
B2, copying a matrix f' identical to the activation mapping matrix f, and sorting all element values in the matrix in descending order;
b3, selecting the elements of the top theta% after f' sorting, and finding out the corresponding positions of the elements corresponding to f and the mask matrix; keeping element values of corresponding positions in the mask matrix, and setting the rest positions as 0; obtaining an updated key matrix; the initial value of theta is 100;
b4, encrypting the Image matrix Image by adopting the key matrix updated in the step B3;
Encrypted Image＝Image+α*mask
wherein Encrypted Image is the Encrypted Image matrix, Image is the original Image matrix, α is the hyper-parameter, which is used to control the encryption strength of the key, for example, α equals 1.0, and mask is the key matrix generated in step B3.
And B5, deducing the Encrypted Image matrix Encrypted Image obtained in the step B4 by the neural network model again to obtain a confidence vector output by the neural network model, and calculating JS divergence between the output q (x) and the original output p (x) of the neural network model after the graph is Encrypted.
B6, updating θ in step B3 according to the value of JS divergence, where θ is updated to be θ', and the updating expression is as follows:
θ‘＝θ-λ·JS(p(x),q(x))
b7, iteratively executing steps B3 to B6 until the neural network model can correctly identify the encrypted image, that is, the accuracy of the encrypted image data is lower than a set threshold, the threshold is set by a system designer, the threshold is generally smaller, for example, the accuracy threshold is 0.1, that is, the accuracy of the identified encrypted image is lower than 10%, which indicates correct identification;
b8, returning to the last iteration to obtain a theta value, and storing the theta value in the edge end equipment.
Further, step C includes the following substeps:
c1, inputting all data in the training set into the neural network model by using the same neural network model in the step B1 to obtain an inference vector of the neural network model;
c2, using the neural network model inference vector obtained by the C1 as the input of a single-layer under-complete self-encoder, and updating weight parameters in the self-encoder through the mean square error of the output and the input of the self-encoder;
and C3, deploying the encoder part of the self-encoder trained by the step C2 to the edge-end device, and deploying the decoder part to the cloud server.
Further, mapping and locating the minimum encryption area based on activation of the neural network comprises the following steps:
s1, generating a neural network activation mapping matrix;
s2, positioning the minimum encryption area of the image according to the neural network activation mapping;
s3, training an encoder encryption model of the incomplete auto-encoder to output inferred data;
and S4, decrypting the data by the cloud server.
Further, step S1 includes the following substeps:
s11, acquiring an inference result inference of the network to the image, and acquiring a weight vector W corresponding to the inference result inference on the last convolutional layer according to the inference result;
s12, upsampling the feature output F of the last convolution layer to the original size of the image, and carrying out weighted summation operation with the weight W to obtain the corresponding feature activation mapping.
Further, step S2 includes the following substeps:
s21, obtaining an encryption key matrix according to the inference of the model, and generating a mask matrix which consists of keys and has the same dimension as the input key;
s22, copying a matrix f' identical to the activation mapping matrix f, and sorting all element values in the matrix in descending order;
s23, selecting the elements of the top theta% after f' sorting, and finding out the corresponding positions of the elements corresponding to f and the mask matrix. Keeping element values of positions corresponding to the mask matrix, and setting the rest positions to be 0;
s24, encrypting the Image matrix Image using the following expression;
Encrypted Image＝Image+α*mask
the Encrypted Image is the Encrypted Image matrix, the Image is the original Image matrix, α is a hyper-parameter, and is used to control the encryption strength of the key, for example, α ═ 1.0. mask is the key matrix generated in step B3.
Further, step S4 includes the following substeps:
s41, decrypting the model inference data returned by the edge terminal equipment by using the decoder deployed in the step C4;
s42, obtaining the identification class corresponding to each image according to the model inference data decrypted in S41, and calculating the decryption key corresponding to each image through the decryption key and the encryption rule deployed on the server;
s43, according to the encrypted position information returned by the edge device and the key in S42.
The invention has the beneficial effects that: the method can effectively prevent the stealing of the neural network model in the AIoT scene, avoid the leakage of the user privacy data and ensure the safety of the user privacy data, and has the following advantages:
1. the image encryption key is a symmetric key, but is not easy to crack, because the determination of the key needs to obtain the inference of an original input image, the encryption and decryption rule and the polynomial encryption and decryption hyper-parameters at the same time, and the leakage of the three must be completed in the cloud server, an attacker can not crack the key only by monitoring the data communication between the edge end and the cloud, and if the security performance of the cloud server is higher, the effect of the AIoT defense model stealing method can be ensured;
2. the overhead of encryption and decryption is small. Because a large amount of training is completed before deployment, the edge end equipment can complete the encryption of the image and the output of the model only by utilizing the model and the inference result after the inference is completed and matching with the trained parameters, the cost for decrypting the image is smaller than that for encrypting the image, and the decryption process can be completed only by executing key searching and polynomial calculation because the decryption process does not depend on the model;
3. the method is suitable for model stealing defense under the low-power-consumption AIoT scene.
Drawings
Fig. 1 is a network structure diagram of an auto encoder according to an embodiment of the present invention.
FIG. 2 is a flow chart of a method for generating encryption and decryption keys and encryption and decryption rules according to the present invention.
Fig. 3 is a flowchart of an image encryption minimum area coefficient generation method of the present invention.
FIG. 4 is a deployment phase module composition diagram of the present invention.
FIG. 5 is a flowchart of a method for locating an image encryption area according to the present invention.
Fig. 6 is a flowchart of a method for decrypting data by the cloud server according to the present invention.
Detailed Description
Different from the existing model stealing detection and defense method scene, the invention provides a brand-new model stealing defense method considering that in the AIoT scene, data leakage may occur when the edge device transmits data to the cloud server to cause model stealing. The invention provides a model stealing defense method based on a neural network based on the characteristics of the neural network; the defense for model stealing is realized by encrypting and deducing data images and labels, the encrypted images are based on a neural network type activation mapping technology, and the encrypted labels are based on an under-complete self-encoder technology.
To facilitate an understanding of the present invention by those skilled in the art, the following prior art will now be described:
1. neural network class activation mapping technology
Neural network class activation mapping techniques were first proposed by Bolei Zhou et al, where the class activation mapping is a weighted linear sum of different visual patterns at different spatial locations. By simply upsampling the class activation map to the size of the input image, the image regions most relevant to a particular class can be identified. The class activation mapping technology can help to understand which areas of the target play a key role in the final recognition result, and relevant areas of the image are encrypted based on the phenomenon and the principle, so that an attacker is prevented from stealing the model through the image.
2. Self-encoder
An autoencoder is a type of neural network that is trained to attempt to copy an input to an output. As shown in fig. 1, there is a hidden layer h inside the self-encoder that can generate the encoded representation input. The network can be seen as being composed of two parts: an encoder represented by the function h ═ f (x) and a decoder r ═ g (h) that generates the reconstruction. The less-than-perfect autoencoder limits the dimension of h to be smaller than x. Learning an under-complete representation will force the self-encoder to capture the most salient features in the training data.
The present invention is further illustrated with reference to fig. 2-6.
The method comprises a training phase and a deployment phase, wherein the training phase is performed on a server, the deployment phase is performed after the training phase is completed, and data in the training phase is deployed to edge-end equipment and a cloud server. The server in the training phase may be a local server or a cloud server.
The training stage mainly comprises the generation of encryption and decryption keys and encryption and decryption rules, the generation of a minimum region coefficient theta and the training of an under-complete self-encoder, and the specific implementation process is as follows:
step a, generation of encryption and decryption keys and encryption and decryption rules, as shown in fig. 2, step a includes the following sub-steps:
step A1: all training data are obtained from the training set, and an average image of each recognition class in the training set is calculated, wherein the image in the training set is a batch of data, the data size of the batch of data can be set by self, for example, 32, 128 and the like, and the size of the batch of data can be set according to actual conditions similarly to the batch size of the training network. In this step, the training set is image data and corresponding label data collected in advance according to a specific recognition scene, and the recognition targets such as automobiles, buildings and the like are recognized.
Step A2: calculating an average confidence vector of each recognition class;
step A3: and sequentially clustering all the recognition class average confidence coefficient vectors. And if the group does not exist, establishing a new group, putting the identification class corresponding to the first piece of data into the group, entering a new piece of data each time, sequentially traversing the average confidence coefficient vector corresponding to each piece of data in the existing group, and calculating the cosine similarity of the two data. If the cosine similarity is greater than the threshold (for example, the threshold in the CIFAR-10 dataset is 0.01), adding the identification class corresponding to the data into the group, and if all groups cannot be added after traversing, newly building a group and adding the identification class corresponding to the data into the newly built group.
One skilled in the art will appreciate that the CIFAR-10 dataset is a labeled dataset having 60000 color images, 32X 32 images, divided into 10 classes of 6000 images each. The inner part is 50000 for training, and 5 training batches are formed, wherein 10000 graphs in each batch are formed; another 10000 was used for testing, constituting a batch individually. From the test lot data, 1000 sheets were randomly taken from each of 10 categories. The remainder is randomly arranged to form a training batch. Note that the number of images in each class is not necessarily the same in a training batch, and there are 5000 images in each class for the training batch as a whole.
The threshold value in step a3 is set in relation to the actual situation of the data distribution of the cosine similarity calculated,
step A4: and determining an encryption and decryption key and a rule according to the clustering result. The average image calculated in a1 for the identification classes in the same group is the symmetric encryption/decryption key. Then generating an encryption and decryption rule according to a specific rule: assume that there are three recognition classes a, B, C in a group. Then, the image mean in the B identification class is the encryption key of all the images identified as a, the image mean in the C identification class is the encryption key of all the images identified as B, and the image mean in the a identification class is the encryption key of all the images identified as C. In this way, the decryption key and the decryption rules are also determined at the same time. After the encryption key and the encryption rule are generated, the cloud server and the edge device both keep the symmetric key and the encryption and decryption rule, and the symmetric key and the encryption and decryption rule are respectively used for encrypting and decrypting the image.
Step B image encryption minimum area coefficient generation, as shown in fig. 3, step B includes the following sub-steps:
and step B1, training a neural network model according to the pre-collected training set data, acquiring the inference, acquiring an encryption key matrix according to the inference, and generating a mask matrix which consists of keys and has the same dimension as the input key. The matrix is used for encrypting the image, the initial value of the matrix is an encryption key matrix, and then the effect of encrypting the image is controlled by continuously modifying the value in the mask.
And step B2, copying a matrix f' identical to the activation mapping matrix f, and sorting all element values in the matrix in descending order, wherein the step is used for selecting the elements meeting the conditions in the matrix in the subsequent step.
And step B3, selecting the elements of the top theta% after f' sorting (the initial value of theta is 100), and finding out the corresponding positions of the elements corresponding to f and the mask matrix. And reserving element values of positions corresponding to the mask matrix, and setting the rest positions to be 0.
At step B4, the Image matrix Image is encrypted using the following polynomial.
Encrypted Image＝Image+α*mask
The Encrypted Image is an Encrypted Image matrix, the Image is an original Image matrix, α is a hyper-parameter, and is used to control the encryption strength of the key, for example, α ═ 1.0. mask is the key matrix generated in step B3.
And step B5, deducing the Encrypted Image Encrypted in the step B4 by the neural network model again to obtain a confidence vector output by the model, and calculating JS divergence between the output q (x) and the original output p (x) of the neural network model after the Image is Encrypted.
Step B6, updating θ in step B3 according to the value of JS divergence.
θ‘＝θ-λ·JS(p(x),q(x))
Wherein theta' is the updated theta; lambda is the learning rate, and the intensity of the change of the theta value along with the JS divergence is controlled; JS (p (x), q (x)) is JS divergence of a model output result corresponding to the encrypted image and the original image, and is used for controlling the direction and the size of the updated theta.
And step B7, iteratively executing the steps B1 to B6 until the encrypted image can be correctly identified by the neural network model.
And B8, returning to the last iteration to obtain a value theta, and acquiring the size and the position of the minimum area of any image needing to be encrypted through the value theta, such as the B3 step. The value of theta is stored in the edge device.
And C: training an incomplete auto-encoder for encrypting the inferred data of the model; the step C comprises the following sub-steps:
and step C1, inputting all data in the training data set into the neural network model by using the neural network model trained in the step B to obtain an inference vector of the neural network model.
And step C2, taking the model inference vector obtained by the step C1 as the input of the single-layer under-complete self-encoder, and updating weight parameters in the self-encoder through the mean square error of the output and the input of the self-encoder.
And step C3, deploying the encoder part of the trained self-encoder to the edge terminal equipment, and deploying the decoder part to the cloud server.
In the deployment stage, as shown in fig. 4, the edge of the whole model defense system has several modules, one is a model inference module for executing normal AI tasks of the edge device, the second is an image encryption region positioning module for mapping and positioning the minimum encryption region based on activation of the neural network, and the third is an encryption module for encrypting the image and encrypting the model output inference data. The server only has one decryption module, and the encrypted data transmitted back by the edge terminal equipment is decrypted.
As shown in fig. 5, the image encryption area positioning method includes the following steps:
s1: generating a neural network activation mapping matrix, the detailed steps of step S1 including:
step S11 obtains the inference result inference of the network for the image, and obtains the weight vector W corresponding to the inference result inference on the last convolutional layer according to the inference result.
And step S12, upsampling the feature output F of the last convolutional layer to the original size of the image, and performing weighted summation operation with the weight W to obtain corresponding feature activation mapping.
S2: locating the minimum encrypted region of the image based on the neural network activation map, the detailed steps of step S2 include:
and step S21, obtaining an encryption key matrix according to the inference of the model, and generating a mask matrix which consists of keys and has the same dimension as the input. The matrix is used for encrypting the image, the initial value of the matrix is an encryption key matrix, and then the effect of encrypting the image is controlled by continuously modifying the value in the mask.
In step S22, a matrix f' identical to the activation mapping matrix f is copied, and all the element values in the matrix are sorted in descending order, which is used in the following steps to select the elements in the matrix that meet the conditions.
Step S23, select the top θ% elements (θ is already stored in the edge device) after f' sorting, and find their corresponding positions in f and mask matrix. And reserving element values of positions corresponding to the mask matrix, and setting the rest positions to be 0.
In step S24, the Image matrix Image is encrypted using a polynomial.
Encrypted Image＝Image+α*mask
The Encrypted Image is the Encrypted Image matrix, the Image is the original Image matrix, α is a hyper-parameter, and is used to control the encryption strength of the key, for example, α ═ 1.0. mask is the key matrix generated in step B3.
S3: the encoder encryption model using the less than complete autoencoder outputs the inferred data,
s4: and the cloud server decrypts the data. The cloud server receives the encrypted data transmitted back by the edge terminal equipment, and gradually decrypts the data by using a previously agreed decryption method; as shown in fig. 6, the detailed step of step S4 includes:
step S41: the model inference data returned by the edge end device is decrypted using the decoder deployed in step C4.
Step S42: and obtaining the identification class corresponding to each image according to the model inference data decrypted in the step S41, and calculating the decryption key corresponding to each image according to the decryption key and the encryption rule which are deployed on the server.
Step S43: according to the encrypted location information returned by the edge device and the key in S42. Similar to step S23, the reserved key matrix corresponds to the encrypted position element values, and the remaining positions are all set to 0, the decryption key is regenerated, and decryption is performed using the complement of the same encryption polynomial in step S24 and in the edge end device.
Decrypted Image＝Encrypted Image-α*mask
Decrypted Image is the Decrypted Image, Encrypted Image is the Encrypted Image of the edge device, α is the same as the settings in the edge device, and mask is the decryption key recalculated in step S43.
It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (10)
1. A stealing defense method of a neural network model under an AIoT scene is characterized in that a defense system based on the method comprises the following steps: the system comprises a cloud server and edge-end equipment, wherein training of an incomplete self-encoder is completed on the cloud server, an encoder part of the self-encoder which is completed through training is deployed to the edge-end equipment, and a decoding part of the self-encoder which is completed through training is deployed to the cloud server; the edge end device includes: the device comprises a neural network model inference module used for executing AI tasks of edge end equipment, an image encryption area positioning module based on activation mapping of a neural network and used for positioning a minimum encryption area, and an encryption module used for encrypting images and outputting inference data by the neural network model; the remote server at least comprises a decryption module used for decrypting the encrypted data transmitted back by the edge terminal equipment.
2. The method for stealing and defending a neural network model under the AIoT scene as claimed in claim 1, wherein the cloud server completes training of an under-complete self-encoder, specifically comprising the steps of:
A. determining an encryption and decryption key and generating an encryption and decryption rule; the training set is image data and corresponding label data which are collected in advance according to a specific recognition scene;
B. calculating an image encryption minimum area coefficient according to a neural network model trained by the training set and the neural network model, and storing the calculated image encryption minimum area coefficient to the edge terminal equipment;
C. an under-complete autocoder is trained.
3. The method for stealing and defending against the neural network model under the AIoT scene as claimed in claim 2, wherein the step A comprises the following sub-steps:
a1, acquiring all training data in a training set, and calculating the image mean value of each recognition class in the training set;
a2, calculating an average confidence coefficient vector of each recognition class;
a3, sequentially clustering all recognition class average confidence coefficient vectors;
and A4, determining an encryption and decryption key according to the clustering result, generating an encryption and decryption rule, and obtaining an average image, which is obtained by calculating the identification class in the same group in A1, as a symmetric encryption and decryption key.
4. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 2, wherein step B comprises the following sub-steps:
b1, training a neural network model according to pre-acquired training set data, acquiring an encryption key matrix according to the inference of the neural network model, and generating a mask matrix which consists of keys and has the same dimension as the input key matrix;
b2, copying a matrix f' identical to the activation mapping matrix f, and sorting all element values in the matrix in descending order;
b3, selecting the elements of the top theta% after f' sorting, and finding out the corresponding positions of the elements corresponding to f and the mask matrix; keeping element values of corresponding positions in the mask matrix, and setting the rest positions as 0; obtaining an updated key matrix;
b4, encrypting the Image matrix Image by adopting the key matrix updated in the step B3;
Encrypted Image＝Image+α*mask
wherein the Encrypted Image is an Encrypted Image matrix, the Image is an original Image matrix, and α is a hyper-parameter;
b5, deducing the Encrypted Image matrix Encrypted Image obtained in the step B4 by the model again to obtain a confidence vector output by the neural network model, and calculating JS divergence between the output q (x) of the neural network model and the original output p (x) after the Image is Encrypted;
b6, updating θ in step B3 according to the value of JS divergence;
b7, iteratively executing the steps B3 to B6 until the encrypted image can be correctly identified by the neural network model;
b8, returning to the last iteration to obtain a theta value, and storing the theta value in the edge end equipment.
5. The method as claimed in claim 4, wherein the initial value θ is 100.
6. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 2, wherein step C comprises the following sub-steps:
c1, inputting all data in the training set into the neural network model by using the neural network model trained in the step B1 to obtain an inference vector of the neural network model;
c2, using the neural network model inference vector obtained by the C1 as the input of a single-layer under-complete self-encoder, and updating weight parameters in the self-encoder through the mean square error of the output and the input of the self-encoder;
and C3, deploying the encoder part of the self-encoder trained by the step C2 to the edge-end device, and deploying the decoder part to the cloud server.
7. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 1, wherein mapping and locating the minimum encryption area based on activation of neural network comprises the following steps:
s1, generating a neural network activation mapping matrix;
s2, positioning the minimum encryption area of the image according to the neural network activation mapping;
s3, training an encoder encryption model of the incomplete auto-encoder to output inferred data;
and S4, decrypting the data by the cloud server.
8. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 7, wherein step S1 comprises the following sub-steps:
s11, acquiring an inference result inference of the network to the image, and acquiring a weight vector W corresponding to the inference result inference on the last convolutional layer according to the inference result;
s12, upsampling the feature output F of the last convolution layer to the original size of the image, and carrying out weighted summation operation with the weight W to obtain the corresponding feature activation mapping.
9. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 7, wherein step S2 comprises the following sub-steps:
s21, obtaining an encryption key matrix according to the inference of the model, and generating a mask matrix which consists of keys and has the same dimension as the input key;
s22, copying a matrix f' identical to the activation mapping matrix f, and sorting all element values in the matrix in descending order;
s23, selecting the elements of the first theta% after f' sorting, finding the corresponding positions of the elements corresponding to f and the mask matrix, keeping the element values of the positions corresponding to the mask matrix, and setting the other positions as 0;
s24, encrypting the Image matrix Image using the following expression;
Encrypted Image＝Image+α*mask
the Encrypted Image is the Encrypted Image matrix, the Image is the original Image matrix, α is the hyper-parameter, and is used to control the encryption strength of the key, and the mask is the key matrix generated in step B3.
10. The method for stealing prevention of neural network model under AIoT scene as claimed in claim 7, wherein step S4 comprises the following sub-steps:
s41, decrypting the model inference data returned by the edge terminal equipment by using the decoder deployed in the step C4;
s42, obtaining the identification class corresponding to each image according to the model inference data decrypted in S41, and calculating the decryption key corresponding to each image through the decryption key and the encryption rule deployed on the server;
s43, according to the encrypted position information returned by the edge device and the key in S42.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911173524.XA CN110941855B (en) | 2019-11-26 | 2019-11-26 | Stealing and defending method for neural network model under AIoT scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911173524.XA CN110941855B (en) | 2019-11-26 | 2019-11-26 | Stealing and defending method for neural network model under AIoT scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110941855A true CN110941855A (en) | 2020-03-31 |
| CN110941855B CN110941855B (en) | 2022-02-15 |
Family
ID=69908106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911173524.XA Active CN110941855B (en) | 2019-11-26 | 2019-11-26 | Stealing and defending method for neural network model under AIoT scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110941855B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200234A (en) * | 2020-09-30 | 2021-01-08 | 支付宝(杭州)信息技术有限公司 | Method and device for preventing model stealing in model classification process |
| CN112528299A (en) * | 2020-12-04 | 2021-03-19 | 电子科技大学 | Deep neural network model safety protection method in industrial application scene |
| CN112765607A (en) * | 2021-01-19 | 2021-05-07 | 电子科技大学 | Neural network model backdoor attack detection method |
| CN112800467A (en) * | 2021-02-18 | 2021-05-14 | 支付宝(杭州)信息技术有限公司 | Online model training method, device and equipment based on data privacy protection |
| CN113098897A (en) * | 2021-04-26 | 2021-07-09 | 哈尔滨工业大学 | Data asymmetric encryption transmission method and system in networked control system |
| CN113204766A (en) * | 2021-05-25 | 2021-08-03 | 华中科技大学 | Distributed neural network deployment method, electronic device and storage medium |
| CN113381995A (en) * | 2021-06-08 | 2021-09-10 | 珠海格力电器股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113537400A (en) * | 2021-09-14 | 2021-10-22 | 浙江捷瑞电力科技有限公司 | Branch neural network-based edge computing node allocation and exit method |
| WO2022152153A1 (en) * | 2021-01-18 | 2022-07-21 | 北京灵汐科技有限公司 | Image processing method and device, key generation method and device, training method, and computer readable medium |
| CN114978899A (en) * | 2022-05-11 | 2022-08-30 | 业成科技（成都）有限公司 | AIoT equipment updating method and device |
| CN116032666A (en) * | 2023-03-29 | 2023-04-28 | 广东致盛技术有限公司 | Bian Yun cooperative equipment camouflage identification method and system based on learning model |
| TWI802161B (en) * | 2021-12-21 | 2023-05-11 | 建國科技大學 | Using aiot architecture to design an artificial intelligence identification system for transfer learning |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6281517B1 (en) * | 1998-08-17 | 2001-08-28 | Roche Diagnostics Corporation | Apparatus for monitoring pipetting operations |
| AU2013100576A4 (en) * | 2013-04-23 | 2013-06-06 | Huang, Xu PROF | Human Identification with Electroencephalogram (EEG) for the Future Network Security |
| CN109684855A (en) * | 2018-12-17 | 2019-04-26 | 电子科技大学 | A kind of combined depth learning training method based on secret protection technology |
| AU2019100349A4 (en) * | 2019-04-04 | 2019-05-09 | Lyu, Xinyue Miss | Face - Password Certification Based on Convolutional Neural Network |
| CN110035063A (en) * | 2019-03-08 | 2019-07-19 | 佛山市云米电器科技有限公司 | A kind of intelligent appliance data ciphering method neural network based |
| CN110188775A (en) * | 2019-05-28 | 2019-08-30 | 创意信息技术股份有限公司 | A kind of picture material based on united NNs model describes automatic generation method |
| CN110188827A (en) * | 2019-05-29 | 2019-08-30 | 创意信息技术股份有限公司 | A kind of scene recognition method based on convolutional neural networks and recurrence autocoder model |
| CN110287720A (en) * | 2019-07-01 | 2019-09-27 | 国网内蒙古东部电力有限公司 | A kind of access control method based on image recognition and user gradation |
| CN110490128A (en) * | 2019-08-16 | 2019-11-22 | 南京邮电大学 | A kind of hand-written recognition method based on encryption neural network |
-
2019
- 2019-11-26 CN CN201911173524.XA patent/CN110941855B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6281517B1 (en) * | 1998-08-17 | 2001-08-28 | Roche Diagnostics Corporation | Apparatus for monitoring pipetting operations |
| AU2013100576A4 (en) * | 2013-04-23 | 2013-06-06 | Huang, Xu PROF | Human Identification with Electroencephalogram (EEG) for the Future Network Security |
| CN109684855A (en) * | 2018-12-17 | 2019-04-26 | 电子科技大学 | A kind of combined depth learning training method based on secret protection technology |
| CN110035063A (en) * | 2019-03-08 | 2019-07-19 | 佛山市云米电器科技有限公司 | A kind of intelligent appliance data ciphering method neural network based |
| AU2019100349A4 (en) * | 2019-04-04 | 2019-05-09 | Lyu, Xinyue Miss | Face - Password Certification Based on Convolutional Neural Network |
| CN110188775A (en) * | 2019-05-28 | 2019-08-30 | 创意信息技术股份有限公司 | A kind of picture material based on united NNs model describes automatic generation method |
| CN110188827A (en) * | 2019-05-29 | 2019-08-30 | 创意信息技术股份有限公司 | A kind of scene recognition method based on convolutional neural networks and recurrence autocoder model |
| CN110287720A (en) * | 2019-07-01 | 2019-09-27 | 国网内蒙古东部电力有限公司 | A kind of access control method based on image recognition and user gradation |
| CN110490128A (en) * | 2019-08-16 | 2019-11-22 | 南京邮电大学 | A kind of hand-written recognition method based on encryption neural network |
Non-Patent Citations (1)
| Title |
|---|
| 侯英利: "基于蜜罐学习的神经网络入侵检测模型的研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11461701B2 (en) | 2020-09-30 | 2022-10-04 | Alipay (Hangzhou) Information Technology Co., Ltd. | Method and apparatus for preventing model theft during model-based classification |
| CN112200234A (en) * | 2020-09-30 | 2021-01-08 | 支付宝(杭州)信息技术有限公司 | Method and device for preventing model stealing in model classification process |
| CN112200234B (en) * | 2020-09-30 | 2022-04-12 | 支付宝(杭州)信息技术有限公司 | Method and device for preventing model stealing in model classification process |
| CN112528299B (en) * | 2020-12-04 | 2022-03-04 | 电子科技大学 | Deep neural network model safety protection method in industrial application scene |
| CN112528299A (en) * | 2020-12-04 | 2021-03-19 | 电子科技大学 | Deep neural network model safety protection method in industrial application scene |
| WO2022152153A1 (en) * | 2021-01-18 | 2022-07-21 | 北京灵汐科技有限公司 | Image processing method and device, key generation method and device, training method, and computer readable medium |
| CN112765607B (en) * | 2021-01-19 | 2022-05-17 | 电子科技大学 | Neural network model backdoor attack detection method |
| CN112765607A (en) * | 2021-01-19 | 2021-05-07 | 电子科技大学 | Neural network model backdoor attack detection method |
| CN112800467A (en) * | 2021-02-18 | 2021-05-14 | 支付宝(杭州)信息技术有限公司 | Online model training method, device and equipment based on data privacy protection |
| CN112800467B (en) * | 2021-02-18 | 2022-08-26 | 支付宝(杭州)信息技术有限公司 | Online model training method, device and equipment based on data privacy protection |
| CN113098897B (en) * | 2021-04-26 | 2023-05-23 | 哈尔滨工业大学 | Asymmetric encryption transmission method and system for data in networked control system |
| CN113098897A (en) * | 2021-04-26 | 2021-07-09 | 哈尔滨工业大学 | Data asymmetric encryption transmission method and system in networked control system |
| CN113204766B (en) * | 2021-05-25 | 2022-06-17 | 华中科技大学 | Distributed neural network deployment method, electronic device and storage medium |
| CN113204766A (en) * | 2021-05-25 | 2021-08-03 | 华中科技大学 | Distributed neural network deployment method, electronic device and storage medium |
| CN113381995A (en) * | 2021-06-08 | 2021-09-10 | 珠海格力电器股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113381995B (en) * | 2021-06-08 | 2023-07-07 | 珠海格力电器股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113537400A (en) * | 2021-09-14 | 2021-10-22 | 浙江捷瑞电力科技有限公司 | Branch neural network-based edge computing node allocation and exit method |
| CN113537400B (en) * | 2021-09-14 | 2024-03-19 | 浙江捷瑞电力科技有限公司 | Distribution and exit method of edge computing nodes based on branch neural network |
| TWI802161B (en) * | 2021-12-21 | 2023-05-11 | 建國科技大學 | Using aiot architecture to design an artificial intelligence identification system for transfer learning |
| CN114978899A (en) * | 2022-05-11 | 2022-08-30 | 业成科技（成都）有限公司 | AIoT equipment updating method and device |
| CN114978899B (en) * | 2022-05-11 | 2024-04-16 | 业成光电（深圳）有限公司 | AIoT equipment updating method and device |
| CN116032666A (en) * | 2023-03-29 | 2023-04-28 | 广东致盛技术有限公司 | Bian Yun cooperative equipment camouflage identification method and system based on learning model |
| CN116032666B (en) * | 2023-03-29 | 2023-09-22 | 广东致盛技术有限公司 | Bian Yun cooperative equipment camouflage identification method and system based on learning model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110941855B (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110941855B (en) | Stealing and defending method for neural network model under AIoT scene | |
| Bae et al. | Security and privacy issues in deep learning | |
| Namba et al. | Robust watermarking of neural network with exponential weighting | |
| Song et al. | Constructing unrestricted adversarial examples with generative models | |
| De Cristofaro | An overview of privacy in machine learning | |
| Chen et al. | Link prediction adversarial attack via iterative gradient attack | |
| CN113536383B (en) | Method and device for training graph neural network based on privacy protection | |
| Wang et al. | Adversarial attacks and defenses in machine learning-empowered communication systems and networks: A contemporary survey | |
| US20230306107A1 (en) | A Method of Training a Submodule and Preventing Capture of an AI Module | |
| US20210150042A1 (en) | Protecting information embedded in a machine learning model | |
| Lodeiro-Santiago et al. | Secure UAV‐based system to detect small boats using neural networks | |
| Wang et al. | Computational intelligence for information security: A survey | |
| Usama et al. | The adversarial machine learning conundrum: can the insecurity of ml become the achilles' heel of cognitive networks? | |
| Kaur et al. | A secure data classification model in cloud computing using machine learning approach | |
| US10956598B1 (en) | Method for preventing breach of original data for deep learning and data breach preventing device using them | |
| CN115687758A (en) | User classification model training method and user detection method | |
| Gangula et al. | Network intrusion detection system for Internet of Things based on enhanced flower pollination algorithm and ensemble classifier | |
| Jia et al. | Subnetwork-lossless robust watermarking for hostile theft attacks in deep transfer learning models | |
| US20210224688A1 (en) | Method of training a module and method of preventing capture of an ai module | |
| Benkraouda et al. | Image reconstruction attacks on distributed machine learning models | |
| Xie et al. | A survey on vulnerability of federated learning: A learning algorithm perspective | |
| Karampidis et al. | StegoPass–utilization of steganography to produce a novel unbreakable biometric based password authentication scheme | |
| Chen et al. | DeepGuard: Backdoor Attack Detection and Identification Schemes in Privacy‐Preserving Deep Neural Networks | |
| Pavate et al. | Analyzing probabilistic adversarial samples to attack cloud vision image classifier service | |
| US20230050484A1 (en) | Method of Training a Module and Method of Preventing Capture of an AI Module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |