CN112329052A - Model privacy protection method and device - Google Patents
Model privacy protection method and device Download PDFInfo
- Publication number
- CN112329052A CN112329052A CN202011155392.0A CN202011155392A CN112329052A CN 112329052 A CN112329052 A CN 112329052A CN 202011155392 A CN202011155392 A CN 202011155392A CN 112329052 A CN112329052 A CN 112329052A
- Authority
- CN
- China
- Prior art keywords
- model
- training
- data
- data set
- final
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 83
- 238000012549 training Methods 0.000 claims abstract description 226
- 230000008569 process Effects 0.000 claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 49
- 235000015250 liver sausages Nutrition 0.000 claims abstract 3
- XYSQXZCMOLNHOI-UHFFFAOYSA-N s-[2-[[4-(acetylsulfamoyl)phenyl]carbamoyl]phenyl] 5-pyridin-1-ium-1-ylpentanethioate;bromide Chemical compound [Br-].C1=CC(S(=O)(=O)NC(=O)C)=CC=C1NC(=O)C1=CC=CC=C1SC(=O)CCCC[N+]1=CC=CC=C1 XYSQXZCMOLNHOI-UHFFFAOYSA-N 0.000 claims abstract 3
- 238000010801 machine learning Methods 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 8
- 230000002776 aggregation Effects 0.000 claims description 7
- 238000004220 aggregation Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioethics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model privacy protection method, which comprises the following steps: determining a target protection mode for privacy protection of the model based on the current training scene; if the target protection mode is a PATE mode, in the process of using the training data set to train the model, the noise processing is carried out on the voting result generated in the middle, and after the training is finished, the final first model to be used is obtained; and if the target protection mode is a DPSGD mode with a reduced random gradient of difference privacy, in the process of using the training data set to train the model, performing noise addition on the gradient calculated after the training data set is input into the model, and obtaining a final second model to be used after training is completed. By applying the technical scheme provided by the application, privacy protection can be better and effectively carried out, and the safety of training data and models is improved. The application also discloses a model privacy protection device, which has corresponding technical effects.
Description
Technical Field
The present application relates to the field of computer application technologies, and in particular, to a model privacy protection method and apparatus.
Background
With the rapid development of computer technology, the development of machine learning is becoming more mature. Machine learning is widely applied to various scenes such as a recommendation system, face recognition, smart homes, unmanned automobiles and the like, and brings a lot of changes to life of people.
The performance of the machine learning model is related to the size of the training data set. Training data can relate to shopping information, medical records of physical examination and the like, and a user does not want the data relating to personal privacy to be leaked, but a machine learning model can implicitly remember the details in the training process, and inadvertently reveals the details in the reasoning and forecasting processes, and can be obtained by a malicious person in a data stealing mode. Therefore, machine learning itself has security problems such as privacy security and privacy disclosure.
In summary, how to effectively perform privacy protection in the model training process and improve the safety of training data and models thereof is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a model privacy protection method and device so as to effectively protect privacy in a model training process and improve the safety of training data and a model.
In order to solve the technical problem, the application provides the following technical scheme:
a model privacy protection method, comprising:
determining a target protection mode for privacy protection of the model based on the current training scene;
if the target protection mode is a PATE mode, in the process of using a training data set to train the model, noise processing is carried out on the voting result generated in the middle, and after the training is finished, a final first model to be used is obtained;
and if the target protection mode is a DPSGD mode with a reduced random gradient of difference privacy, in the process of using the training data set to train the model, performing noise addition on the gradient calculated after inputting the training data set into the model, and obtaining a final second model to be used after training.
In a specific embodiment of the present application, the training data set includes a private data set and a public data set, the denoising processing is performed on the voting result generated in the middle in the process of performing model training using the training data set, and after the training is completed, the final first model to be used is obtained, including:
dividing the private data set into disjoint N private data subsets, N being a positive integer;
respectively using each privacy data subset to train a machine learning model to obtain N teacher models;
performing category prediction on each data in the public data set by using the N teacher models to obtain the vote number of each category of each data in the public data set;
performing noise aggregation processing on the obtained votes, and determining a final label of each datum;
training a student model using the data in the common data set and the final labels for the corresponding data;
and determining the trained student model as a final first model to be used.
In a specific embodiment of the present application, the performing noise aggregation processing on the obtained votes to determine a final label of each data includes:
carrying out noise and disturbance adding processing on the votes of all categories of each data in the public data set;
and determining a final label of each data based on the votes of the various categories after the noise disturbance processing.
In an embodiment of the present application, the determining a final label of each data based on the votes of the respective categories after the noise-added perturbation processing includes:
for each data, determining the category of the maximum vote number of the data based on the vote number of each category subjected to noise disturbance processing;
and if the vote number of the category with the maximum vote number is larger than a preset vote number threshold value, determining the category with the maximum vote number of the data as a final label of the data.
In a specific embodiment of the present application, the determining the final label of each data based on the votes of the respective categories after the noise disturbance processing includes:
determining a standby label of each data under each judgment threshold value in a plurality of different judgment threshold values based on the votes of each category after the noise disturbance processing;
for each decision threshold in the plurality of different decision thresholds, determining the accuracy of the decision result under the decision threshold based on the standby label of each data and the historical label of the corresponding data under the decision threshold;
and determining the standby label of each data under the judgment threshold corresponding to the highest judgment result accuracy as the final label of the corresponding data.
In a specific embodiment of the present application, in the process of training a model using the training data set, the noise adding process is performed on a gradient calculated after the training data set is input to the model, and after the training is completed, a final second model to be used is obtained, including:
in the iterative training process of each batch of each round, inputting preset batch size data obtained by sampling the training data set into a model, and calculating a gradient;
cutting the calculated gradient, and adding a noise disturbance gradient to obtain a new gradient;
updating model parameters using the new gradient;
after a plurality of rounds of iterative training, a final second model to be used is obtained.
In one embodiment of the present application, the method further includes:
in the iterative training process, determining the accuracy of the model after the first round of training;
if the accuracy of the model after the first round of training is not lower than a preset accuracy threshold, continuing to perform iterative training;
and if the accuracy of the model after the first round of training is lower than the accuracy threshold, performing network initialization again and performing iterative training again.
In a specific embodiment of the present application, the accuracy threshold is determined by:
before formal iterative training, performing multiple pretests, and executing M rounds of training for each pretest, wherein M is a positive integer;
determining the accuracy of the model after each pre-test round of training;
and determining an accuracy threshold based on the determined accuracy of the model after each pre-test round of training.
In one embodiment of the present application, the method further includes:
and determining a differential privacy budget according to the parameters in the training process.
A model privacy preserving apparatus comprising:
the protection mode determining module is used for determining a target protection mode for privacy protection of the model based on the current training scene;
a first model obtaining module, configured to, if the target protection mode is a PATE mode, perform noise addition on a voting result generated in the middle in a process of performing model training using a training data set, and obtain a final first model to be used after the training is completed;
and the second model obtaining module is used for performing noise adding processing on the calculated gradient after the training data set is input into the model in the process of performing model training by using the training data set if the target protection mode is a differential privacy random gradient descent (DPSGD) mode, and obtaining a final second model to be used after the training is finished.
By applying the technical scheme provided by the embodiment of the application, a target protection mode for privacy protection of the model is determined based on a current training scene, when the determined target protection mode is a PATE mode, in the process of training the model by using the training data set, noise processing is performed on a voting result generated in the middle, after the training is completed, the first model to be used is obtained, when the determined target protection mode is a DPSGD mode, in the process of training the model by using the training data set, noise processing is performed on a gradient calculated after the training data set is input into the model, and after the training is completed, the final second model to be used is obtained. After the first model to be used or the second model to be used is obtained, the first model or the second model can be applied to an actual application scene. In the process of model training, noise adding processing is carried out on the voting result generated in the middle, or noise adding processing is carried out on the gradient obtained through calculation, so that privacy protection can be effectively carried out, and the safety of training data and the safety of the model are improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of an implementation of a model privacy protection method in an embodiment of the present application;
FIG. 2 is a schematic diagram of a training process of a teacher model and a student model in an embodiment of the present application;
FIG. 3 is a diagram illustrating an iterative training process in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a model privacy protection apparatus in an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, a flowchart of an implementation of a model privacy protection method provided in an embodiment of the present application is shown, where the method may include the following steps:
s110: and determining a target protection mode for privacy protection of the model based on the current training scene.
In the embodiment of the present application, different training scenarios may correspond to different target protection modes for privacy protection of the model. The corresponding relation between the training scene and the target protection mode can be established in advance through experiments, historical data and the like.
If the training scene is a scene with a larger training data set, the corresponding target protection mode may be a PATE (private Aggregation of Teacher Ensemble) mode;
if the training scenario is a deep learning scenario or a Gradient descent scenario used in a conventional machine learning model, the corresponding target protection mode may be a dpsgd (differential priority storage Gradient) mode.
When the model training is needed, the current training scenario, such as the size of the training data set, whether gradient descent is used, etc., may be determined first. Based on the current training scenario, it may be determined whether the target protection mode for privacy protection of the model is the PATE mode or the DPSGD mode.
S120: and if the target protection mode is a PATE mode, performing noise adding processing on the voting result generated in the middle in the process of performing model training by using the training data set, and obtaining the final first model to be used after the training is finished.
And based on the current training scene, after a target protection mode for privacy protection of the model is determined, if the target protection mode is determined to be a PATE mode, in the process of training the model by using the training data set, noise is added to a voting result generated in the middle, and after training is finished, a final first model to be used is obtained.
In one embodiment of the present application, as shown in fig. 2, the training data set includes a private data set and a public data set, and the step may include the steps of:
the method comprises the following steps: dividing a privacy data set into disjoint N privacy data subsets, wherein N is a positive integer;
step two: respectively using each privacy data subset to train a machine learning model to obtain N teacher models;
step three: carrying out category prediction on each data in the public data set by using the N teacher models to obtain the vote number of each category of each data in the public data set;
step four: performing noise aggregation processing on the obtained votes, and determining a final label of each datum;
step five: training a student model using the data in the common data set and the final labels of the corresponding data;
step six: and determining the trained student model as a final first model to be used.
For convenience of description, the above steps are combined for illustration.
In embodiments of the present application, the training data set may include a private data set and a public data set.
When model data is to be performed, the private data set may be divided into disjoint N private data subsets, such as data set 1, data set 2, … …, and data set N shown in fig. 2, where N is a positive integer. And respectively using each privacy data subset to train the machine learning model, so as to obtain N teacher models. I.e. training independently on each subset of private data results in a corresponding teacher model.
And performing category prediction on each data in the public data set by using the N teacher models to obtain the vote number of each category of each data in the public data set. Here, the prediction of each teacher model is considered as one vote. Each data in the common data set may receive N votes for a class prediction, each class of each data having a corresponding number of votes.
The resulting number of votes is noise aggregated and a final label for each data in the common data set can be determined. Specifically, noise disturbance processing may be performed on the vote count of each category of each data in the common data set, and then the final label of each data may be determined based on the vote count of each category after the noise disturbance processing. The noise adding processing is carried out on the voting result generated in the middle, randomness can be introduced, so that the influence caused by a single data individual is not fixed, but the influence of the overall data still exists, and the model training can be carried out while the privacy of the individual is protected.
In one embodiment of the present application, after noise disturbance processing is performed on the votes of each category of each data in the public data set, the number of votes after noise disturbance of each category of each data in the public data set can be obtained, and for each data, the category with the maximum number of votes can be determined as the final label of the data. For example, in the public data set, the number of votes after noise disturbance for the category a of one data is 45, and the number of votes after noise disturbance for the category B is 55, and the category B can be directly determined as the final label of the data.
In another embodiment of the present application, for each data in the common data set, the category of the maximum number of votes for the data may be determined based on the number of votes for each category subjected to the noise disturbance processing, and if the number of votes for the category of the maximum number of votes is greater than a preset threshold of votes, the category of the maximum number of votes for the data may be determined as the final label of the data.
In this embodiment, a threshold of the number of tickets may be preset, and the threshold of the number of tickets may be set and adjusted according to actual conditions. When the vote number of a category of data is larger than the vote number threshold value, the teacher consensus degree is considered to be higher, the category is determined as the final label of the data, and the training effect on the subsequent student model is better. In the above example, if the threshold of the number of votes is 60, although the number of votes after noise disturbance of the class B is the largest, since the number of votes 55 is smaller than the threshold of the number of votes 60, the class B is not determined as the final label of the data in the embodiment of the present application, and the training of the student model using the data is abandoned. If the vote count threshold is 52, then since the noise perturbed vote count for category B is the largest and its vote count 55 is greater than the vote count threshold 52, category B may be determined to be the final label for the data.
In another specific embodiment of the present application, the data in the common data set has a history tag, and when determining the final tag of each data, the standby tag of each data under each of a plurality of different decision thresholds can be determined based on the votes of the categories after the noise disturbance processing, for each of the plurality of different decision thresholds, the accuracy of the decision result under the decision threshold is determined based on the standby tag of each data under the decision threshold and the history tag of the corresponding data, and the standby tag of each data under the decision threshold corresponding to the highest accuracy of the decision result is determined as the final tag of the corresponding data.
In this embodiment, each data in the common data set has a history label, which may be calibrated at the time of previous model training. A plurality of determination threshold values may be set in advance. And after obtaining the vote number of each category of each data in the public data set, carrying out noise disturbance processing on the vote number of each category of each data to obtain the vote number of each category of each data after noise disturbance. The inactive label for each data at each of a plurality of different decision thresholds can be determined based on the number of votes for each category after the noisy-perturbation process. That is, for each decision threshold of the multiple decision thresholds, if the vote count of the category of the maximum vote count after noise disturbance of one data is greater than the decision threshold, the category of the maximum vote count of the data is determined as the standby label of the data. And if not, abandoning the training of the subsequent student model by using the data.
For each of a plurality of different decision thresholds, the accuracy of the decision result at the decision threshold may be determined based on the standby label of each data and the historical label of the corresponding data at the decision threshold. For example, there are two decision thresholds, namely decision threshold X, at which the standby label of data 1 is class a, the standby label of data 2 is class a, and decision threshold Y, at which the standby label of data 1 is class B, and the standby label of data 2 is class a. While the history label for data 1 is class B and the history label for data 2 is class a. By comparing the standby label and the history label of each data, it can be seen that the accuracy of the determination result is 50% at the determination threshold 1 and 100% at the determination threshold 2.
After the determination result accuracy under each determination threshold is obtained, the standby label of each data under the determination threshold corresponding to the highest determination result accuracy can be determined as the final label of the corresponding data. The correctness of the judgment result depends on the high consistency of the consensus of the teacher models, and the higher accuracy of the judgment result means that the teacher models predict higher consensus, namely, the number of the teacher models which are considered to be the same label is larger, which means that the judgment result has universality, and less individual privacy is exposed. Some good synergy between privacy and learning is intuitively reflected.
And determining the final label of each datum, and if some data do not obtain the final label, giving up the use of the data for subsequent training of the student model.
Using the data in the common data set and the final labels for the corresponding data, a student model can be trained. And determining the trained student model as a final first model to be used. And then the first model to be used can be applied to practical application scenes, such as application scenes of a recommendation system, face recognition, smart homes, unmanned automobiles and the like. The student models are trained by using the data sets labeled by the teacher models, prediction service is performed by using the student models, the student models used for disclosure can be indirectly trained under the condition of not using sensitive data, and the model inversion attack can be prevented from stealing original data.
S130: and if the target protection mode is a DPSGD mode with a reduced random gradient of difference privacy, in the process of using the training data set to train the model, performing noise addition on the gradient calculated after the training data set is input into the model, and obtaining a final second model to be used after training is completed.
Based on the current training scene, after a target protection mode for privacy protection of the model is determined, if the target protection mode is a DPSGD mode, noise adding processing can be performed on the gradient obtained by calculation after the training data set is input into the model in the process of model training by using the training data set, namely, disturbance is added to the gradient in the iterative process of random gradient reduction. And after the training is finished, obtaining a final second model to be used.
In one embodiment of the present application, the step may comprise the steps of:
the method comprises the following steps: in the iterative training process of each batch of each round, inputting preset batch size data obtained by sampling a training data set into a model, and calculating a gradient;
step two: cutting the calculated gradient, and adding a noise disturbance gradient to obtain a new gradient;
step three: updating the model parameters using the new gradient;
step four: after a plurality of rounds of iterative training, a final second model to be used is obtained.
For convenience of description, the above steps are combined for illustration.
In performing model training, multiple rounds of iterative training may be performed, each round of training may include multiple batches.
It should be noted that, in this embodiment, during model training, multiple batches of training may be performed in each iteration of training, a round may be represented as an Epoch, each round may be trained using data of a preset batch size, each round performs multiple batches of training, and after each round of training is completed, all data in the training data set is used. The Batch may be denoted Batch.
As shown in fig. 3, the model parameters C and σ may be set first. Wherein, C is a clipping threshold value used for distinguishing whether clipping is needed, and sigma is a standard deviation used for controlling the addition amount of noise. In the iterative training process of each batch of each round, the training data set can be sampled to obtain data with a preset batch size, and then the data with the batch size obtained by sampling is input into the model to calculate the gradient g. And performing clipping processing on the calculated gradient. The second-order norm E of the gradient g may be calculated first, and it is determined whether E/C is greater than 1, if so, it is determined that clipping processing is required, and if not, clipping processing may not be performed. Noise is then added to perturb the gradient and obtain a new gradient. Gaussian noise with standard deviation σ C may be added. This may be such that the single gradient effect is not too great. And updating model network parameters by using the new gradient. This is done for each round of iterative training for each batch. After a plurality of rounds of iterative training, a final second model to be used is obtained.
In practical application, the total iteration round number may be set, and the final second model to be used is obtained when the total iteration round number is reached, or an accuracy threshold may be set, and after multiple rounds of iterative training are performed, if the model prediction accuracy reaches the accuracy threshold, the training is stopped, and the current model is determined as the second model to be used. And then the second model to be used can be applied to practical application scenes, such as application scenes of a recommendation system, face recognition, smart homes, unmanned automobiles and the like.
In the practice process, the influence of sampling, noise randomness, initialization and other factors is found, and in the initial stage, if the model accuracy is low, the final effect is not ideal after a plurality of rounds of iterative training, and even the model accuracy continuously decreases in the training process.
In view of this, in the embodiment of the present application, the accuracy of the model after the first round of training may be determined first, and if the accuracy of the model after the first round of training is not lower than the preset accuracy threshold, the iterative training is continued. A model accuracy threshold value can be preset, after the first round of iterative training is finished, the model accuracy after the first round of training is determined, if the model accuracy is not lower than the model accuracy threshold value, the current training effect can be considered to be good, and the iterative training can be continued. If the accuracy of the model after the first round of training is lower than the accuracy threshold, the iterative training can be stopped, and the network initialization and the iterative training are carried out again. Therefore, the situation that the accuracy rate of the model is continuously low can be effectively avoided, invalid resource occupation is avoided, and the precision of the training model is improved.
The model accuracy threshold can be set and adjusted according to actual conditions.
In one embodiment, the model accuracy may be determined by:
before formal iterative training, performing multiple pretests, and executing M rounds of training for each pretest, wherein M is a positive integer;
determining the accuracy of the model after each pre-test round of training;
and determining an accuracy threshold based on the determined accuracy of the model after each pre-test round of training.
Before formal iterative training, a plurality of pretests can be performed, each pretest can execute M rounds of training, and M is a positive integer. The model accuracy after each pretest round of training can be determined. Each pretest may yield M model accuracies.
Based on the determined model accuracy for each pretest round of training, an accuracy threshold may be determined. Specifically, the median of the M model accuracy rates obtained in each pretest may be determined, and then the average of the median may be determined as the accuracy rate threshold.
In an embodiment of the application, after the model training is completed, the differential privacy budget may also be determined according to parameters in the training process. Specifically, teacher voting data or data such as noise volume and iteration turns can be used for privacy analysis by using the Ranyi differential privacy to quantitatively express the privacy protection degree, determine the privacy protection effect and seek the balance between the model performance and the privacy protection degree.
The method includes the steps that firstly, a target protection mode used for protecting the privacy of a model is determined based on a current training scene, when the determined target protection mode is a PATE mode, noise is added to a voting result generated in the middle in the process of training the model by using a training data set, the first model to be used is obtained after training is completed, when the determined target protection mode is a DPSGD mode, in the process of training the model by using the training data set, noise is added to a gradient obtained by calculation after the training data set is input into the model, and a final second model to be used is obtained after training is completed. After the first model to be used or the second model to be used is obtained, the first model or the second model can be applied to an actual application scene. In the process of model training, noise adding processing is carried out on the voting result generated in the middle, or noise adding processing is carried out on the gradient obtained through calculation, so that privacy protection can be effectively carried out, and the safety of training data and the safety of the model are improved.
The model privacy protection method provided by the embodiment of the application can solve the privacy protection problem of training data in the current machine learning service, namely, the training data stealing possibly from a malicious attacker can be resisted, and the method comprises model reverse attack (model inversion attack), member reasoning attack and the like. Noise according with a certain rule is introduced in the training process of the machine learning model, so that the influence of data individuals used for training on the final model is limited, the overall statistical rule is still available, and the individual privacy of training data is protected while the model training can be completed.
Corresponding to the above method embodiment, the present application further provides a model privacy protection apparatus, and the model privacy protection apparatus described below and the model privacy protection method described above may be referred to in correspondence.
Referring to fig. 4, the apparatus may include the following modules:
a protection mode determining module 410, configured to determine, based on a current training scenario, a target protection mode for privacy protection of a model;
a first model obtaining module 420, configured to, if the target protection mode is a PATE mode, perform noise addition on a voting result generated in the middle in a process of performing model training using a training data set, and obtain a final to-be-used first model after training is completed;
and a second model obtaining module 430, configured to, if the target protection mode is a differential privacy random gradient descent (DPSGD) mode, perform noise addition on a gradient calculated after the training data set is input to the model in a process of performing model training using the training data set, and obtain a final second model to be used after the training is completed.
By applying the device provided by the embodiment of the application, a target protection mode for privacy protection of a model is determined based on a current training scene, when the determined target protection mode is a PATE mode, in the process of training the model by using a training data set, noise processing is performed on a voting result generated in the middle, after training is completed, a first model to be used is obtained, when the determined target protection mode is a DPSGD mode, in the process of training the model by using the training data set, noise processing is performed on a gradient calculated after the training data set is input into the model, and after training is completed, a final second model to be used is obtained. After the first model to be used or the second model to be used is obtained, the first model or the second model can be applied to an actual application scene. In the process of model training, noise adding processing is carried out on the voting result generated in the middle, or noise adding processing is carried out on the gradient obtained through calculation, so that privacy protection can be effectively carried out, and the safety of training data and the safety of the model are improved.
In one embodiment of the present application, the training data set includes a private data set and a public data set, and the first model obtaining module 420 is configured to:
dividing a privacy data set into disjoint N privacy data subsets, wherein N is a positive integer;
respectively using each privacy data subset to train a machine learning model to obtain N teacher models;
carrying out category prediction on each data in the public data set by using the N teacher models to obtain the vote number of each category of each data in the public data set;
performing noise aggregation processing on the obtained votes, and determining a final label of each datum;
training a student model using the data in the common data set and the final labels of the corresponding data;
and determining the trained student model as a final first model to be used.
In one embodiment of the present application, the first model obtaining module 420 is configured to:
carrying out noise disturbance processing on the votes of all categories of each data in the public data set;
and determining a final label of each data based on the votes of the various categories after the noise disturbance processing.
In one embodiment of the present application, the first model obtaining module 420 is configured to:
for each data, determining the category of the maximum vote number of the data based on the vote number of each category subjected to noise disturbance processing;
and if the vote number of the category with the maximum vote number is larger than a preset vote number threshold value, determining the category with the maximum vote number of the data as a final label of the data.
In one embodiment of the present application, the data in the private data set has a history tag, and the first model obtaining module 420 is configured to:
determining a standby label of each data under each judgment threshold value in a plurality of different judgment threshold values based on the votes of each category after the noise disturbance processing;
for each decision threshold value in a plurality of different decision threshold values, determining the accuracy of the decision result under the decision threshold value based on the standby label of each data and the historical label of the corresponding data under the decision threshold value;
and determining the standby label of each data under the judgment threshold corresponding to the highest judgment result accuracy as the final label of the corresponding data.
In a specific embodiment of the present application, the second model obtaining module 430 is configured to:
in the iterative training process of each batch of each round, inputting preset batch size data obtained by sampling a training data set into a model, and calculating a gradient;
cutting the calculated gradient, and adding a noise disturbance gradient to obtain a new gradient;
updating the model parameters using the new gradient;
after a plurality of rounds of iterative training, a final second model to be used is obtained.
In a specific embodiment of the present application, the second model obtaining module 430 is further configured to:
in the iterative training process, determining the accuracy of the model after the first round of training;
if the accuracy of the model after the first round of training is not lower than the preset accuracy threshold, continuing to carry out iterative training;
and if the accuracy of the model after the first round of training is lower than the accuracy threshold, performing network initialization again and performing iterative training again.
In one embodiment of the present application, the second model obtaining module 430 is further configured to determine the accuracy threshold by:
before formal iterative training, performing multiple pretests, and executing M rounds of training for each pretest, wherein M is a positive integer;
determining the accuracy of the model after each pre-test round of training;
and determining an accuracy threshold based on the determined accuracy of the model after each pre-test round of training.
In a specific embodiment of the present application, the system further includes a privacy budgeting module, configured to:
and determining a differential privacy budget according to the parameters in the training process.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The principle and the implementation of the present application are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A method for model privacy protection, comprising:
determining a target protection mode for privacy protection of the model based on the current training scene;
if the target protection mode is a PATE mode, in the process of using a training data set to train the model, noise processing is carried out on the voting result generated in the middle, and after the training is finished, a final first model to be used is obtained;
and if the target protection mode is a DPSGD mode with a reduced random gradient of difference privacy, in the process of using the training data set to train the model, performing noise addition on the gradient calculated after inputting the training data set into the model, and obtaining a final second model to be used after training.
2. The method according to claim 1, wherein the training data set includes a private data set and a public data set, and the noise adding processing is performed on the voting result generated in the middle during the model training process using the training data set, and after the training is completed, the obtaining of the final first model to be used includes:
dividing the private data set into disjoint N private data subsets, N being a positive integer;
respectively using each privacy data subset to train a machine learning model to obtain N teacher models;
performing category prediction on each data in the public data set by using the N teacher models to obtain the vote number of each category of each data in the public data set;
performing noise aggregation processing on the obtained votes, and determining a final label of each datum;
training a student model using the data in the common data set and the final labels for the corresponding data;
and determining the trained student model as a final first model to be used.
3. The method of claim 2, wherein the performing a noise aggregation process on the obtained votes to determine a final label for each datum comprises:
carrying out noise and disturbance adding processing on the votes of all categories of each data in the public data set;
and determining a final label of each data based on the votes of the various categories after the noise disturbance processing.
4. The method of claim 3, wherein determining the final label for each data based on the votes for the respective categories after the noisy perturbation process comprises:
for each data, determining the category of the maximum vote number of the data based on the vote number of each category subjected to noise disturbance processing;
and if the vote number of the category with the maximum vote number is larger than a preset vote number threshold value, determining the category with the maximum vote number of the data as a final label of the data.
5. The method of claim 3, wherein the data in the private data set has historical tags, and wherein determining the final tag for each data based on the number of votes for each category after the noisy perturbation process comprises:
determining a standby label of each data under each judgment threshold value in a plurality of different judgment threshold values based on the votes of each category after the noise disturbance processing;
for each decision threshold in the plurality of different decision thresholds, determining the accuracy of the decision result under the decision threshold based on the standby label of each data and the historical label of the corresponding data under the decision threshold;
and determining the standby label of each data under the judgment threshold corresponding to the highest judgment result accuracy as the final label of the corresponding data.
6. The method according to claim 1, wherein in the process of training the model by using the training data set, the noise processing is performed on the gradient calculated after the training data set is input into the model, and after the training is completed, a final second model to be used is obtained, including:
in the iterative training process of each batch of each round, inputting preset batch size data obtained by sampling the training data set into a model, and calculating a gradient;
cutting the calculated gradient, and adding a noise disturbance gradient to obtain a new gradient;
updating model parameters using the new gradient;
after a plurality of rounds of iterative training, a final second model to be used is obtained.
7. The method of claim 6, further comprising:
in the iterative training process, determining the accuracy of the model after the first round of training;
if the accuracy of the model after the first round of training is not lower than a preset accuracy threshold, continuing to perform iterative training;
and if the accuracy of the model after the first round of training is lower than the accuracy threshold, performing network initialization again and performing iterative training again.
8. The method of claim 7, wherein the accuracy threshold is determined by:
before formal iterative training, performing multiple pretests, and executing M rounds of training for each pretest, wherein M is a positive integer;
determining the accuracy of the model after each pre-test round of training;
and determining an accuracy threshold based on the determined accuracy of the model after each pre-test round of training.
9. The method of any one of claims 1 to 8, further comprising:
and determining a differential privacy budget according to the parameters in the training process.
10. A model privacy protection apparatus, comprising:
the protection mode determining module is used for determining a target protection mode for privacy protection of the model based on the current training scene;
a first model obtaining module, configured to, if the target protection mode is a PATE mode, perform noise addition on a voting result generated in the middle in a process of performing model training using a training data set, and obtain a final first model to be used after the training is completed;
and the second model obtaining module is used for performing noise adding processing on the calculated gradient after the training data set is input into the model in the process of performing model training by using the training data set if the target protection mode is a differential privacy random gradient descent (DPSGD) mode, and obtaining a final second model to be used after the training is finished.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011155392.0A CN112329052B (en) | 2020-10-26 | 2020-10-26 | Model privacy protection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011155392.0A CN112329052B (en) | 2020-10-26 | 2020-10-26 | Model privacy protection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112329052A true CN112329052A (en) | 2021-02-05 |
| CN112329052B CN112329052B (en) | 2024-08-06 |
Family
ID=74311704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011155392.0A Active CN112329052B (en) | 2020-10-26 | 2020-10-26 | Model privacy protection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112329052B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112766422A (en) * | 2021-03-15 | 2021-05-07 | 山东大学 | Privacy protection method based on lightweight face recognition model |
| CN113360945A (en) * | 2021-06-29 | 2021-09-07 | 招商局金融科技有限公司 | Noise adding method, device, equipment and medium based on differential privacy |
| CN113536373A (en) * | 2021-07-07 | 2021-10-22 | 河南大学 | Desensitization meteorological data generation method |
| CN114003949A (en) * | 2021-09-30 | 2022-02-01 | 清华大学 | Model training method and device based on private data set |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784091A (en) * | 2019-01-16 | 2019-05-21 | 福州大学 | A kind of list data method for secret protection merging difference privacy GAN and PATE model |
| CN110647765A (en) * | 2019-09-19 | 2020-01-03 | 济南大学 | Privacy protection method and system based on knowledge migration under collaborative learning framework |
| CN111091199A (en) * | 2019-12-20 | 2020-05-01 | 哈尔滨工业大学（深圳） | Federal learning method and device based on differential privacy and storage medium |
| CN111400754A (en) * | 2020-03-11 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Construction method and device of user classification system for protecting user privacy |
-
2020
- 2020-10-26 CN CN202011155392.0A patent/CN112329052B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784091A (en) * | 2019-01-16 | 2019-05-21 | 福州大学 | A kind of list data method for secret protection merging difference privacy GAN and PATE model |
| CN110647765A (en) * | 2019-09-19 | 2020-01-03 | 济南大学 | Privacy protection method and system based on knowledge migration under collaborative learning framework |
| CN111091199A (en) * | 2019-12-20 | 2020-05-01 | 哈尔滨工业大学（深圳） | Federal learning method and device based on differential privacy and storage medium |
| CN111400754A (en) * | 2020-03-11 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Construction method and device of user classification system for protecting user privacy |
Non-Patent Citations (1)
| Title |
|---|
| 谭作文等: "机器学习隐私保护研究综述", 软件学报, 15 July 2020 (2020-07-15), pages 2127 - 2156 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112766422A (en) * | 2021-03-15 | 2021-05-07 | 山东大学 | Privacy protection method based on lightweight face recognition model |
| CN113360945A (en) * | 2021-06-29 | 2021-09-07 | 招商局金融科技有限公司 | Noise adding method, device, equipment and medium based on differential privacy |
| CN113536373A (en) * | 2021-07-07 | 2021-10-22 | 河南大学 | Desensitization meteorological data generation method |
| CN114003949A (en) * | 2021-09-30 | 2022-02-01 | 清华大学 | Model training method and device based on private data set |
| CN114003949B (en) * | 2021-09-30 | 2022-08-30 | 清华大学 | Model training method and device based on private data set |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112329052B (en) | 2024-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112329052A (en) | Model privacy protection method and device | |
| CN108737406B (en) | Method and system for detecting abnormal flow data | |
| US11734353B2 (en) | Multi-sampling model training method and device | |
| CN109583332B (en) | Face recognition method, face recognition system, medium, and electronic device | |
| CN113688042B (en) | Determination method and device of test scene, electronic equipment and readable storage medium | |
| CN110741388A (en) | Confrontation sample detection method and device, computing equipment and computer storage medium | |
| CN111260620B (en) | Image anomaly detection method and device and electronic equipment | |
| CN110245488B (en) | Method, device, terminal and computer readable storage medium for detecting password strength | |
| CN111126623B (en) | Model updating method, device and equipment | |
| CN106778357A (en) | The detection method and device of a kind of webpage tamper | |
| CN114417427A (en) | Deep learning-oriented data sensitivity attribute desensitization system and method | |
| CN113221104A (en) | User abnormal behavior detection method and user behavior reconstruction model training method | |
| CN114742319A (en) | Method, system and storage medium for predicting scores of law examination objective questions | |
| CN115659183A (en) | Product detection method, device, equipment and storage medium | |
| CN108696397B (en) | Power grid information security assessment method and device based on AHP and big data | |
| CN114626553A (en) | Training method and device of financial data monitoring model and computer equipment | |
| CN114169439A (en) | Abnormal communication number identification method and device, electronic equipment and readable medium | |
| CN116545764B (en) | Abnormal data detection method, system and equipment of industrial Internet | |
| CN111026087B (en) | Weight-containing nonlinear industrial system fault detection method and device based on data | |
| CN112884480A (en) | Method and device for constructing abnormal transaction identification model, computer equipment and medium | |
| CN113298264A (en) | Equipment authentication method and system based on shallow self-learning algorithm rejection inference and electronic equipment | |
| CN117522586A (en) | Financial abnormal behavior detection method and device | |
| CN112116358A (en) | Transaction fraud prediction method and device and electronic equipment | |
| CN114050941B (en) | Defect account detection method and system based on kernel density estimation | |
| CN114707420A (en) | Credit fraud behavior identification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |