CN117474118A - Federal learning privacy protection method based on improved diffusion model - Google Patents
Federal learning privacy protection method based on improved diffusion model Download PDFInfo
- Publication number
- CN117474118A CN117474118A CN202311205515.0A CN202311205515A CN117474118A CN 117474118 A CN117474118 A CN 117474118A CN 202311205515 A CN202311205515 A CN 202311205515A CN 117474118 A CN117474118 A CN 117474118A
- Authority
- CN
- China
- Prior art keywords
- model
- central server
- federal learning
- noise
- diffusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009792 diffusion process Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000005070 sampling Methods 0.000 claims abstract description 28
- 238000004891 communication Methods 0.000 claims abstract description 19
- 238000013528 artificial neural network Methods 0.000 claims abstract description 18
- 230000004931 aggregating effect Effects 0.000 claims abstract description 3
- 230000006870 function Effects 0.000 claims description 23
- 238000009826 distribution Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 12
- 238000013499 data model Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 6
- 238000007476 Maximum Likelihood Methods 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/098—Distributed learning, e.g. federated learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Bioethics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Medical Informatics (AREA)
- Computer Security & Cryptography (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to personal privacy information protection of users, and discloses a federal learning privacy protection method based on an improved diffusion model, which realizes data protection of a client local model and comprises the following steps: step one: each client receives data distributed by a central server; step two: the data is subjected to all T updating iterations on N clients to obtain an updated model; step three: updating the client with the obtained modelEncryption encoding is carried out by adopting the improved depth neural network of the Diffuse Model to obtain a forward encryption ModelStep four: uploading the encrypted model to a central server, sampling and decoding the encrypted model at the central server, aggregating and updating the model to obtain a new round of global model, and transmitting the model to a client to perform updating iteration of the model of the next round of federal learning. The invention is mainly applied to occasions of federal learning communication privacy protection, and simultaneously greatly improves federal learning communication efficiency.
Description
Technical Field
The invention relates to personal privacy information protection of users, in particular to communication privacy protection in the prior federal study, and adopts a latest deep neural network algorithm for improving a diffusion model, thereby improving the protection degree of client privacy of federal study and greatly improving the communication efficiency.
Background
With the increasing increase of current internet of things devices, more and more distributed data starts to appear, and deep learning is successfully applied to intelligent devices, so that data transmission and reception in mobile devices are also becoming more and more intimate, many operators also start to pay attention to a deep learning framework of a user mobile device terminal, and the framework of many mobile terminals also appears in recent years, for example: tensorFlow Lite, pyTorch Mobile, caffe2, NCNN, MNN, etc.
In a distributed data system, data is typically divided into a plurality of small blocks, each of which is stored on a different node. The nodes communicate with each other through a network to coordinate the reading, writing, backup, recovery, etc. of data. Meanwhile, a distributed data system generally provides various data consistency and fault tolerance mechanisms so as to ensure the correctness and reliability of data. Distributed data may be used in a variety of applications, such as internet search engines, large-scale data analysis, distributed storage systems, blockchains, and the like. The method can help enterprises and organizations to better manage and process data, improve the efficiency and quality of data processing, reduce the cost of data storage and maintenance, and improve the expandability and reliability of the system.
At present, federal learning can significantly improve the speed and efficiency of machine learning, and although the storage or transmission of distributed data of mobile devices is threatened and challenged, the demand for distributed machine learning is increasing with the wide application of mobile devices and internet of things technologies. In this case, how to guarantee data privacy and security on the mobile device becomes an important issue. Because data may be hacked or stolen during transmission, revealing the user's private information. Therefore, the traditional federal learning framework has a great vulnerability in privacy protection, and the traditional federal learning specific framework is as follows:
step one: selecting participants: several participants are selected from the data holder who will participate in the training process.
Step two: model initialization: before training can begin, a global model needs to be initialized.
Step three: participant training: each participant trains the model using local data. This may be based on a conventional machine learning algorithm or a deep learning algorithm.
Step four: model aggregation: after a certain round of local training, the participants upload their model parameters to a central server and then aggregate the models. Common aggregation methods include weighted averaging and gradient averaging.
Step five: updating the global model: the global model is updated using the aggregated model parameters.
Step six: repeating the iteration: repeating the above steps until a predetermined convergence condition or number of iterations is reached.
Thus, there are a great deal of research and improvement methods for the problems of both the vulnerability of privacy protection and the communication efficiency in federal learning at present. WANG or the like uses the countermeasure generation network (Generative Adversarial Networks, GAN) to generate the mGAN-AI by multiplexing on a server, judges the authenticity, category and home subscriber of the data, and reconstructs typical data of the participants by using the updated value. ZHU and the like find that malicious attackers can steal the original training data according to the partially updated gradient information. Accordingly, related schemes are proposed that protect gradient confidentiality using techniques based on differential privacy (Differential Privacy, DP), secure Multi-party computing (SMC), homomorphic encryption (Homomorphic Encryption, HE), and the like.
Disclosure of Invention
In the existing privacy protection vulnerability problem and communication efficiency problem of federal learning, the invention aims to adopt the latest improved diffusion model deep neural network algorithm, adopts a Markov-like decision process in the stage of encrypting a data model, and particularly receives data distributed by a central server side; the data is subjected to all T updating iterations at the local client to obtain an updated modelUpdating the client with the resulting model->Encryption coding is carried out by adopting a depth neural network of an improved diffusion model, and a forward encryption model is obtained by adopting a logarithmic maximum likelihood method when the encryption coding is used>Uploading the encrypted model to a central server, sampling and decoding the encrypted model at the central server, aggregating and updating the model to obtain a new round of global model, and transmitting the model to a client to perform updating iteration of the model of the next round of federal learning.
In order to realize the efficient communication and privacy protection of the federal study, the invention comprises the following steps:
step one: each client receives central server data: the origin center server initializes the global model w 0 And broadcasting the global model to N clients, wherein the local updating iteration times of the clients are r times, the global overall training times are T times, and the encryption layer number of the improved diffusion model deep neural network (IDMN) is S.
Step two: the client updates the model data: each client performs r updating iterations locally on the model received from the central server, and for N clients, the central server broadcasts m global models, and for N clients, the local model updated by the central server is
Step three: an improved diffusion model deep neural network (IDMN) encryption model is used: setting the diffusion model layer number of the IDMN as an S layer, and giving a data distribution x 0 ～q(x 0 ) Defining a forward noise process q by adding a variance beta at time s s Gaussian noise of e (0, 1) to generate latency x S -N (0, 1) to x S Then the original diffusion model is as follows:
given a sufficiently large S and a well behaved beta s Scheduling, potential x S Almost isotropic gaussian distribution. Thus, if the exact inverse distribution q (x s-1 |x s ) Can be relative to x S N (0, 1) and run the process in reverse to go from q (x) 0 ) Samples were obtained. Due to q (x s-1 |x s ) Depending on the overall data distribution, it is approximated using a neural network, as follows:
p θ (x s-1 |x s ):＝N(x s-1 ；μ θ (x s ,s),∑ θ (x s ,s)) (3)
the combination of q and p is a variant automatic encoder, so the Variant Lower Bound (VLB) can be written as follows:
L vlb :＝L 0 +L 1 +...+L S-1 +L S (4)
L 0 :＝－logp θ (x 0 |x 1 ) (5)
L s-1 :＝D KL (q(x s-1 |x s ,x 0 )||pθ(x s-1 |x s )) (6)
L S :＝D KL (q(x S |x 0 )||p(x S )) (7)
in addition to L 0 Besides, each term of equation 4 is the KL divergence between two gaussian distributions, and thus can be evaluated in a closed form. To evaluate L of model 0 Broadcasting a central server to 300 clients and calculating p θ (x 0 |x 1 )。
The noise processing defined in equation 2 allows direct input of x 0 Sampling is performed for any step of conditional noise latency. When alpha is s :＝1-β t AndWhen the edge distribution can be written as:
while using bayesian theorem, q (x s-1 |x s ,x 0 ) By usingAndto represent the following definitions therein:
the improved algorithm for improving the privacy protection performance of the federal learning and the communication efficiency of the federal learning by using the maximum likelihood of the IDMN in the invention is specifically described herein, and can be added to x through the deep neural network prediction 0 Is shown below:
in combination with the re-weighted loss function, a new loss function L is defined simple ：
This loss function can be regarded as L vlb Is a re-weighted version of (c). Direct optimization of L by optimizing this re-weighted target ratio vlb The result accuracy of (2) is much higher. In addition, the IDMN changes the diffusion steps compared with the original diffusion model, and the time required by the encryption model is greatly reduced.
Since the first few steps of noise addition in the diffusion model play a decisive role in the lower bound of variation, the learning effect amount Σ is defined here θ (x s S) to improve log likelihood, where v is the model predictionThe calculation method is as follows:
because of the loss function L in the above equation 13 hybrid Is not subjected to sigma θ (x s S), a loss function L is thus defined again hybrid And λ=0.001 is set to reduce L vlb The influence calculation method is as follows:
L hybrid ＝L simple +λL vlb (15)
in addition, a noise time table with better effect is constructed on the basis, and relevant noise parameters are set by using a cosine function under the condition of more flexible optimization of an objective functionThe sum function f(s) is calculated as follows:
step four: uploading the model encrypted by the IDMN to a central server, and firstly, sampling and decoding at the central server side, wherein the specific mode is as follows:
because in step three the variance β is defined by equation 16 s Let the variance beta at the same time s The value of (2) is not more than 0.999, singular points are avoided when diffusion is carried out near s=s, and the variance beta is avoided s The calculation method is as follows:
because the method adopted by the invention improves the communication efficiency of federal learning in a mode of reducing the diffusion model encryption layer number S while ensuring the privacy protection of the federal learning model in the model encryption stage, the method is used for carrying out the feedback sampling decoding stageIn models trained with S diffusion steps, the same sequence of S values (1, 2, … S) will typically be used in the training. However, any sub-sequence a of s values may also be used for sampling. Given training noise schedulingFor a given sequence A, a sampling noise schedule can be obtained>Sampling noise scheduling can then be used>To obtain the corresponding sampling variance->Andthe specific calculation mode is as follows:
after the above-mentioned calculation and inverse sampling decoding, the central server uploads the model w to all clients i And performing aggregation updating, so as to generate a new round of global model at the central server, further issuing the global model, broadcasting the global model to each client, and performing iterative updating of the federal learning data model of the next round, namely repeating the calculation of the steps.
Therefore, the invention shortens the diffusion model encryption layer number from the original S step to the A step through the calculation, and greatly shortens the diffusion layer number no matter the forward encryption model process or the reverse sampling decoding process, which also shows that the privacy protection method for federal learning of the invention ensures the privacy protection of the client and improves the communication efficiency of federal learning.
The present federal learning privacy protection technology mostly only considers the protection of client data in one direction, but lacks the improvement of communication efficiency important to federation learning, so the present invention adopts an improved deep neural network algorithm of IDMN, improves the logarithmic maximum likelihood in noise data by improving the traditional noise and variance adding method, makes relevant improvement on the loss function of the traditional diffusion model, controls the lower bound loss function of variation and the learning method of model training, reduces the steps of model encryption and reverse sampling decryption, thereby improving the communication efficiency of federal learning privacy protection, establishing a federal learning privacy protection model, and preventing the information of the model transmitted in the federal learning network from being acquired by a gradient reverse propagation deducing mode.
Drawings
FIG. 1 is a block diagram of a federal learning privacy protection method based on an improved diffusion model.
FIG. 2 is a federal learning privacy protection method model training flow diagram based on an improved diffusion model.
Fig. 3 is a graph showing accuracy of the FedIDMN algorithm of the present invention after model training under the CIFAR-10 dataset and the MNIST dataset under the condition of independent synchronous distribution (IID) and non-independent identical distribution (no-IID) of the data model, compared with the conventional federal learning algorithm FedAvg algorithm.
Detailed Description
The invention provides an improved federal learning privacy protection method of the diffusion model, and the IDMN algorithm of the diffusion model encryption model has greater advantages than the privacy protection algorithm of the traditional federal learning, not only improves the privacy protection performance on the basis of the traditional federal learning algorithm, but also accelerates the most important communication efficiency in federal learning. In addition, most of traditional federal learning algorithms have too many limitations, the model data privacy is protected by adding noise interference in a gradient back propagation mode, but the model data cannot be restored to the original model data with higher accuracy in the decoding process, the accuracy of the data model in federal learning is also influenced, the communication efficiency of federal learning cannot be improved, meanwhile, the protection of the data model privacy is better provided, and complete consideration is difficult. Therefore, considering various factors in federal learning, the invention combines the influence factors, and provides a federal learning privacy protection method of an improved diffusion model taking an IDMN algorithm as a core.
The invention uses CIFAR-10 data set and MNIST data set when training the model of IDMN algorithm, uses 60000 data to divide into 200 groups of data slices with the size of 300, then divides the data slices for each client, and randomly divides the data slices in consideration of the condition of independent and uniform distribution of data in federal learning, thereby ensuring that the invention is applied to the actual privacy protection scene of federal learning.
The specific implementation steps are as follows:
step one: each client receives central server data: the origin center server initializes the global model w 0 And broadcasting the global model to N clients, wherein the local updating iteration times of the clients are r times, the global overall training times are T times, and the encryption layer number of the improved diffusion model deep neural network (IDMN) is S.
Step two: the client updates the model data: each client performs r updating iterations locally on the model received from the central server, and for N clients, the central server broadcasts m global models, and for N clients, the local model updated by the central server is
Step three: an improved diffusion model deep neural network (IDMN) encryption model is used: setting the diffusion model layer number of the IDMN as an S layer, and giving a data distribution x 0 ～q(x 0 ) Defining a forward noise process q by adding a variance beta at time s s Gaussian noise of e (0, 1) to generate latency x S -N (0, 1) to x S Then the original diffusion model is as follows:
given a sufficiently large S and a well behaved beta s Scheduling, potential x S Almost isotropic gaussian distribution. Thus, if the exact inverse distribution q (x s-1 |x s ) Can be relative to x S N (0, 1) and run the process in reverse to go from q (x) 0 ) Samples were obtained. Due to q (x s-1 |x s ) Depending on the overall data distribution, it is approximated using a neural network, as follows:
p θ (x s-1 |x s ):＝N(x s-1 ；μ θ (x s ,s),∑ θ (x s ,s)) (3)
the combination of q and p is a variant automatic encoder, so the Variant Lower Bound (VLB) can be written as follows:
L vlb :＝L 0 +L 1 +...+L S-1 +L S (4)
L 0 :＝－logp θ (x 0 |x 1 ) (5)
L s-1 :＝D KL (q(x s-1 |x s ,x 0 )||pθ(x s-1 |x s )) (6)
L S :＝D KL (q(x S |x 0 )||p(x S )) (7)
in addition to L 0 Besides, each term of equation 4 is the KL divergence between two gaussian distributions, and thus can be evaluated in a closed form. To evaluate L of model 0 Broadcasting a central server to 300 clients and calculating p θ (x 0 |x 1 )。
The noise processing defined in equation 2 allows direct input of x 0 Sampling is performed for any step of conditional noise latency. When alpha is s :＝1-β t AndWhen the edge distribution can be written as:
while using bayesian theorem, q (x s-1 |x s ,x 0 ) By usingAndto represent the following definitions therein:
the improved algorithm for improving the privacy protection performance of the federal learning and the communication efficiency of the federal learning by using the maximum likelihood of the IDMN in the invention is specifically described herein, and can be added to x through the deep neural network prediction 0 Is shown below:
in combination with the re-weighted loss function, a new loss function L is defined simple ：
This loss function can be regarded as L vlb Is a re-weighted version of (c). Direct optimization of L by optimizing this re-weighted target ratio vlb The result accuracy of (2) is much higher. In addition, the IDMN changes the diffusion steps compared with the original diffusion model, and the time required by the encryption model is greatly reduced.
Since the first few steps of noise addition in the diffusion model play a decisive role in the lower bound of variation, the learning effect amount Σ is defined here θ (x s S) to improve the log likelihood, where v is interpolated as model prediction, the calculation method is as follows:
because of the loss function L in the above equation 13 hybrid Is not subjected to sigma θ (x s S), a loss function L is thus defined again hybrid And λ=0.001 is set to reduce L vlb The influence calculation method is as follows:
L hybrid ＝L simple +λL vlb (15)
in addition, a noise time table with better effect is constructed on the basis, and relevant noise parameters are set by using a cosine function under the condition of more flexible optimization of an objective functionThe sum function f(s) is calculated as follows:
step four: uploading the model encrypted by the IDMN to a central server, and firstly, sampling and decoding at the central server side, wherein the specific mode is as follows:
because in step three the variance β is defined by equation 16 s Let the variance beta at the same time s The value of (2) is not more than 0.999, singular points are avoided when diffusion is carried out near s=s, and the variance beta is avoided s The calculation method is as follows:
because the method adopted by the invention improves the communication efficiency of federal learning in a mode of reducing the diffusion model encryption layer number S while ensuring the privacy protection of the federal learning model in the model encryption stage, the same S value sequence (1, 2, … S) is usually used in training for the model trained by S diffusion steps in the reverse sampling decoding stage. However, any sub-sequence a of s values may also be used for sampling. Given training noise schedulingFor a given sequence A, a sampling noise schedule can be obtained>Sampling noise scheduling can then be used>To obtain the corresponding sampling variance->Andthe specific calculation mode is as follows:
after the above-described calculated upsampling decoding, the centerModel w uploaded by server to all clients i And performing aggregation updating, so as to generate a new round of global model at the central server, further issuing the global model, broadcasting the global model to each client, and performing iterative updating of the federal learning data model of the next round, namely repeating the calculation of the steps.
Claims (2)
1. The federal learning privacy protection method based on the improved diffusion model is characterized by comprising the following steps of: in order to realize the efficient communication and privacy protection of the federal study, the invention comprises the following steps:
step one: each client receives central server data: the origin center server initializes the global model w 0 Broadcasting the global model to N clients, wherein the local updating iteration times of the clients are r times, the global overall training times are T times, and the encryption layer number of an improved diffusion model deep neural network (IDMN) is S;
step two: the client updates the model data: each client performs r updating iterations locally on the model received from the central server, and for N clients, the central server broadcasts m global models, and for N clients, the local model updated by the central server is
Step three: an improved diffusion model deep neural network (IDMN) encryption model is used: setting the diffusion model layer number of the IDMN as an S layer, and giving a data distribution x 0 ～q(x 0 ) Defining a forward noise process q by adding a variance beta at time s s Gaussian noise of e (0, 1) to generate latency x S -N (0, 1) to x S Then the original diffusion model is as follows:
given a sufficiently large S and a well behaved beta s Scheduling, potential x S Is an almost isotropic gaussian distribution, so if the exact inverse distribution q (x s-1 |x s ) Can be relative to x S N (0, 1) and run the process in reverse to go from q (x) 0 ) Samples are obtained, since q (x s-1 |x s ) Depending on the overall data distribution, it is approximated using a neural network, as follows:
p θ (x s-1 |x s ):＝N(x s-1 ；μ θ (x s ,s),∑ θ (x s ,s)) (3)
the combination of q and p is a variant automatic encoder, so the Variant Lower Bound (VLB) can be written as follows:
L vlb :＝L 0 +L 1 +...+L S-1 +L S (4)
L 0 :＝-log p θ (x 0 |x 1 ) (5)
L s-1 :＝D KL (q(x s-1 |x s ,x 0 )||p θ (x s-1 |x s )) (6)
L S :＝D KL (q(x S |x 0 )||p(x S )) (7)
in addition to L 0 Besides, each term of equation 4 is the KL divergence between two Gaussian distributions, and thus can be evaluated in a closed form, in order to evaluate the L of the model 0 Broadcasting a central server to 300 clients and calculating p θ (x 0 |x 1 )；
The noise processing defined in equation 2 allows direct input of x 0 Sampling is performed for any step of conditional noise latency, when alpha s :＝1-β t AndWhen the edge distribution can be written as:
while using bayesian theorem, q (x s-1 |x s ,x 0 ) By usingAnd->To represent the following definitions therein:
the improved algorithm for improving the privacy protection performance of the federal learning and the communication efficiency of the federal learning by using the maximum likelihood of the IDMN in the invention is specifically described herein, and can be added to x through the deep neural network prediction 0 Is shown below:
in combination with the re-weighted loss function, a new loss function L is defined simple ：
This loss function may beTo be regarded as L vlb Directly optimizing L by optimizing this re-weighted target ratio vlb The result accuracy of (2) is much higher, in addition, the IDMN changes the step number of diffusion compared with the original diffusion model, so that the time required by the encryption model is greatly reduced;
since the first few steps of noise addition in the diffusion model play a decisive role in the lower bound of variation, the learning effect amount Σ is defined here θ (x s S) to improve the log likelihood, where v is interpolated as model prediction, the calculation method is as follows:
because of the loss function L in the above equation 13 hybrid Is not subjected to sigma θ (x s S), a loss function L is thus defined again hybrid And λ=0.001 is set to reduce L vlb The influence calculation method is as follows:
L hybrid ＝L simple +λL vlb (15)
in addition, a noise time table with better effect is constructed on the basis, and relevant noise parameters are set by using a cosine function under the condition of more flexible optimization of an objective functionThe sum function f(s) is calculated as follows:
step four: uploading the model encrypted by the IDMN to a central server, and firstly, sampling and decoding at the central server side, wherein the specific mode is as follows:
because in step three the variance β is defined by equation 16 s Let the variance beta at the same time s The value of (2) is not more than 0.999, singular points are avoided when diffusion is carried out near s=s, and the variance beta is avoided s The calculation method is as follows:
because the method adopted by the invention improves the communication efficiency of federal learning in a mode of reducing the diffusion model encryption layer number S while ensuring the privacy protection of the federal learning model in the model encryption stage, the same S value sequence (1, 2, … S) is usually used in training for the model trained by S diffusion steps in the reverse sampling decoding stage, however, any subsequence A of S values can also be used for sampling, and the training noise scheduling is givenFor a given sequence A, a sampling noise schedule can be obtained>Sampling noise scheduling can then be used>To obtain the corresponding sampling variance->And->The specific calculation mode is as follows:
after the above-mentioned calculation and inverse sampling decoding, the central server uploads the model w to all clients i Performing aggregation update to generate a new round of global model at the central server, further issuing the global model, broadcasting the global model to each client, performing iterative update of the federal learning data model of the next round,i.e. the calculation of repeating the above steps.
2. A federal learning privacy protection method based on an improved diffusion model as claimed in claim 1, wherein in the IDMN algorithm of the present invention, step one: each client receives data distributed by a central server;
step two: the data is subjected to all T updating iterations in N clients to obtain an updated model
Step three: updating the client with the obtained modelEncryption coding is carried out by adopting a depth neural network of an improved diffusion model, so that a forward encryption model is obtained>
Step four: uploading the encrypted model to a central server, sampling and decoding the encrypted model at the central server, aggregating and updating the model to obtain a new round of global model, and transmitting the model to a client to perform updating iteration of the model of the next round of federal learning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311205515.0A CN117474118A (en) | 2023-09-18 | 2023-09-18 | Federal learning privacy protection method based on improved diffusion model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311205515.0A CN117474118A (en) | 2023-09-18 | 2023-09-18 | Federal learning privacy protection method based on improved diffusion model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117474118A true CN117474118A (en) | 2024-01-30 |
Family
ID=89622863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311205515.0A Pending CN117474118A (en) | 2023-09-18 | 2023-09-18 | Federal learning privacy protection method based on improved diffusion model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117474118A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117910601A (en) * | 2024-03-20 | 2024-04-19 | 浙江大学滨江研究院 | Personalized federal potential diffusion model learning method and system |
-
2023
- 2023-09-18 CN CN202311205515.0A patent/CN117474118A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117910601A (en) * | 2024-03-20 | 2024-04-19 | 浙江大学滨江研究院 | Personalized federal potential diffusion model learning method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| So et al. | Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning | |
| Balle et al. | Privacy amplification via random check-ins | |
| CN109684855B (en) | Joint deep learning training method based on privacy protection technology | |
| US20210158216A1 (en) | Method and system for federated learning | |
| Liu et al. | Privacy-enhanced federated learning against poisoning adversaries | |
| CN115310121B (en) | Real-time reinforced federal learning data privacy security method based on MePC-F model in Internet of vehicles | |
| Besser et al. | Wiretap code design by neural network autoencoders | |
| CN117474118A (en) | Federal learning privacy protection method based on improved diffusion model | |
| CN111291411B (en) | Safe video anomaly detection system and method based on convolutional neural network | |
| CN115841133A (en) | Method, device and equipment for federated learning and storage medium | |
| US8977855B2 (en) | Secure function evaluation between semi-honest parties | |
| CN113240129A (en) | Multi-type task image analysis-oriented federal learning system | |
| CN115643105B (en) | Federal learning method and device based on homomorphic encryption and depth gradient compression | |
| CN115879152A (en) | Self-adaptive privacy protection method, device and system based on minimum mean square error criterion | |
| Zhang et al. | Safelearning: Enable backdoor detectability in federated learning with secure aggregation | |
| CN115021900A (en) | Method for realizing comprehensive privacy protection of distributed gradient lifting decision tree | |
| CN115481415A (en) | Communication cost optimization method, system, device and medium based on longitudinal federal learning | |
| Talwar | Differential secrecy for distributed data and applications to robust differentially secure vector summation | |
| Gad et al. | Joint Knowledge Distillation and Local Differential Privacy for Communication-Efficient Federated Learning in Heterogeneous Systems | |
| CN116760634B (en) | Data privacy protection method, system, equipment and storage medium | |
| Hu et al. | MASKCRYPT: Federated Learning with Selective Homomorphic Encryption | |
| Dasu et al. | PROV-FL: Privacy-preserving round optimal verifiable federated learning | |
| CN117349685A (en) | Clustering method, system, terminal and medium for communication data | |
| CN117171814A (en) | Federal learning model integrity verification method, system, equipment and medium based on differential privacy | |
| Leemaqz et al. | Corruption-resistant privacy preserving distributed em algorithm for model-based clustering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |