CN114363043A - Asynchronous federated learning method based on verifiable aggregation and differential privacy in peer-to-peer network - Google Patents

Asynchronous federated learning method based on verifiable aggregation and differential privacy in peer-to-peer network Download PDF

Info

Publication number
CN114363043A
CN114363043A CN202111657350.1A CN202111657350A CN114363043A CN 114363043 A CN114363043 A CN 114363043A CN 202111657350 A CN202111657350 A CN 202111657350A CN 114363043 A CN114363043 A CN 114363043A
Authority
CN
China
Prior art keywords
model
client
training
local
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111657350.1A
Other languages
Chinese (zh)
Other versions
CN114363043B (en
Inventor
张磊
高圆圆
姚鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
East China Normal University
Original Assignee
East China Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by East China Normal University filed Critical East China Normal University
Priority to CN202111657350.1A priority Critical patent/CN114363043B/en
Publication of CN114363043A publication Critical patent/CN114363043A/en
Application granted granted Critical
Publication of CN114363043B publication Critical patent/CN114363043B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Landscapes

  • Computer And Data Communications (AREA)

Abstract

The invention discloses an asynchronous federated learning method based on verifiable aggregation and differential privacy in a peer-to-peer network. The federal learning method mainly comprises five stages: the method comprises a system initialization stage, a registration stage, a local model training stage, a model distribution stage and a model aggregation stage. The invention provides a verifiable federal learning scheme based on local data set testing and cosine value detection, aiming at the problems of data privacy protection and model performance in asynchronous federal learning. Before local model updating is carried out, effective model updating is screened out through a model verification scheme, and model updating with poor performance in verification tests is abandoned, so that the performance of the aggregation model is improved. Meanwhile, a privacy protection method combined with a local differential privacy method is designed in the scheme so as to ensure the safety of user data. The invention achieves the design targets of high reliability, high safety and high performance in the asynchronous federal learning scene, and has stronger practical application value.

Description

Asynchronous federated learning method based on verifiable aggregation and differential privacy in peer-to-peer network
Technical Field
The invention relates to the field of information security, in particular to a scheme for privacy protection, model update verification and aggregation of asynchronous federal learning data, which relates to the problems of privacy protection and model quality verification in asynchronous federal learning; and more particularly to an asynchronous federated learning method based on verifiable aggregation and differential privacy in peer-to-peer networks.
Background
With the rapid development of the internet of things technology, a large amount of user application data is generated, and the data often contains privacy data of some users. In order to process and analyze network data of increasingly large volume, a machine learning method has been widely used in various fields. However, due to the requirements of data security, user privacy protection, and regulatory approaches, it has been difficult to collect data and perform calculations by traditional centralized machine learning methods, i.e., by a central server. The european union general data protection regulations state that organizations cannot share sensitive data to third parties for data training, and put higher demands on data resource protection. Meanwhile, the data volume of single equipment or organization is small, and the performance of the model obtained by training is difficult to guarantee.
Federal learning is a recently proposed distributed learning method that breaks the barriers between data sets, enables data owners to train data sets locally, protects privacy of data owners, and implements marginal intelligence. In federal learning, the typical architecture is a client-server model, which includes a server (also called an aggregator) and a set of clients with their own data sets. In the client-server mode, the entire learning process includes multiple rounds of training, each client obtaining a global model from the server, training the model using its local data set, and then sending the updated model to the server. However, this mode presents several challenges. First, the reliability of the server is very important, since the server is the only node performing the aggregation. As a system core in this mode, the learning process must be halted as soon as the server is down for an external attack. Secondly, this model is not suitable for some dynamic applications, such as in an ad hoc network in a vehicle, where the moving nature of the vehicle makes it difficult to maintain continuous and stable communication between the server and the vehicle, and the exit of the client may interrupt the learning process. Third, there are sometimes no trusted third parties that can act as servers.
In order to adapt federated learning to the requirements of dynamic application scenarios, asynchronous federated learning in peer-to-peer network-based environments has been created. This model allows each participant to exchange update models directly, without the assistance of third parties, and to complete aggregation locally at the client, as opposed to a client-server model. Thus, it does not require a trusted third party and training suspension due to individual client or aggregator failures can be avoided. At the same time, the exit of the participant does not interrupt the training process.
However, asynchronous federal learning still needs to address the issues of data privacy and model performance. Recent research has found that various types of attacks, such as gradient analysis, membership inference attack and reconstruction attack, may still be encountered in the manner of exchanging model updates between users participating in federal learning, and semi-honest or malicious users may obtain user privacy data from the exchanged updates. Therefore, a protection mechanism needs to be established in federal learning to ensure the privacy and safety of data in the process of federal learning. For model performance, the accuracy of the aggregated model is affected due to poor quality of the model obtained by local training or the malicious client intentionally issuing a low-quality updated model. A verification method is designed, so that the received updated model can be verified, low-quality updating is avoided, and the accuracy of the final model is effectively improved.
In order to improve the security of federal learning, many privacy protection schemes based on secure multi-party computation and differential privacy protection have been proposed. For secure multiparty computing schemes, secret sharing, homomorphic encryption and other methods are adopted, but the schemes need multiple rounds of communication and have high computing cost. Meanwhile, these privacy protection schemes are either computationally expensive, costly to communicate, or limit the number of clients, or require additional trust assumptions. Also, since low quality update models can easily affect the accuracy of the aggregation model, different approaches have been proposed to improve the performance of federated learning, such as using asynchronous model updates to improve aggregation efficiency, however, they do not consider the problem of low quality model updates.
Disclosure of Invention
In order to solve the problems, the invention aims to provide a privacy protection asynchronous federated learning method based on verifiable aggregation and differential privacy in a peer-to-peer network, so as to solve the problems of data privacy and model precision when asynchronous federated learning is deployed to distributed scenes such as collaborative driving.
The specific technical scheme for realizing the invention is as follows:
an asynchronous federated learning method based on verifiable aggregation and differential privacy in a peer-to-peer network comprises the following steps:
1) a system initialization stage:
a) client database initialization: u for ith client participating in asynchronous federated learning in peer-to-peer network environmentiRepresents, each client uiMaintain their own local data sets D respectivelyi＝{(x1,y1),...,(xn,yn) In which xiIs the characteristic value, y, of the ith data in the client data setiA tag value for the ith data; in the system initialization phase, the client uiFor own local data set DiDividing, wherein the training set is used for model training, and the verification set is used for screening the received model update in the model verification stage;
b) initializing communication parameters: model updates are transmitted by establishing a trusted channel; the CA is a trusted third party mechanism and participates in the construction of a trusted channel; a trusted third party CA generates system parameters necessary for establishing a channel in an initialization stage;
2) a registration stage:
a trusted third party CA firstly generates a group of public and private key pairs, wherein a public key generated by the CA is represented as mpk, a private key is represented as msk, the group of public and private key pairs is represented as (msk, mpk), and a signature scheme sigma is specified, and the CA issues signature certificates for various clients in a federal learning system by using the signature scheme sigma to provide identity authentication of the clients; the signature scheme sigma and a public key mpk in the CA public-private key pair are both disclosed by the CA; each client in the system generates its own public-private key pair, wherein the client uiFor the generated public key
Figure BDA0003446304690000021
For expressing, for, the private key
Figure BDA0003446304690000022
Indicating that the set of public-private key pairs are used
Figure BDA0003446304690000031
Indicating that the public-private key pair corresponds to the signature scheme Σ; client uiWill its public key
Figure BDA0003446304690000032
Sending the member certificate to a trusted third party CA, and generating a member certificate signed by a private key msk for each client by the CA, so that the identity of the client in the system can be authenticated;
3) a local model training stage:
the client side carries out local model training according to the training set segmented in the initialization stage; for generating an initialization model during the first training
Figure BDA0003446304690000033
Show that
Figure BDA0003446304690000034
As an initial model for training, for performing the t-th round modelThe user training model for type training is
Figure BDA0003446304690000035
Setting the maximum training round as T, representing the current training round by T, and using a client uiThe model obtained by performing t-round training is represented as
Figure BDA0003446304690000036
Setting a privacy budget parameter in the tth round of training to epsilontPrivacy budget εtThe noise interference amount added is controlled when the local differential privacy is realized, the noise is added by adopting a Gaussian mechanism, the mean value of noise distribution obedience is 0, and the standard deviation is
Figure BDA0003446304690000037
Is a Gaussian distribution N (0, σ)2) (ii) a In order to resist reasoning attacks from semi-honest clients or malicious clients, a local differential privacy method is introduced to protect private data from being influenced by curious or malicious clients, and the method comprises the following steps: the client is disturbed by adding noise, the data privacy is effectively protected while the model precision is ensured, and the client u for the t-th model training is subjected toiIts local model is
Figure BDA0003446304690000038
Client u sends this local model to other clients before it sends it to themiAccording to the set privacy budget εtControlling the amount of noise added by calculation
Figure BDA0003446304690000039
Adding noise to obtain a model for distribution
Figure BDA00034463046900000310
Local differential privacy is realized;
4) a model distribution stage:
after the client finishes the local model training, entering a model distribution and verification stage; in asynchronous federal learning, the training rounds of each client are allowed to be inconsistent, and the number of models received from surrounding clients is also allowed to be inconsistent;
in the model distribution phase: client uiSelecting m clients with good surrounding communication conditions, and sending models obtained by local training to the selected m clients; the credible security channel ensures the security of the transmitted data, and the model data can not be obtained by an external adversary in the transmission process;
5) a model polymerization stage:
in the peer-to-peer network, the clients update and aggregate the models respectively without setting a central aggregation server, and the client uiFrom the client u received in the training of the t-th round modeljIs updated to
Figure BDA00034463046900000311
The accuracy of the model updating from other clients is verified by each client, and the verified model updating is used for model aggregation to generate a new model;
client uiIn the verification stage, firstly, a precision parameter beta (0) is specified according to the precision requirement of the precision parameter beta<Beta is less than or equal to 1), the reference value of beta is 0.5, and the larger the beta value is, the higher the precision requirement is; then, firstly, performing data set quality verification, and then performing model updating quality verification, wherein the method specifically comprises the following steps:
a) and (3) verifying the quality of the training data set: evaluating the quality of the data sets of other clients according to the similarity between the received model updates from other clients and the local model so as to verify whether the data sets of other clients are attacked by poison; calculating the cosine values of the model update and the local model to calculate the similarity: first, calculate client uiLocal model of
Figure BDA0003446304690000041
Model update with received client
Figure BDA0003446304690000042
The inner product between, i.e.:
Figure BDA0003446304690000043
the cosine values of the local model and the updated model are then calculated by:
Figure BDA0003446304690000044
each client independently selects whether to adopt the received model for updating or not according to the verification result; if the cosine values of the local model and the updated model are lower than the parameter beta, the data set for training the updated model has lower quality and is likely to be attacked by poison;
b) and (3) verifying the updating quality of the model: after the model updating passes the data set quality verification, performing temporary aggregation, testing the precision of the temporary aggregation model by using the test set data, and if the precision of the temporary aggregation model is lower than that of the local model, giving up the low-quality model updating by the client and continuing to use the local model for training; to prevent the server from obtaining the original model update to infer the client's private information, an asynchronous aggregation method is used, where U is { U } for a set of k clients1,u2,...,ukEach of them uiAll maintain their own data set Di，DiIs the whole user data set { [ U ] D of the whole federal studyiA subset of. In asynchronous federated learning, client uiThe method aims to obtain an optimal model M through model training, wherein the model M takes a characteristic value x as an independent variable, and a model parameter w is taken as a function h (w, x) of an independent variable coefficient, namely M is h (w, x); l isj(w) is the loss function of the model with model parameters w for the jth sample, Fi(w) model with model parameters w for client uiData set DiThe optimization objective of the model training is as follows:
Figure BDA0003446304690000045
the training target of each client in asynchronous federated learning is to minimize the loss function of the client, and each client locally uses a gradient descent algorithm to perform model aggregation training; in the invention, the client updates the model by using the data sent by m clients with good communication conditions to obtain the aggregation model
Figure BDA0003446304690000046
The model update is performed using the following algorithm:
Figure BDA0003446304690000051
wherein
Figure BDA0003446304690000052
Is a model update sent by other clients;
the client continuously runs a machine learning algorithm on the own data set based on the model result of the iteration, and trains a new local model by utilizing random gradient descent; repeating the stages 3) to 5) until the maximum number of training iterations T is reached.
Aiming at the characteristics of high computing cost, high communication cost, multiple limiting conditions and inapplicability to peer-to-peer networks commonly existing in the conventional federal learning encryption method, the invention provides a verifiable privacy protection federal learning scheme so as to solve the problems of data privacy and model precision when asynchronous federal learning is deployed to distributed scenes such as cooperative driving and the like. A novel verifiable aggregation method is provided, updated models can be directly exchanged among clients, the quality of received model updates is verified, low-quality models are eliminated, and the model convergence speed is accelerated.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
The federal learning scheme of the present invention will be described in detail below with reference to fig. 1.
1) A system initialization stage:
a) client database initialization: in a point-to-point network, client uiMaintain its own database Di＝{(x1,y1),...,(xn,yn) U clientiThe data in the database is divided to generate a training data set for local model training and a test data set for verifying the accuracy of the received model.
b) Initializing communication parameters: the model updating in the invention is transmitted by establishing a trusted channel, and a trusted third party CA generates system parameters necessary for establishing the channel in an initialization stage.
2) A registration stage:
the first step is as follows: a trusted third party CA generates a public and private key pair (msk, mpk) of the trusted third party and a specified signature scheme sigma according to system parameters generated in an initialization stage, and the CA issues a signature certificate for a client according to the scheme so as to provide identity authentication of the client;
the second step is that: the CA public signature scheme Σ and the public key mpk in its own key pair (msk, mpk);
the third step: each client u in the systemiAll generate own public and private key pair
Figure BDA0003446304690000053
The public and private key pair corresponds to a signature scheme sigma and its public key is assigned
Figure BDA0003446304690000054
Sending to a trusted third party CA, which is used for each client uiAnd generating a member certificate signed by a private key msk of the member certificate so that the identity of a client in the system can be authenticated.
3) A local model training stage:
the first step is as follows: at first training, client (u)i) Initializing a model to
Figure BDA0003446304690000061
Model training is carried out by using training set data, and for a user training model for carrying out the t round model training, the training set data is
Figure BDA0003446304690000062
Setting the learning rate as alpha and the maximum iteration turn T;
the second step is that: disturbing noise added to the model by using a local differential privacy method, and carrying out t-th model training on a user uiLocal model
Figure BDA0003446304690000063
Setting a privacy budget parameter in the tth round of training to epsilontPrivacy budget εtThe noise interference amount added is controlled when the local differential privacy is realized, the noise is added by adopting a Gaussian mechanism, the mean value of noise distribution obedience is 0, and the standard deviation is
Figure BDA0003446304690000064
Is a Gaussian distribution N (0, σ)2) (ii) a According to the set privacy budget εtThe added noise amount is controlled, and the data privacy is effectively protected while the model precision is ensured. The scheme adopts a Gaussian mechanism to calculate
Figure BDA0003446304690000065
Adding noise to the model achieves local differential privacy.
4) A model distribution stage:
in asynchronous federal learning, the training rounds performed by each client may be inconsistent, and the number of models received from surrounding clients may also be inconsistent, so that model distribution may be performed without waiting for all clients in the same round to finish training:
the first step is as follows: client uiSelecting m clients with good surrounding communication conditions;
the second step is that: negotiating with the selected m clients through the identity authentication generated in the registration stage, and establishing a secure channel;
the third step: the local training models are sent to the m clients. The credible security channel ensures the security of the transmission data, and the model data can not be obtained by an external adversary in the transmission process.
5) A model polymerization stage:
in the peer-to-peer network, the client side updates and aggregates the models respectively, a central aggregation server is not arranged, the accuracy of the models from other client sides is verified by the client side, and the verification stage mainly comprises three steps.
The first step is as follows: client uiIn the verification stage, firstly, a precision parameter beta (0) is specified according to the precision requirement of the precision parameter beta<Beta is less than or equal to 1), the reference value of beta is 0.5, and the reference parameter is used as the reference parameter (0) for verifying the accuracy of the data set<Beta is less than or equal to 1), the larger the beta value is, the higher the precision requirement is.
The second step is that: and (3) verifying the quality of the training data set: and evaluating the quality of the training data set according to the direction similarity of the updated model and the local model. First, calculate client uiLocal model of
Figure BDA0003446304690000066
Model update with received client
Figure BDA0003446304690000067
The inner product between, i.e.:
Figure BDA0003446304690000068
the cosine values of the local model and the updated model are then calculated by:
Figure BDA0003446304690000071
and each client independently selects whether to adopt the received model update or not according to the verification result. If the cosine values of the local model and the updated model are less than the precision parameter beta, the updated model is sent
Figure BDA0003446304690000072
The training data set quality of the client is low, and the data set DjMay be subject to a toxic attack.
The third step: and (5) verifying the updating quality of the model. After the model updating passes the data set quality verification, carrying out temporary aggregation, then testing the precision of the temporary aggregation model by using the test set data, and if the precision of the temporary aggregation model is lower than that of the local model
Figure BDA0003446304690000073
The client will abandon this low quality update model. By the two verification methods, the low-quality models can be prevented from being aggregated, so that the performance of the models is effectively improved.
For a set of k clients, U ═ U1,u2,...,ukEach of them uiAll maintain their own data set Di＝{(x1,y1),...,(xn,yn) And then all user data sets for federal learning are { [ U ] Di}. The client side carries out aggregation on the verified models, and the aggregation process comprises two steps.
The first step is as follows: and (4) optimizing the target definition. In asynchronous federated learning, client uiThe method aims to obtain an optimal model M through model training, wherein the model M takes a characteristic value x as an independent variable, a model parameter w is taken as a function h (w, x) of an independent variable coefficient, namely M is h (w, x), and an optimization target of the model is defined as:
Figure BDA0003446304690000074
wherein L isj(w) is the loss function for the jth sample with model parameters w, and the goal of each client is to minimize its own loss function.
The second step is that: each client locally performs model aggregation training using a gradient descent algorithm. Setting the training minimum batch as B, and the gradient calculated by the client in each batch as
Figure BDA0003446304690000075
In the t round, the client will use the m surrounding clientsData, model update was performed using the following algorithm:
Figure BDA0003446304690000076
wherein
Figure BDA0003446304690000077
Is a validated update model generated by other clients.
And repeating the stages 3) to 5) until the training times reach the initially set maximum iteration times T, and finishing the model training.

Claims (1)

1. An asynchronous federated learning method based on verifiable aggregation and differential privacy in peer-to-peer networks, characterized in that the method comprises the following steps:
1) a system initialization stage:
a) client database initialization: u for ith client participating in asynchronous federated learning in peer-to-peer network environmentiRepresents, each client uiMaintain their own local data sets D respectivelyi＝{(x1,y1),...,(xn,yn) In which xiIs the characteristic value, y, of the ith data in the client data setiA tag value for the ith data; in the system initialization phase, the client uiFor own local data set DiDividing, wherein the training set is used for model training, and the verification set is used for screening the received model update in the model verification stage;
b) initializing communication parameters: model updates are transmitted by establishing a trusted channel; the CA is a trusted third party mechanism and participates in the construction of a trusted channel; a trusted third party CA generates system parameters necessary for establishing a channel in an initialization stage;
2) a registration stage:
a trusted third party CA first generates a set of public-private key pairs, wherein a public key generated by the CA is denoted as mpk, a private key is denoted as msk, and the set of public-private key pairs is advanced by (msk, mpk)The row represents and specifies a signature scheme sigma that the CA will use to issue signing certificates for various clients in the Federal learning system to provide client authentication; the signature scheme sigma and a public key mpk in the CA public-private key pair are both disclosed by the CA; each client in the system generates its own public-private key pair, wherein the client uiFor the generated public key
Figure FDA0003446304680000011
For expressing, for, the private key
Figure FDA0003446304680000012
Indicating that the set of public-private key pairs are used
Figure FDA0003446304680000013
Indicating that the public-private key pair corresponds to the signature scheme Σ; client uiWill its public key
Figure FDA0003446304680000014
Sending the member certificate to a trusted third party CA, and generating a member certificate signed by a private key msk for each client by the CA, so that the identity of the client in the system can be authenticated;
3) a local model training stage:
the client side carries out local model training according to the training set segmented in the initialization stage; for generating an initialization model during the first training
Figure FDA0003446304680000015
Show that
Figure FDA0003446304680000016
As an initial model for training, the training model for the user who performs the t-th round of model training is
Figure FDA0003446304680000017
Setting the maximum training round as T, representing the current training round by T, and using a client uiThe model obtained by performing t-round training is represented as
Figure FDA0003446304680000018
Setting a privacy budget parameter in the tth round of training to epsilontPrivacy budget εtThe noise interference amount added is controlled when the local differential privacy is realized, the noise is added by adopting a Gaussian mechanism, the mean value of noise distribution obedience is 0, and the standard deviation is
Figure FDA0003446304680000019
Is a Gaussian distribution N (0, σ)2) (ii) a In order to resist reasoning attacks from semi-honest clients or malicious clients, a local differential privacy method is introduced to protect private data from being influenced by curious or malicious clients, and the method comprises the following steps: the client is disturbed by adding noise, the data privacy is effectively protected while the model precision is ensured, and the client u for the t-th model training is subjected toiThe local model is
Figure FDA0003446304680000021
Client u sends this local model to other clientsiAccording to the set privacy budget εtControlling the amount of noise added by calculation
Figure FDA0003446304680000022
Adding noise to obtain a model for distribution
Figure FDA0003446304680000023
Local differential privacy is realized;
4) a model distribution stage:
after the client finishes the local model training, entering a model distribution and verification stage; in asynchronous federal learning, the training rounds of each client are allowed to be inconsistent, and the number of models received from surrounding clients is also allowed to be inconsistent;
in the model distribution phase: client uiWill select ambient communicationSending the models obtained by local training to the m clients with good conditions; the credible security channel ensures the security of the transmitted data, and the model data can not be obtained by an external adversary in the transmission process;
5) a model polymerization stage:
in the peer-to-peer network, the clients update and aggregate the models respectively without setting a central aggregation server, and the client uiFrom the client u received in the training of the t-th round modeljIs updated to
Figure FDA0003446304680000024
The accuracy of the model updating from other clients is verified by each client, and the verified model updating is used for model aggregation to generate a new model;
client uiIn the verification stage, firstly, a precision parameter beta (0) is specified according to the precision requirement of the precision parameter beta<Beta is less than or equal to 1), the reference value of beta is 0.5, and the larger the beta value is, the higher the precision requirement is; then, firstly, performing data set quality verification, and then performing model updating quality verification, wherein the method specifically comprises the following steps:
a) and (3) verifying the quality of the training data set: evaluating the quality of the data sets of other clients according to the similarity between the received model updates from other clients and the local model so as to verify whether the data sets of other clients are attacked by poison; calculating the cosine values of the model update and the local model to calculate the similarity: first, calculate client uiLocal model of
Figure FDA0003446304680000025
And the received client ujModel update of
Figure FDA0003446304680000026
Inner product μ (u) betweeni,uj) Namely:
Figure FDA0003446304680000027
the local model is then calculated by
Figure FDA0003446304680000028
And updating the model
Figure FDA0003446304680000029
Cosine value of cos (u)i,uj)：
Figure FDA0003446304680000031
Each client independently selects whether to adopt the received model for updating or not according to the verification result; if cosine values cos (u) of the local model and the updated modeli,uj) If the parameter beta is less than the parameter beta, the quality of the data set for training the updated model is low, and the updated model is possibly attacked by poison;
b) and (3) verifying the updating quality of the model: after the model updating passes the data set quality verification, performing temporary aggregation, testing the precision of the temporary aggregation model by using the test set data, and if the precision of the temporary aggregation model is lower than that of the local model, giving up the low-quality model updating by the client and continuing to use the local model for training; to prevent the server from obtaining the original model update to infer the client's private information, an asynchronous aggregation method is used, where U is { U } for a set of k clients1,u2,...,ukEach of them uiAll maintain their own data set Di，DiIs the whole user data set { [ U ] D of the whole federal studyiA subset of { right } points; in asynchronous federated learning, client uiThe method aims to obtain an optimal model M through model training, wherein the model M takes a characteristic value x as an independent variable, and a model parameter w is taken as a function h (w, x) of an independent variable coefficient, namely M is h (w, x); l isj(w) is the loss function of the model with model parameters w for the jth sample, Fi(w) model with model parameters w for client uiData set DiAverage loss ofAnd (4) a loss function, and then the optimization target of model training is:
Figure FDA0003446304680000032
the training target of each client in asynchronous federated learning is to minimize the loss function of the client, and each client locally uses a gradient descent algorithm to perform model aggregation training; the client updates the model by using the data sent by the m clients with good communication conditions to obtain an aggregation model
Figure FDA0003446304680000033
The polymerization method is as follows:
Figure FDA0003446304680000034
through the two verification methods, the low-quality models can be prevented from being aggregated, so that the performance of the models is effectively improved; the client continuously runs a machine learning algorithm on the own data set based on the model result of the iteration, and trains a new local model by utilizing random gradient descent;
repeating the above stages 3) to 5) until the maximum number of iterations T is reached.
CN202111657350.1A 2021-12-30 2021-12-30 Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network Active CN114363043B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111657350.1A CN114363043B (en) 2021-12-30 2021-12-30 Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111657350.1A CN114363043B (en) 2021-12-30 2021-12-30 Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network

Publications (2)

Publication Number Publication Date
CN114363043A true CN114363043A (en) 2022-04-15
CN114363043B CN114363043B (en) 2023-09-08

Family

ID=81105111

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111657350.1A Active CN114363043B (en) 2021-12-30 2021-12-30 Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network

Country Status (1)

Country Link
CN (1) CN114363043B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115828302A (en) * 2022-12-20 2023-03-21 华北电力大学 Credible privacy calculation-based microgrid grid-connected control privacy protection method
CN116720594A (en) * 2023-08-09 2023-09-08 中国科学技术大学 Decentralized hierarchical federal learning method
CN117436078A (en) * 2023-12-18 2024-01-23 烟台大学 Bidirectional model poisoning detection method and system in federal learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10873456B1 (en) * 2019-05-07 2020-12-22 LedgerDomain, LLC Neural network classifiers for block chain data structures
CN113010305A (en) * 2021-02-08 2021-06-22 北京邮电大学 Federal learning system deployed in edge computing network and learning method thereof
CN113407963A (en) * 2021-06-17 2021-09-17 北京工业大学 Federal learning gradient safety aggregation method based on SIGNSGD
CN113434873A (en) * 2021-06-01 2021-09-24 内蒙古大学 Federal learning privacy protection method based on homomorphic encryption

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10873456B1 (en) * 2019-05-07 2020-12-22 LedgerDomain, LLC Neural network classifiers for block chain data structures
CN113010305A (en) * 2021-02-08 2021-06-22 北京邮电大学 Federal learning system deployed in edge computing network and learning method thereof
CN113434873A (en) * 2021-06-01 2021-09-24 内蒙古大学 Federal learning privacy protection method based on homomorphic encryption
CN113407963A (en) * 2021-06-17 2021-09-17 北京工业大学 Federal learning gradient safety aggregation method based on SIGNSGD

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
周俊;方国英;吴楠;: "联邦学习安全与隐私保护研究综述", 西华大学学报(自然科学版), no. 04 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115828302A (en) * 2022-12-20 2023-03-21 华北电力大学 Credible privacy calculation-based microgrid grid-connected control privacy protection method
CN115828302B (en) * 2022-12-20 2023-07-07 华北电力大学 Micro-grid-connected control privacy protection method based on trusted privacy calculation
CN116720594A (en) * 2023-08-09 2023-09-08 中国科学技术大学 Decentralized hierarchical federal learning method
CN116720594B (en) * 2023-08-09 2023-11-28 中国科学技术大学 Decentralized hierarchical federal learning method
CN117436078A (en) * 2023-12-18 2024-01-23 烟台大学 Bidirectional model poisoning detection method and system in federal learning
CN117436078B (en) * 2023-12-18 2024-03-12 烟台大学 Bidirectional model poisoning detection method and system in federal learning

Also Published As

Publication number Publication date
CN114363043B (en) 2023-09-08

Similar Documents

Publication Publication Date Title
CN114363043B (en) Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network
CN112714106B (en) Block chain-based federal learning casual vehicle carrying attack defense method
CN112749392B (en) Method and system for detecting abnormal nodes in federated learning
Chen et al. Privacy-preserving image multi-classification deep learning model in robot system of industrial IoT
Yang et al. A practical cross-device federated learning framework over 5g networks
CN113300828B (en) Distributed differential privacy aggregation method
Li et al. Efficient privacy-preserving federated learning with unreliable users
CN114679332A (en) APT detection method of distributed system
CN111581648B (en) Method of federal learning to preserve privacy in irregular users
Zhang et al. Safelearning: Enable backdoor detectability in federated learning with secure aggregation
CN116049897A (en) Verifiable privacy protection federal learning method based on linear homomorphic hash and signcryption
CN115238172A (en) Federal recommendation method based on generation of countermeasure network and social graph attention network
Zhou et al. Securing federated learning enabled NWDAF architecture with partial homomorphic encryption
Yang et al. Efficient and secure federated learning with verifiable weighted average aggregation
Sun et al. Privacy-preserving vertical federated logistic regression without trusted third-party coordinator
Gao et al. Privacy-Preserving and Reliable Decentralized Federated Learning
Smahi et al. BV-ICVs: A privacy-preserving and verifiable federated learning framework for V2X environments using blockchain and zkSNARKs
Sun et al. Fed-DFE: A Decentralized Function Encryption-Based Privacy-Preserving Scheme for Federated Learning.
Li et al. Catfl: Certificateless authentication-based trustworthy federated learning for 6g semantic communications
Zhang et al. Visual object detection for privacy-preserving federated learning
CN117216788A (en) Video scene identification method based on federal learning privacy protection of block chain
CN116340986A (en) Block chain-based privacy protection method and system for resisting federal learning gradient attack
Feng et al. Secure distributed outsourcing of large-scale linear systems
Kong et al. Information encryption transmission method of automobile communication network based on neural network
Liu et al. A Comprehensive Privacy-Preserving Federated Learning Scheme with Secure Authentication and Aggregation for Internet of Medical Things

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant