CN114978533A - Verifiable security aggregation method based on weighted layered asynchronous federated learning - Google Patents
Verifiable security aggregation method based on weighted layered asynchronous federated learning Download PDFInfo
- Publication number
- CN114978533A CN114978533A CN202210519513.8A CN202210519513A CN114978533A CN 114978533 A CN114978533 A CN 114978533A CN 202210519513 A CN202210519513 A CN 202210519513A CN 114978533 A CN114978533 A CN 114978533A
- Authority
- CN
- China
- Prior art keywords
- client
- server
- parameters
- updating
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Computer And Data Communications (AREA)
- Storage Device Security (AREA)
Abstract
The invention belongs to the technical field of safe federal learning and discloses a verifiable safety polymerization method based on weighted layered asynchronous federal learning, which mainly comprises the following steps: 1) key agreement; 2) the client executes; 3) and (4) server execution. In order to improve the safety and efficiency of federal learning, the invention provides a verifiable safety aggregation method of weighted layered asynchronous federal learning, and a server can distribute corresponding aggregation weights for each round of clients participating in updating, thereby improving the efficiency of a model; the server can still recover correct aggregation parameters from the aggregated ciphertext under the condition that plaintext parameters of other legal clients cannot be obtained; the server can know whether the parameter to be aggregated of each client is maliciously tampered or not only by generating and verifying a short signature, so that the purposes of improving the federal learning efficiency and ensuring the federal learning safety can be achieved.
Description
Technical Field
The invention belongs to the technical field of safe federal learning, and particularly relates to a verifiable safe polymerization method based on weighted layered asynchronous federal learning.
Background
With the development trend of a new era of mass data, machine learning technology is applied to various industries at an unprecedented speed. However, the problem of data islanding makes the development of traditional machine learning reach a bottleneck, which not only severely restricts the application of artificial intelligence technology in enterprises, but also causes the waste of precious data resources. In order to effectively solve the problem of data resource waste caused by data isolated island, the federal learning technology is released, and a new hope is brought to the development of information technology.
Federated learning obtains a central model on a global server by aggregating locally trained models on local clients, but the communication overhead between clients and servers is large. In 2020, Chen et al provides an efficient hierarchical asynchronous federated learning, and through the strategies of hierarchical asynchronous learning on a client and time weighted aggregation on a server, the high precision of a model is kept, meanwhile, the communication overhead between the client and the server is reduced, and meanwhile, the difficulty of privacy protection of the hierarchical asynchronous federated learning is increased. The mode of using the aggregation model parameters in federal learning protects the data privacy of a local client to a certain extent, but still has the security problems of reasoning attack and the like.
Some clients in the federal learning system frequently participate in model updating (diligent clients), some clients occasionally participate in model updating (lazy clients), and it is not reasonable that the server only weights the importance of each local model according to the number of data points of training data on the clients. In the federal learning parameter security aggregation process, not only the privacy of the updated parameters but also whether the parameters are maliciously tampered by a third party are considered, and the maliciously tampered updated parameters can cause the aggregation result of the server to deviate from the correct value.
In order to solve the above problems, the present application proposes a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning.
Disclosure of Invention
The invention aims to provide a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning, so as to solve the problems in the background technology.
In order to achieve the above purpose, the invention provides the following technical scheme: a verifiable safety aggregation method based on weighted layered asynchronous federal learning, wherein a deep neural network is widely adopted as a local model in the federal learning, and the safety aggregation method mainly comprises the following steps:
1) and (3) key agreement: the client and the server mutually negotiate respective secret keys for encryption, decryption and signature verification;
2) the client executes: the local client independently updates to obtain an updating parameter, then encrypts and signs the updating parameter by using a private key of the client, and sends the encrypted updating parameter and the signature to the server;
3) service period execution: the central server firstly aggregates the signatures of all the clients to generate a short signature and carries out verification, and then aggregates all the ciphertexts after the verification is passed, and recovers the correct aggregation parameters.
Preferably, for the main characteristics of the deep neural network, the number of parameters in the shallow layer is relatively less and is more critical to the performance of the central model, and accordingly, the update frequency of the shallow layer should be higher than that of the deep layer; therefore, the parameters at the shallow layer and the deep layer in the model can be updated asynchronously, so that the data volume sent between the server and the client is reduced, and the updating mode is called layered asynchronous updating.
Preferably, shallow learning of the deep neural network is applicable to general features of different tasks and data sets, which means that a relatively small part of parameters in the deep neural network, i.e. parameters at a shallow layer, represent general features of different data sets; deep learning of deep neural networks is associated with specific features of a particular data set, and a large number of parameters focus on the learned features of the specific data.
Preferably, when aggregating client training parameters, the server in federal learning not only weights the importance of each local model according to the number of data points of training data on corresponding clients, but also allocates appropriate weight values q to different clients participating in training in each round k (ii) a Because some clients in the system frequently participate in model updating, namely diligent clients, and some clients occasionally participate in model updating, namely lazy clients, the server adopts a time-weighted aggregation strategy more reasonably.
Preferably, in the federal learning system, even if the client does not directly upload the data of the user, an attacker can still indirectly deduce the characteristics of the client tag data and the member information of the data set through the model parameters uploaded by the client; the exposed model parameters are trained by the local data set of the client, namely the local data of the client is coded into the model parameters; however, if an attacker can construct a corresponding decoder according to the exposed model parameters, the attacker can reversely infer the private data local to the client according to the model parameters of the client, and this attack mode is called inference attack.
Preferably, each client k chooses its own secret value a k,0 And only knows the client k according to the secret value a k,0 Construct a polynomial S k (x) And substituting the ID of the client into a polynomial S k (x) And sent to the server, which then uses the secret value a k,0 And masking the update parameters and sending the update parameters to the server.
Preferably, client K generates a vector of K +1 dimensionsThe masked updated result is thenPut on the first component, and set a '1' on the k +1 th component, the other components are '0', that is:then the client k signs the covering result to obtain
Preferably, the server receives the masked results of the K clientsAnd a vector of K +1 dimensionsAnd signature information σ k . The server adds a corresponding weight value q to each client k k And aggregate into a short signatureThen for short signature sigma Agg Verifying and judging bilinear pairs E 1 :e(σ Agg Beta) is equal to bilinear pair E 2 :
Preferably, the server assigns a weight value q to each client k k Added to the masked resultAnd polymerized to obtainThe server uses the negotiated total secret key S to cover and offset the aggregation result to obtain an offset resultThe server can calculate the result of the security aggregation of the central model according to the total data point N
Preferably, the specific steps of 1) are:
A. the client k selects the secret value a according to itself k,0 And the number of points n of the local data point k Respectively construct h-1 polynomialAnd
B. the client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server, this time the server is to S k (x) Substituting the weighted value q corresponding to the client k k ,N k (x) Keeping the same, then the server arranges the polynomial with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k ;
C. S for server to pass k clients k And N k Recovering the total secret S and the total data point N by a Lagrange interpolation method, and selecting G 1 、G 2 And G T All are cyclic groups of prime number p, establishing bilinear mapping e: G 1 ×G 2 →G T ,Is G 2 →G 1 Is capable of efficiently calculating isomorphism and generating bilinear groupSelect one generator β ← G 2 \ {1 }; each client shares a private keyAnd calculating u ═ beta γ (ii) a Select a hash function H: {0,1} * ×{0,1} * →G 1 (ii) a Public key PK ═ (G, H, β, u) is used for server authentication signature, and private key SK ═ γ is used forA signature is generated at the client.
Preferably, the specific steps of 2) are:
client k receives the latest ω from the server t,g ,ω t,s Flag is used for updating in the current round;
client-side first-to-shallow depth neural network parametersIs updated to obtainThen to the updated resultIs masked by secret parameters to obtainThen to mask the resultSignature derivation σ k,g ;
Finally, the result will be maskedAnd a signature σ k,g And sending the data to a server. If flag is 'yes', then the deep depth neural network parameters are processedUpdating, masking and signing are carried out in the same way; if flag is 'No', the process is directly carried out.
Preferably, the specific steps of 3) are:
A. server initialization parameter omega 0 And T, determining that K clients exist in the system, and acquiring a public key PK (G, H, beta, u); the server randomly selects a set m of participants and then enters model updating;
B. and (4) updating the current update round module to Rlp, if the result belongs to the Set, marking the flag as ' yes ', otherwise, marking the flag as ' yesIf 'NO', if 'yes', the server will update newlyAnd flag is sent to client k, otherwise updatedAnd flag is sent to the client k;
C. each client k locally and independently executes model parameter updating, covering and signing and sends the model parameter updating, covering and signing to the server, and the server assigns the current turn t to the client kIf flag is 'yes', the server reassigns the current turn t to
D. When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k (ii) a If the flag is 'yes', the server firstly aggregates deep signatures of all clients k and verifies and adds the weight value q k After the signature passes the verification, carrying out safe aggregation on the deep updating parameters, otherwise, quitting;
E. and finally, uniformly aggregating shallow signatures of all clients k by the server, verifying and adding weight values q k And after the signature passes the verification, safely aggregating the shallow layer updating parameters, otherwise, quitting, and updating the central model parameters by the server according to the aggregation result, and entering the next round of model updating until the model converges.
The invention has the following beneficial effects:
1. in order to improve the safety and efficiency of federal learning, the invention provides a verifiable safety aggregation method of weighted layered asynchronous federal learning, and a server can distribute corresponding aggregation weights for each round of clients participating in updating, thereby improving the efficiency of a model; the server can still recover correct aggregation parameters from the aggregated ciphertext under the condition that plaintext parameters of other legal clients cannot be obtained; the server can know whether the parameter to be aggregated of each client is maliciously tampered or not only by generating and verifying one short signature, so that the purposes of improving the federal learning efficiency and ensuring the federal learning safety can be achieved.
Drawings
FIG. 1 is a flow chart of a client implementation of the present invention;
FIG. 2 is a flow chart of a server implementation of the present invention;
FIG. 3 is a flow chart of the client local update of the present invention;
FIG. 4 is a general diagram of the model of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 to 4, the present invention provides a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning, in the key agreement part:
client k construction polynomialAnd polynomialWherein x k Is ID of client K, h is threshold value and h is less than or equal to K, a k,0 Is a secret value, n k Points are counted for local data.
The client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server. Server pair S at this time k (x) Substituting the weighted value q corresponding to the client k k ,N k (x) Is maintained atAnd (6) changing. Then, the server arranges the polynomials with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k . The server can share S the secret by k clients k And data point quanta N k And recovering the total secret S and the total data point N by a Lagrange interpolation method.
Get G 1 、G 2 And G T Is a cyclic group with a prime number p, satisfying a bilinear mapping e: G 1 ×G 2 →G T ,g 1 And g 2 Are respectively group G 1 And G 2 The generation element(s) of (a),is G 1 To G 2 Can be calculated isomorphically and
3) Select a hash function H: {0,1} * ×{0,1} * →G 1 ;
4) The output p, public key PK ═ (G, H, β, u) and private key SK ═ γ.
The private key SK is used for signing the masking result of the local update parameters by each client k, and the public key PK is used for verifying the aggregation signature by the server.
In order to further explain the technical scheme content of the invention, the specific implementation mainly comprises three processes: a key agreement process, a client execution process, and a server execution process. The implementation steps of each process are as follows:
first, key negotiation
Let gf (p) be a finite field, where p is a large prime number. There are a total of K clients in the system, each client K selecting its own secret a k,0 And number of local data points n k And randomly selecting a secret parameter a k,j Where K1, 1., K and j 1., h-1, secret a k,0 And a random parameter a k,j Are random numbers uniformly chosen over gf (p). Construction of h-1 degree polynomialAndwherein x k Is the ID of the client K, h is a threshold value and h is less than or equal to K.
The client k constructs a polynomial S by itself k (x) And N k (x) Sending to the server, wherein x is e { x 1 ,...,x K Denotes IDs of all clients.
The server receives S from the client k k (x) And N k (x) At this time, the server pair S k (x) Substituting the weighted value q corresponding to the client k k 。
The server arranges the polynomials with the same client ID to obtain S (x) k ) And N (x) k ) In which S (x) k ) And N (x) k ) Denotes the client ID as x k The polynomial of (2) collates the results.
Then, the server pairs S (x) respectively k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k 。
Secret share S of server through k clients k And data point quanta N k The lagrange interpolation method can recover the total secret S and the total data point N, namely: lagrangian parameter for client k
Signature setting:
let G 1 、G 2 And G T Is a cyclic group with a prime number p, satisfying a bilinear mapping e: G 1 ×G 2 →G T ,g 1 And g 2 Are respectively group G 1 And G 2 The generation element of (a) is generated,is G 1 To G 2 Can be calculated isomorphically and
3) Select a hash function H: {0,1} * ×{0,1} * →G 1 ;
4) The output p, public key PK ═ (G, H, β, u) and private key SK ═ γ.
The private key SK is used for each client k to sign the masking result of the local update parameters, and the public key PK is used for the server to verify the aggregated signature.
Second, the client executes
With reference to fig. 1, the client specifically executes the following steps:
1) client k randomly selects mask parameter a k,0 And gamma is used as the private key of the signature, where gamma is one private signature key shared between each client and known only to the client.
2) Client k receives the latest ω from the server t,g ,ω t,s And flag, which is used for updating the current round.
3) Client k first-to-shallow DNN weight parameterPerforming local update to obtain the updated resultThe updating process is as shown in figure 3.
4) Client k pairsUpdating resultsSecret parameter covering is carried out to ensure the safety of shallow layer updating parameters; then the masked result is comparedSigning is carried out, and the result is maskedAnd a signature σ k,g And sending the data to a server.
5) If flag is marked as 'yes', client k further performs weighting on deep DNN (digital DNN)Executing local update, the update process is the same as the step 3), and obtaining the update result
6) Client k pair update resultsSecret parameter masking is carried out to ensure the safety of deep updating parameters; then the masked result is processedCarry out signature and mask the resultAnd a signature σ k.s And sending the data to a server.
7) And if the flag is marked as 'no', directly exiting, and finishing the execution of the client.
Parameter masking and signature process:
a) client k uniformly selects random number a on GF (p) k,0 The method is used for carrying out secret parameter masking on the updated result, and firstly carrying out secret parameter masking on the updated result of the shallow DNNNamely:
b) the client K generates a vector with K +1 dimensionsThe masked updated result is thenPut on the first component, and set a '1' on the k +1 th bit, and the other bits are '0', that is:
d) updating result of client k for covering secret parameterAnd the generated K + 1-dimensional vectorAnd a signature σ k,g And sending the data to a server.
The secret parameter masking and signing process for the deep DNN update results is the same as for the shallow DNN. In the same way, the client k can obtain a deep DNN updating result with the parameters coveredAnd the generated K + 1-dimensional vectorAnd a signature σ k,s And all the data are sent to the server, and the execution of the client ends at the moment.
Third, the server executes
With reference to fig. 2, the server specifically executes the following steps:
1) the server first pair the parameter omega 0 And T is initialized. Wherein: omega 0 And converging the DNN model through a T-round model as an initial weight parameter of the DNN model.
2) And selecting a client set, and determining that K clients exist in the whole system.
3) The server obtains the public key PK ═ (G, H, β, u) for verification of subsequent aggregated signatures.
4) Randomly selecting a set m of participants, wherein m represents the client set after learning is exited actively or passively in all the client sets K, and h is less than or equal to | m |. If m < h, quitting learning. Where h is the threshold value for secret sharing.
5) And (8) performing digital analogy on the current round Rlp, if the result belongs to the Set, marking the flag as 'yes', and otherwise, marking the flag as 'no'. Rlp are groupings of update round numbers, Set specifies a Set of deep DNN updates in each group of update round numbers.
6) If flag is 'yes', the server will update newlyAnd the flag is sent to the client k; if the flag is 'NO', the server updates the latest dataAnd flag is sent to client k.
7) Then each client k independently executes model parameter updating locally, carries out secret parameter masking and signing on the updating result, and masks the resultAnd a signature σ k And sending the data to a server.
8) The server assigns the current round t to the client kIf flag is 'yes',the server reassigns the current round t toDenotes tsp k Is updated as client k participates in its turn. If the client k does not participate in the turn, tsp k The value of (c) is the round of the previous round, i.e., no update has occurred.
9) When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k . The client k not belonging to the m set selects the masking result of the previous round and adds the weight valueThe closer the client participating in the update is to the current round, the larger the weight value owned, and conversely, the smaller the weight value owned.
10) If the flag is 'yes', the server firstly aggregates deep signatures of all the client sides k and verifies the added weight value q k The latter signature; if E 1,s =E 2,s And if not, the deep updating parameters are safely aggregated, and otherwise, the operation is quitted.
11) Since the shallow DNN parameters need to participate in updating in each round, the server uniformly aggregates shallow signatures of all clients k and verifies the added weight value q k The latter signature; if E 1,g =E 2,g And if not, the shallow updated parameters are safely aggregated, otherwise, the operation is quitted.
12) And updating the central model parameters according to the aggregation result, and entering the next round of model updating until the model converges. Signature verification and masking cancellation:
a) the server receives the masking results of the K clientsAnd a K +1 dimensional vectorAnd signature information (sigma) k,g ,σ k,s ) Which isK is 1.
b) If flag is 'yes', the signatures of the deep DNN are firstly aggregated. In the aggregation process, the weight value q of each client k is weighted k Added to the signature σ k,s And polymerized to obtain
c) Then, the aggregate signature is verified, and whether the bilinear pairs are equal is judged, namely
d) If E 1,s =E 2,s If the verification is passed, the weight value q of each client k is determined k Added to the masked resultAnd carrying out polymerization to obtainNamely:if E 1,s ≠E 2,s And then directly quit.
e) The server then uses the client secret share S k The recovered total secret S is used for masking the weighted resultCarrying out cancellation to obtain a cancellation result m Agg,s I.e. weighted plaintext aggregation results.
f) The server can obtain the safety aggregation result omega with the weighted deep DNN parameters t+1,s Where N is the total number of data points recovered by the server.
Then, the signature σ of the shallow DNN is further applied k,g And masking the resultsPerforming aggregate verification and mask cancellation in the same manner as deep DNN. If E 1,g =E 2,g That is, the server can obtain the security aggregation result ω of the weighted shallow DNN after verification t+1,g 。
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (12)
1. A verifiable safety aggregation method based on weighted layered asynchronous federal learning, wherein a deep neural network is widely adopted as a local model in the federal learning, and the method is characterized in that: the safe polymerization method mainly comprises the following steps:
1) and (3) key agreement: mutually negotiating respective secret keys between the client and the server for encryption, decryption and signature verification;
2) the client executes: the local client independently updates to obtain an updating parameter, then encrypts and signs the updating parameter by using a private key of the client, and sends the encrypted updating parameter and the signature to the server;
3) the service period executes: the central server firstly aggregates the signatures of all the clients to generate a short signature and carries out verification, and then aggregates all the ciphertexts after the verification is passed, and recovers the correct aggregation parameters.
2. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: for the main features of the deep neural network, the number of parameters in a shallow layer is relatively less and more critical to the performance of the central model, and accordingly, the update frequency of the shallow layer should be higher than that of the deep layer; therefore, the parameters at the shallow layer and the deep layer in the model can be updated asynchronously, so that the data volume sent between the server and the client is reduced, and the updating mode is called layered asynchronous updating.
3. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: shallow learning of the deep neural network is suitable for general features of different tasks and data sets, which means that a relatively small part of parameters in the deep neural network, namely parameters at a shallow layer, represent general features of different data sets; deep learning of deep neural networks is associated with specific features of a particular data set, and a large number of parameters focus on the learned features of the specific data.
4. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: when the server aggregates the client training parameters in the federal learning, the importance of each local model is weighted according to the number of data points of training data on the corresponding client, and the server distributes proper weight values q to different clients participating in training in each round k (ii) a Since some of the systemsThe client frequently participates in model updating, namely a diligence client, and some clients occasionally participate in model updating, namely a lazy client, so that the server is more reasonable in adopting a time weighted aggregation strategy.
5. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: in the federal learning system, even if a client does not directly upload user data, an attacker can still indirectly deduce the characteristics of client label data and member information of a data set through model parameters uploaded by the client; the exposed model parameters are trained by the local data set of the client, namely the local data of the client is coded into the model parameters; however, if an attacker can construct a corresponding decoder according to the exposed model parameters, the attacker can reversely infer the private data local to the client according to the model parameters of the client, and this attack mode is called inference attack.
6. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 5, characterized in that: each client k selects its secret value a k,0 And only knows the client k according to the secret value a k,0 Constructing a polynomial S k (x) And substituting the ID of the client into a polynomial S k (x) And sent to the server, which then uses the secret value a k,0 And masking the update parameters and sending the update parameters to the server.
7. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the client K generates a vector with K +1 dimensionsThe masked updated result is thenPut on the first component, and set a '1' on the k +1 th component, the other components are '0', that is:then the client k signs the covering result to obtain
8. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the server receives the masking results of the K clientsAnd a vector of K +1 dimensionsAnd signature information σ k The server adds a corresponding weight value q to each client k k And aggregate into a short signatureThen for short signature sigma Agg Verifying and judging bilinear pairs E 1 :e(σ Agg Beta) is equal to a bilinear pair
9. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the server weights q of each client k k Added to the masked resultAnd carrying out polymerization to obtainThe server uses the negotiated total secret key S to cover and offset the aggregation result to obtain an offset resultThe server can calculate the result of the security aggregation of the central model according to the total data point N
10. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that the specific steps of 1) are:
A. the client k selects the secret value a according to the client k k,0 And the number of points n of the local data point k Respectively construct h-1 polynomialAnd
B. the client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server, this time the server is to S k (x) Substituting the weighted value q corresponding to the client k k ,N k (x) Keeping the same, then the server arranges the polynomial with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k ;
C. S for server to pass k clients k And N k Recovering the total secret S and the total data point N by a Lagrange interpolation method, and selecting G 1 、G 2 And G T Are all cyclic groups of prime order p, constructedVertical bilinear map e G 1 ×G 2 →G T ,Is G 2 →G 1 Is capable of efficiently calculating isomorphism and generating bilinear groupSelect one generator β ← G 2 \ {1 }; each client shares a private keyAnd calculating u ═ beta γ (ii) a Select a hash function H: {0,1} * ×{0,1} * →G 1 (ii) a The public key PK ═ (G, H, β, u) is used for server verification of the signature, and the private key SK ═ γ is used for client generation of the signature.
11. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that, the specific steps of 2) are:
client k receives the latest ω from the server t,g ,ω t,s Flag is used for updating in the current round;
client-side first-to-shallow depth neural network parametersIs updated to obtainThen to the updated resultCarrying out secret parameter masking to obtainThen to mask the resultSignature derivation σ k,g ;
12. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that, the specific steps of 3) are:
A. server initialization parameter omega 0 And T, determining that K clients exist in the system, and acquiring a public key PK (G, H, beta, u); the server randomly selects a set m of participants and then enters model updating;
B. updating Rlp the round of module currently, if the result belongs to the Set, marking flag as 'yes', otherwise marking flag as 'no'; if flag is 'yes', the server will update newly And flag is sent to client k, otherwise updated newlyAnd the flag is sent to the client k;
C. each client k locally and independently executes model parameter updating, covering and signing and sends the model parameter updating, covering and signing to the server, and the server assigns the current turn t to the client kIf flag is 'yes', the server reassigns the current turn t to
D. When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k (ii) a If the flag is 'yes', the server firstly aggregates deep signatures of all clients k and verifies and adds the weight value q k After the signature passes the verification, carrying out safe aggregation on the deep updating parameters, otherwise, quitting;
E. and finally, uniformly aggregating shallow signatures of all clients k by the server, verifying and adding weight values q k And after the signature passes the verification, safely aggregating the shallow layer updating parameters, otherwise, quitting, and updating the central model parameters by the server according to the aggregation result, and entering the next round of model updating until the model converges.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210519513.8A CN114978533B (en) | 2022-05-12 | 2022-05-12 | Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210519513.8A CN114978533B (en) | 2022-05-12 | 2022-05-12 | Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114978533A true CN114978533A (en) | 2022-08-30 |
CN114978533B CN114978533B (en) | 2023-06-30 |
Family
ID=82984175
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210519513.8A Active CN114978533B (en) | 2022-05-12 | 2022-05-12 | Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114978533B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116052832A (en) * | 2023-04-03 | 2023-05-02 | 青岛市妇女儿童医院(青岛市妇幼保健院、青岛市残疾儿童医疗康复中心、青岛市新生儿疾病筛查中心) | Tamper-proof transmission method based on medical information |
TWI818708B (en) * | 2022-09-02 | 2023-10-11 | 英業達股份有限公司 | Method for verifying model update |
CN117811722A (en) * | 2024-03-01 | 2024-04-02 | 山东云海国创云计算装备产业创新中心有限公司 | Global parameter model construction method, secret key generation method, device and server |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112149160A (en) * | 2020-08-28 | 2020-12-29 | 山东大学 | Homomorphic pseudo-random number-based federated learning privacy protection method and system |
CN113221105A (en) * | 2021-06-07 | 2021-08-06 | 南开大学 | Robustness federated learning algorithm based on partial parameter aggregation |
-
2022
- 2022-05-12 CN CN202210519513.8A patent/CN114978533B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112149160A (en) * | 2020-08-28 | 2020-12-29 | 山东大学 | Homomorphic pseudo-random number-based federated learning privacy protection method and system |
CN113221105A (en) * | 2021-06-07 | 2021-08-06 | 南开大学 | Robustness federated learning algorithm based on partial parameter aggregation |
Non-Patent Citations (2)
Title |
---|
FEIYUE WANG, ETC.: "Federated data: Toward new generation of credible and trustable artificial intelligence", 《IEEE》 * |
成艺: "联合学习环境下保护隐私的数据聚合技术研究", 《中国优秀硕士学位论文全文数据库(电子期刊)》 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI818708B (en) * | 2022-09-02 | 2023-10-11 | 英業達股份有限公司 | Method for verifying model update |
CN116052832A (en) * | 2023-04-03 | 2023-05-02 | 青岛市妇女儿童医院(青岛市妇幼保健院、青岛市残疾儿童医疗康复中心、青岛市新生儿疾病筛查中心) | Tamper-proof transmission method based on medical information |
CN117811722A (en) * | 2024-03-01 | 2024-04-02 | 山东云海国创云计算装备产业创新中心有限公司 | Global parameter model construction method, secret key generation method, device and server |
CN117811722B (en) * | 2024-03-01 | 2024-05-24 | 山东云海国创云计算装备产业创新中心有限公司 | Global parameter model construction method, secret key generation method, device and server |
Also Published As
Publication number | Publication date |
---|---|
CN114978533B (en) | 2023-06-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Xu et al. | VerifyNet: Secure and verifiable federated learning | |
Hao et al. | Efficient and privacy-enhanced federated learning for industrial artificial intelligence | |
CN114978533B (en) | Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning | |
Li et al. | Privacy-preserving machine learning with multiple data providers | |
CN109951443B (en) | Set intersection calculation method and system for privacy protection in cloud environment | |
CN114219483B (en) | Method, equipment and storage medium for sharing block chain data based on LWE-CPBE | |
CN110912897B (en) | Book resource access control method based on ciphertext attribute authentication and threshold function | |
CN113489591B (en) | Traceable comparison attribute encryption method based on multiple authorization centers | |
CN111581648B (en) | Method of federal learning to preserve privacy in irregular users | |
CN112383550A (en) | Dynamic authority access control method based on privacy protection | |
CN116049897B (en) | Verifiable privacy protection federal learning method based on linear homomorphic hash and signcryption | |
WO2021106077A1 (en) | Update method for neural network, terminal device, calculation device, and program | |
CN117216805A (en) | Data integrity audit method suitable for resisting Bayesian and hordeolum attacks in federal learning scene | |
CN118381600B (en) | Federal learning privacy protection method and system | |
CN113346993A (en) | Layered dynamic group key negotiation method based on privacy protection | |
CN113132104A (en) | Active and safe ECDSA (electronic signature SA) digital signature two-party generation method | |
Yang et al. | Efficient and secure federated learning with verifiable weighted average aggregation | |
CN117421762A (en) | Federal learning privacy protection method based on differential privacy and homomorphic encryption | |
Mahdavi et al. | IoT-friendly, pre-computed and outsourced attribute based encryption | |
Meraouche et al. | Learning asymmetric encryption using adversarial neural networks | |
CN110740034A (en) | Method and system for generating QKD network authentication key based on alliance chain | |
CN113938275B (en) | Quantum homomorphic signature method based on d-dimension Bell state | |
CN111581663B (en) | Federal deep learning method for protecting privacy and facing irregular users | |
CN108011723A (en) | Invade the undetachable digital signatures method of rebound | |
Meraouche et al. | Tree Parity Machine-Based Symmetric Encryption: A Hybrid Approach |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |