CN114978533A - Verifiable security aggregation method based on weighted layered asynchronous federated learning - Google Patents

Verifiable security aggregation method based on weighted layered asynchronous federated learning Download PDF

Info

Publication number
CN114978533A
CN114978533A CN202210519513.8A CN202210519513A CN114978533A CN 114978533 A CN114978533 A CN 114978533A CN 202210519513 A CN202210519513 A CN 202210519513A CN 114978533 A CN114978533 A CN 114978533A
Authority
CN
China
Prior art keywords
client
server
parameters
updating
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210519513.8A
Other languages
Chinese (zh)
Other versions
CN114978533B (en
Inventor
秦宝东
杨国栋
郑东
郭瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Posts and Telecommunications
Original Assignee
Xian University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Posts and Telecommunications filed Critical Xian University of Posts and Telecommunications
Priority to CN202210519513.8A priority Critical patent/CN114978533B/en
Publication of CN114978533A publication Critical patent/CN114978533A/en
Application granted granted Critical
Publication of CN114978533B publication Critical patent/CN114978533B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computer And Data Communications (AREA)
  • Storage Device Security (AREA)

Abstract

The invention belongs to the technical field of safe federal learning and discloses a verifiable safety polymerization method based on weighted layered asynchronous federal learning, which mainly comprises the following steps: 1) key agreement; 2) the client executes; 3) and (4) server execution. In order to improve the safety and efficiency of federal learning, the invention provides a verifiable safety aggregation method of weighted layered asynchronous federal learning, and a server can distribute corresponding aggregation weights for each round of clients participating in updating, thereby improving the efficiency of a model; the server can still recover correct aggregation parameters from the aggregated ciphertext under the condition that plaintext parameters of other legal clients cannot be obtained; the server can know whether the parameter to be aggregated of each client is maliciously tampered or not only by generating and verifying a short signature, so that the purposes of improving the federal learning efficiency and ensuring the federal learning safety can be achieved.

Description

Verifiable security aggregation method based on weighted layered asynchronous federated learning
Technical Field
The invention belongs to the technical field of safe federal learning, and particularly relates to a verifiable safe polymerization method based on weighted layered asynchronous federal learning.
Background
With the development trend of a new era of mass data, machine learning technology is applied to various industries at an unprecedented speed. However, the problem of data islanding makes the development of traditional machine learning reach a bottleneck, which not only severely restricts the application of artificial intelligence technology in enterprises, but also causes the waste of precious data resources. In order to effectively solve the problem of data resource waste caused by data isolated island, the federal learning technology is released, and a new hope is brought to the development of information technology.
Federated learning obtains a central model on a global server by aggregating locally trained models on local clients, but the communication overhead between clients and servers is large. In 2020, Chen et al provides an efficient hierarchical asynchronous federated learning, and through the strategies of hierarchical asynchronous learning on a client and time weighted aggregation on a server, the high precision of a model is kept, meanwhile, the communication overhead between the client and the server is reduced, and meanwhile, the difficulty of privacy protection of the hierarchical asynchronous federated learning is increased. The mode of using the aggregation model parameters in federal learning protects the data privacy of a local client to a certain extent, but still has the security problems of reasoning attack and the like.
Some clients in the federal learning system frequently participate in model updating (diligent clients), some clients occasionally participate in model updating (lazy clients), and it is not reasonable that the server only weights the importance of each local model according to the number of data points of training data on the clients. In the federal learning parameter security aggregation process, not only the privacy of the updated parameters but also whether the parameters are maliciously tampered by a third party are considered, and the maliciously tampered updated parameters can cause the aggregation result of the server to deviate from the correct value.
In order to solve the above problems, the present application proposes a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning.
Disclosure of Invention
The invention aims to provide a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning, so as to solve the problems in the background technology.
In order to achieve the above purpose, the invention provides the following technical scheme: a verifiable safety aggregation method based on weighted layered asynchronous federal learning, wherein a deep neural network is widely adopted as a local model in the federal learning, and the safety aggregation method mainly comprises the following steps:
1) and (3) key agreement: the client and the server mutually negotiate respective secret keys for encryption, decryption and signature verification;
2) the client executes: the local client independently updates to obtain an updating parameter, then encrypts and signs the updating parameter by using a private key of the client, and sends the encrypted updating parameter and the signature to the server;
3) service period execution: the central server firstly aggregates the signatures of all the clients to generate a short signature and carries out verification, and then aggregates all the ciphertexts after the verification is passed, and recovers the correct aggregation parameters.
Preferably, for the main characteristics of the deep neural network, the number of parameters in the shallow layer is relatively less and is more critical to the performance of the central model, and accordingly, the update frequency of the shallow layer should be higher than that of the deep layer; therefore, the parameters at the shallow layer and the deep layer in the model can be updated asynchronously, so that the data volume sent between the server and the client is reduced, and the updating mode is called layered asynchronous updating.
Preferably, shallow learning of the deep neural network is applicable to general features of different tasks and data sets, which means that a relatively small part of parameters in the deep neural network, i.e. parameters at a shallow layer, represent general features of different data sets; deep learning of deep neural networks is associated with specific features of a particular data set, and a large number of parameters focus on the learned features of the specific data.
Preferably, when aggregating client training parameters, the server in federal learning not only weights the importance of each local model according to the number of data points of training data on corresponding clients, but also allocates appropriate weight values q to different clients participating in training in each round k (ii) a Because some clients in the system frequently participate in model updating, namely diligent clients, and some clients occasionally participate in model updating, namely lazy clients, the server adopts a time-weighted aggregation strategy more reasonably.
Preferably, in the federal learning system, even if the client does not directly upload the data of the user, an attacker can still indirectly deduce the characteristics of the client tag data and the member information of the data set through the model parameters uploaded by the client; the exposed model parameters are trained by the local data set of the client, namely the local data of the client is coded into the model parameters; however, if an attacker can construct a corresponding decoder according to the exposed model parameters, the attacker can reversely infer the private data local to the client according to the model parameters of the client, and this attack mode is called inference attack.
Preferably, each client k chooses its own secret value a k,0 And only knows the client k according to the secret value a k,0 Construct a polynomial S k (x) And substituting the ID of the client into a polynomial S k (x) And sent to the server, which then uses the secret value a k,0 And masking the update parameters and sending the update parameters to the server.
Preferably, client K generates a vector of K +1 dimensions
Figure BDA0003641099260000031
The masked updated result is then
Figure BDA0003641099260000032
Put on the first component, and set a '1' on the k +1 th component, the other components are '0', that is:
Figure BDA0003641099260000033
then the client k signs the covering result to obtain
Figure BDA0003641099260000034
Preferably, the server receives the masked results of the K clients
Figure BDA0003641099260000035
And a vector of K +1 dimensions
Figure BDA0003641099260000036
And signature information σ k . The server adds a corresponding weight value q to each client k k And aggregate into a short signature
Figure BDA0003641099260000037
Then for short signature sigma Agg Verifying and judging bilinear pairs E 1 :e(σ Agg Beta) is equal to bilinear pair E 2 :
Figure BDA0003641099260000038
Preferably, the server assigns a weight value q to each client k k Added to the masked result
Figure BDA0003641099260000039
And polymerized to obtain
Figure BDA0003641099260000041
The server uses the negotiated total secret key S to cover and offset the aggregation result to obtain an offset result
Figure BDA0003641099260000042
The server can calculate the result of the security aggregation of the central model according to the total data point N
Figure BDA0003641099260000043
Preferably, the specific steps of 1) are:
A. the client k selects the secret value a according to itself k,0 And the number of points n of the local data point k Respectively construct h-1 polynomial
Figure BDA0003641099260000044
And
Figure BDA0003641099260000045
B. the client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server, this time the server is to S k (x) Substituting the weighted value q corresponding to the client k k ，N k (x) Keeping the same, then the server arranges the polynomial with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k ；
C. S for server to pass k clients k And N k Recovering the total secret S and the total data point N by a Lagrange interpolation method, and selecting G 1 、G 2 And G T All are cyclic groups of prime number p, establishing bilinear mapping e: G 1 ×G 2 →G T ，
Figure BDA0003641099260000046
Is G 2 →G 1 Is capable of efficiently calculating isomorphism and generating bilinear group
Figure BDA0003641099260000047
Select one generator β ← G 2 \ {1 }; each client shares a private key
Figure BDA0003641099260000048
And calculating u ═ beta γ (ii) a Select a hash function H: {0,1} * ×{0,1} * →G 1 (ii) a Public key PK ═ (G, H, β, u) is used for server authentication signature, and private key SK ═ γ is used forA signature is generated at the client.
Preferably, the specific steps of 2) are:
client k receives the latest ω from the server t,g ,ω t,s Flag is used for updating in the current round;
client-side first-to-shallow depth neural network parameters
Figure BDA0003641099260000051
Is updated to obtain
Figure BDA0003641099260000052
Then to the updated result
Figure BDA0003641099260000053
Is masked by secret parameters to obtain
Figure BDA0003641099260000054
Then to mask the result
Figure BDA0003641099260000055
Signature derivation σ k,g ；
Finally, the result will be masked
Figure BDA0003641099260000056
And a signature σ k,g And sending the data to a server. If flag is 'yes', then the deep depth neural network parameters are processed
Figure BDA0003641099260000057
Updating, masking and signing are carried out in the same way; if flag is 'No', the process is directly carried out.
Preferably, the specific steps of 3) are:
A. server initialization parameter omega 0 And T, determining that K clients exist in the system, and acquiring a public key PK (G, H, beta, u); the server randomly selects a set m of participants and then enters model updating;
B. and (4) updating the current update round module to Rlp, if the result belongs to the Set, marking the flag as ' yes ', otherwise, marking the flag as ' yesIf 'NO', if 'yes', the server will update newly
Figure BDA0003641099260000058
And flag is sent to client k, otherwise updated
Figure BDA0003641099260000059
And flag is sent to the client k;
C. each client k locally and independently executes model parameter updating, covering and signing and sends the model parameter updating, covering and signing to the server, and the server assigns the current turn t to the client k
Figure BDA00036410992600000510
If flag is 'yes', the server reassigns the current turn t to
Figure BDA00036410992600000511
D. When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k (ii) a If the flag is 'yes', the server firstly aggregates deep signatures of all clients k and verifies and adds the weight value q k After the signature passes the verification, carrying out safe aggregation on the deep updating parameters, otherwise, quitting;
E. and finally, uniformly aggregating shallow signatures of all clients k by the server, verifying and adding weight values q k And after the signature passes the verification, safely aggregating the shallow layer updating parameters, otherwise, quitting, and updating the central model parameters by the server according to the aggregation result, and entering the next round of model updating until the model converges.
The invention has the following beneficial effects:
1. in order to improve the safety and efficiency of federal learning, the invention provides a verifiable safety aggregation method of weighted layered asynchronous federal learning, and a server can distribute corresponding aggregation weights for each round of clients participating in updating, thereby improving the efficiency of a model; the server can still recover correct aggregation parameters from the aggregated ciphertext under the condition that plaintext parameters of other legal clients cannot be obtained; the server can know whether the parameter to be aggregated of each client is maliciously tampered or not only by generating and verifying one short signature, so that the purposes of improving the federal learning efficiency and ensuring the federal learning safety can be achieved.
Drawings
FIG. 1 is a flow chart of a client implementation of the present invention;
FIG. 2 is a flow chart of a server implementation of the present invention;
FIG. 3 is a flow chart of the client local update of the present invention;
FIG. 4 is a general diagram of the model of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 to 4, the present invention provides a verifiable security aggregation method based on weighted hierarchical asynchronous federated learning, in the key agreement part:
client k construction polynomial
Figure BDA0003641099260000061
And polynomial
Figure BDA0003641099260000071
Wherein x k Is ID of client K, h is threshold value and h is less than or equal to K, a k,0 Is a secret value, n k Points are counted for local data.
The client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server. Server pair S at this time k (x) Substituting the weighted value q corresponding to the client k k ，N k (x) Is maintained atAnd (6) changing. Then, the server arranges the polynomials with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k . The server can share S the secret by k clients k And data point quanta N k And recovering the total secret S and the total data point N by a Lagrange interpolation method.
Get G 1 、G 2 And G T Is a cyclic group with a prime number p, satisfying a bilinear mapping e: G 1 ×G 2 →G T ，g 1 And g 2 Are respectively group G 1 And G 2 The generation element(s) of (a),
Figure BDA0003641099260000072
is G 1 To G 2 Can be calculated isomorphically and
Figure BDA0003641099260000073
1) generating a bilinear group of values
Figure BDA0003641099260000074
Select one generator β ← G 2 \{1}；
2) Each client shares a private key
Figure BDA0003641099260000075
And calculating u ═ beta γ ；
3) Select a hash function H: {0,1} * ×{0,1} * →G 1 ；
4) The output p, public key PK ═ (G, H, β, u) and private key SK ═ γ.
The private key SK is used for signing the masking result of the local update parameters by each client k, and the public key PK is used for verifying the aggregation signature by the server.
In order to further explain the technical scheme content of the invention, the specific implementation mainly comprises three processes: a key agreement process, a client execution process, and a server execution process. The implementation steps of each process are as follows:
first, key negotiation
Let gf (p) be a finite field, where p is a large prime number. There are a total of K clients in the system, each client K selecting its own secret a k,0 And number of local data points n k And randomly selecting a secret parameter a k,j Where K1, 1., K and j 1., h-1, secret a k,0 And a random parameter a k,j Are random numbers uniformly chosen over gf (p). Construction of h-1 degree polynomial
Figure BDA0003641099260000081
And
Figure BDA0003641099260000082
wherein x k Is the ID of the client K, h is a threshold value and h is less than or equal to K.
The client k constructs a polynomial S by itself k (x) And N k (x) Sending to the server, wherein x is e { x 1 ,...,x K Denotes IDs of all clients.
Figure BDA0003641099260000083
Figure BDA0003641099260000084
The server receives S from the client k k (x) And N k (x) At this time, the server pair S k (x) Substituting the weighted value q corresponding to the client k k 。
Figure BDA0003641099260000085
The server arranges the polynomials with the same client ID to obtain S (x) k ) And N (x) k ) In which S (x) k ) And N (x) k ) Denotes the client ID as x k The polynomial of (2) collates the results.
Figure BDA0003641099260000086
Figure BDA0003641099260000091
Then, the server pairs S (x) respectively k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k 。
Figure BDA0003641099260000092
Figure BDA0003641099260000093
Secret share S of server through k clients k And data point quanta N k The lagrange interpolation method can recover the total secret S and the total data point N, namely:
Figure BDA0003641099260000094
Figure BDA0003641099260000095
lagrangian parameter for client k
Figure BDA0003641099260000096
Signature setting:
let G 1 、G 2 And G T Is a cyclic group with a prime number p, satisfying a bilinear mapping e: G 1 ×G 2 →G T ，g 1 And g 2 Are respectively group G 1 And G 2 The generation element of (a) is generated,
Figure BDA0003641099260000097
is G 1 To G 2 Can be calculated isomorphically and
Figure BDA0003641099260000098
1) generating a bilinear group of values
Figure BDA0003641099260000099
Select one generator β ← G 2 \{1}；
2) Each client shares a private key
Figure BDA00036410992600000910
And calculating u ═ beta γ ；
3) Select a hash function H: {0,1} * ×{0,1} * →G 1 ；
4) The output p, public key PK ═ (G, H, β, u) and private key SK ═ γ.
The private key SK is used for each client k to sign the masking result of the local update parameters, and the public key PK is used for the server to verify the aggregated signature.
Second, the client executes
With reference to fig. 1, the client specifically executes the following steps:
1) client k randomly selects mask parameter a k,0 And gamma is used as the private key of the signature, where gamma is one private signature key shared between each client and known only to the client.
2) Client k receives the latest ω from the server t,g ,ω t,s And flag, which is used for updating the current round.
3) Client k first-to-shallow DNN weight parameter
Figure BDA0003641099260000101
Performing local update to obtain the updated result
Figure BDA0003641099260000102
The updating process is as shown in figure 3.
4) Client k pairsUpdating results
Figure BDA0003641099260000103
Secret parameter covering is carried out to ensure the safety of shallow layer updating parameters; then the masked result is compared
Figure BDA0003641099260000104
Signing is carried out, and the result is masked
Figure BDA0003641099260000105
And a signature σ k,g And sending the data to a server.
5) If flag is marked as 'yes', client k further performs weighting on deep DNN (digital DNN)
Figure BDA0003641099260000106
Executing local update, the update process is the same as the step 3), and obtaining the update result
Figure BDA0003641099260000107
6) Client k pair update results
Figure BDA0003641099260000108
Secret parameter masking is carried out to ensure the safety of deep updating parameters; then the masked result is processed
Figure BDA0003641099260000109
Carry out signature and mask the result
Figure BDA00036410992600001010
And a signature σ k.s And sending the data to a server.
7) And if the flag is marked as 'no', directly exiting, and finishing the execution of the client.
Parameter masking and signature process:
a) client k uniformly selects random number a on GF (p) k,0 The method is used for carrying out secret parameter masking on the updated result, and firstly carrying out secret parameter masking on the updated result of the shallow DNNNamely:
Figure BDA00036410992600001011
b) the client K generates a vector with K +1 dimensions
Figure BDA00036410992600001012
The masked updated result is then
Figure BDA00036410992600001013
Put on the first component, and set a '1' on the k +1 th bit, and the other bits are '0', that is:
Figure BDA00036410992600001014
c) the client k signs the masking result, namely:
Figure BDA0003641099260000111
d) updating result of client k for covering secret parameter
Figure BDA0003641099260000112
And the generated K + 1-dimensional vector
Figure BDA0003641099260000113
And a signature σ k,g And sending the data to a server.
The secret parameter masking and signing process for the deep DNN update results is the same as for the shallow DNN. In the same way, the client k can obtain a deep DNN updating result with the parameters covered
Figure BDA0003641099260000114
And the generated K + 1-dimensional vector
Figure BDA0003641099260000115
And a signature σ k,s And all the data are sent to the server, and the execution of the client ends at the moment.
Third, the server executes
With reference to fig. 2, the server specifically executes the following steps:
1) the server first pair the parameter omega 0 And T is initialized. Wherein: omega 0 And converging the DNN model through a T-round model as an initial weight parameter of the DNN model.
2) And selecting a client set, and determining that K clients exist in the whole system.
3) The server obtains the public key PK ═ (G, H, β, u) for verification of subsequent aggregated signatures.
4) Randomly selecting a set m of participants, wherein m represents the client set after learning is exited actively or passively in all the client sets K, and h is less than or equal to | m |. If m < h, quitting learning. Where h is the threshold value for secret sharing.
5) And (8) performing digital analogy on the current round Rlp, if the result belongs to the Set, marking the flag as 'yes', and otherwise, marking the flag as 'no'. Rlp are groupings of update round numbers, Set specifies a Set of deep DNN updates in each group of update round numbers.
6) If flag is 'yes', the server will update newly
Figure BDA0003641099260000116
And the flag is sent to the client k; if the flag is 'NO', the server updates the latest data
Figure BDA0003641099260000117
And flag is sent to client k.
7) Then each client k independently executes model parameter updating locally, carries out secret parameter masking and signing on the updating result, and masks the result
Figure BDA0003641099260000118
And a signature σ k And sending the data to a server.
8) The server assigns the current round t to the client k
Figure BDA0003641099260000119
If flag is 'yes',the server reassigns the current round t to
Figure BDA00036410992600001110
Denotes tsp k Is updated as client k participates in its turn. If the client k does not participate in the turn, tsp k The value of (c) is the round of the previous round, i.e., no update has occurred.
9) When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k . The client k not belonging to the m set selects the masking result of the previous round and adds the weight value
Figure BDA0003641099260000121
The closer the client participating in the update is to the current round, the larger the weight value owned, and conversely, the smaller the weight value owned.
10) If the flag is 'yes', the server firstly aggregates deep signatures of all the client sides k and verifies the added weight value q k The latter signature; if E 1,s ＝E 2,s And if not, the deep updating parameters are safely aggregated, and otherwise, the operation is quitted.
11) Since the shallow DNN parameters need to participate in updating in each round, the server uniformly aggregates shallow signatures of all clients k and verifies the added weight value q k The latter signature; if E 1,g ＝E 2,g And if not, the shallow updated parameters are safely aggregated, otherwise, the operation is quitted.
12) And updating the central model parameters according to the aggregation result, and entering the next round of model updating until the model converges. Signature verification and masking cancellation:
a) the server receives the masking results of the K clients
Figure BDA0003641099260000122
And a K +1 dimensional vector
Figure BDA0003641099260000123
And signature information (sigma) k,g ,σ k,s ) Which isK is 1.
b) If flag is 'yes', the signatures of the deep DNN are firstly aggregated. In the aggregation process, the weight value q of each client k is weighted k Added to the signature σ k,s And polymerized to obtain
Figure BDA0003641099260000124
c) Then, the aggregate signature is verified, and whether the bilinear pairs are equal is judged, namely
Figure BDA0003641099260000125
d) If E 1,s ＝E 2,s If the verification is passed, the weight value q of each client k is determined k Added to the masked result
Figure BDA0003641099260000126
And carrying out polymerization to obtain
Figure BDA0003641099260000127
Namely:
Figure BDA0003641099260000128
if E 1,s ≠E 2,s And then directly quit.
e) The server then uses the client secret share S k The recovered total secret S is used for masking the weighted result
Figure BDA0003641099260000131
Carrying out cancellation to obtain a cancellation result m Agg,s I.e. weighted plaintext aggregation results.
Figure BDA0003641099260000132
f) The server can obtain the safety aggregation result omega with the weighted deep DNN parameters t+1,s Where N is the total number of data points recovered by the server.
Figure BDA0003641099260000133
Then, the signature σ of the shallow DNN is further applied k,g And masking the results
Figure BDA0003641099260000134
Performing aggregate verification and mask cancellation in the same manner as deep DNN. If E 1,g ＝E 2,g That is, the server can obtain the security aggregation result ω of the weighted shallow DNN after verification t+1,g 。
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (12)

1. A verifiable safety aggregation method based on weighted layered asynchronous federal learning, wherein a deep neural network is widely adopted as a local model in the federal learning, and the method is characterized in that: the safe polymerization method mainly comprises the following steps:
1) and (3) key agreement: mutually negotiating respective secret keys between the client and the server for encryption, decryption and signature verification;
2) the client executes: the local client independently updates to obtain an updating parameter, then encrypts and signs the updating parameter by using a private key of the client, and sends the encrypted updating parameter and the signature to the server;
3) the service period executes: the central server firstly aggregates the signatures of all the clients to generate a short signature and carries out verification, and then aggregates all the ciphertexts after the verification is passed, and recovers the correct aggregation parameters.
2. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: for the main features of the deep neural network, the number of parameters in a shallow layer is relatively less and more critical to the performance of the central model, and accordingly, the update frequency of the shallow layer should be higher than that of the deep layer; therefore, the parameters at the shallow layer and the deep layer in the model can be updated asynchronously, so that the data volume sent between the server and the client is reduced, and the updating mode is called layered asynchronous updating.
3. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: shallow learning of the deep neural network is suitable for general features of different tasks and data sets, which means that a relatively small part of parameters in the deep neural network, namely parameters at a shallow layer, represent general features of different data sets; deep learning of deep neural networks is associated with specific features of a particular data set, and a large number of parameters focus on the learned features of the specific data.
4. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: when the server aggregates the client training parameters in the federal learning, the importance of each local model is weighted according to the number of data points of training data on the corresponding client, and the server distributes proper weight values q to different clients participating in training in each round k (ii) a Since some of the systemsThe client frequently participates in model updating, namely a diligence client, and some clients occasionally participate in model updating, namely a lazy client, so that the server is more reasonable in adopting a time weighted aggregation strategy.
5. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 1, characterized in that: in the federal learning system, even if a client does not directly upload user data, an attacker can still indirectly deduce the characteristics of client label data and member information of a data set through model parameters uploaded by the client; the exposed model parameters are trained by the local data set of the client, namely the local data of the client is coded into the model parameters; however, if an attacker can construct a corresponding decoder according to the exposed model parameters, the attacker can reversely infer the private data local to the client according to the model parameters of the client, and this attack mode is called inference attack.
6. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 5, characterized in that: each client k selects its secret value a k,0 And only knows the client k according to the secret value a k,0 Constructing a polynomial S k (x) And substituting the ID of the client into a polynomial S k (x) And sent to the server, which then uses the secret value a k,0 And masking the update parameters and sending the update parameters to the server.
7. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the client K generates a vector with K +1 dimensions
Figure FDA0003641099250000021
The masked updated result is then
Figure FDA0003641099250000022
Put on the first component, and set a '1' on the k +1 th component, the other components are '0', that is:
Figure FDA0003641099250000023
then the client k signs the covering result to obtain
Figure FDA0003641099250000024
8. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the server receives the masking results of the K clients
Figure FDA0003641099250000025
And a vector of K +1 dimensions
Figure FDA0003641099250000026
And signature information σ k The server adds a corresponding weight value q to each client k k And aggregate into a short signature
Figure FDA0003641099250000031
Then for short signature sigma Agg Verifying and judging bilinear pairs E 1 :e(σ Agg Beta) is equal to a bilinear pair
Figure FDA0003641099250000032
9. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning of claim 6, characterized in that: the server weights q of each client k k Added to the masked result
Figure FDA0003641099250000033
And carrying out polymerization to obtain
Figure FDA0003641099250000034
The server uses the negotiated total secret key S to cover and offset the aggregation result to obtain an offset result
Figure FDA0003641099250000035
The server can calculate the result of the security aggregation of the central model according to the total data point N
Figure FDA0003641099250000036
10. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that the specific steps of 1) are:
A. the client k selects the secret value a according to the client k k,0 And the number of points n of the local data point k Respectively construct h-1 polynomial
Figure FDA0003641099250000037
And
Figure FDA0003641099250000038
B. the client k substitutes the IDs of all the clients into a polynomial S constructed by the client k k (x) And N k (x) And sent to the server, this time the server is to S k (x) Substituting the weighted value q corresponding to the client k k ，N k (x) Keeping the same, then the server arranges the polynomial with the same client ID to obtain S (x) k ) And N (x) k ) And separately for S (x) k ) And N (x) k ) Summing to obtain secret share S of client k k And local data point share N k ；
C. S for server to pass k clients k And N k Recovering the total secret S and the total data point N by a Lagrange interpolation method, and selecting G 1 、G 2 And G T Are all cyclic groups of prime order p, constructedVertical bilinear map e G 1 ×G 2 →G T ，
Figure FDA0003641099250000041
Is G 2 →G 1 Is capable of efficiently calculating isomorphism and generating bilinear group
Figure FDA0003641099250000042
Select one generator β ← G 2 \ {1 }; each client shares a private key
Figure FDA0003641099250000043
And calculating u ═ beta γ (ii) a Select a hash function H: {0,1} * ×{0,1} * →G 1 (ii) a The public key PK ═ (G, H, β, u) is used for server verification of the signature, and the private key SK ═ γ is used for client generation of the signature.
11. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that, the specific steps of 2) are:
client k receives the latest ω from the server t,g ,ω t,s Flag is used for updating in the current round;
client-side first-to-shallow depth neural network parameters
Figure FDA0003641099250000044
Is updated to obtain
Figure FDA0003641099250000045
Then to the updated result
Figure FDA0003641099250000046
Carrying out secret parameter masking to obtain
Figure FDA0003641099250000047
Then to mask the result
Figure FDA0003641099250000048
Signature derivation σ k,g ；
Finally, the result will be masked
Figure FDA0003641099250000049
And signature σ k,g Sending to the server, if flag is 'yes', then processing the deep neural network parameter
Figure FDA00036410992500000410
Updating, masking and signing are carried out in the same way; if flag is 'no', directly exiting.
12. The verifiable security aggregation method based on weighted hierarchical asynchronous federated learning according to claim 1, characterized in that, the specific steps of 3) are:
A. server initialization parameter omega 0 And T, determining that K clients exist in the system, and acquiring a public key PK (G, H, beta, u); the server randomly selects a set m of participants and then enters model updating;
B. updating Rlp the round of module currently, if the result belongs to the Set, marking flag as 'yes', otherwise marking flag as 'no'; if flag is 'yes', the server will update newly
Figure FDA00036410992500000411
Figure FDA00036410992500000412
And flag is sent to client k, otherwise updated newly
Figure FDA00036410992500000413
And the flag is sent to the client k;
C. each client k locally and independently executes model parameter updating, covering and signing and sends the model parameter updating, covering and signing to the server, and the server assigns the current turn t to the client k
Figure FDA0003641099250000051
If flag is 'yes', the server reassigns the current turn t to
Figure FDA0003641099250000052
D. When the clients belonging to the m set are all updated, the server adds the weighted weight value q of time weighting to the masking results of all the K clients k (ii) a If the flag is 'yes', the server firstly aggregates deep signatures of all clients k and verifies and adds the weight value q k After the signature passes the verification, carrying out safe aggregation on the deep updating parameters, otherwise, quitting;
E. and finally, uniformly aggregating shallow signatures of all clients k by the server, verifying and adding weight values q k And after the signature passes the verification, safely aggregating the shallow layer updating parameters, otherwise, quitting, and updating the central model parameters by the server according to the aggregation result, and entering the next round of model updating until the model converges.
CN202210519513.8A 2022-05-12 2022-05-12 Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning Active CN114978533B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210519513.8A CN114978533B (en) 2022-05-12 2022-05-12 Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210519513.8A CN114978533B (en) 2022-05-12 2022-05-12 Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning

Publications (2)

Publication Number Publication Date
CN114978533A true CN114978533A (en) 2022-08-30
CN114978533B CN114978533B (en) 2023-06-30

Family

ID=82984175

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210519513.8A Active CN114978533B (en) 2022-05-12 2022-05-12 Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning

Country Status (1)

Country Link
CN (1) CN114978533B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116052832A (en) * 2023-04-03 2023-05-02 青岛市妇女儿童医院（青岛市妇幼保健院、青岛市残疾儿童医疗康复中心、青岛市新生儿疾病筛查中心） Tamper-proof transmission method based on medical information
TWI818708B (en) * 2022-09-02 2023-10-11 英業達股份有限公司 Method for verifying model update
CN117811722A (en) * 2024-03-01 2024-04-02 山东云海国创云计算装备产业创新中心有限公司 Global parameter model construction method, secret key generation method, device and server

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112149160A (en) * 2020-08-28 2020-12-29 山东大学 Homomorphic pseudo-random number-based federated learning privacy protection method and system
CN113221105A (en) * 2021-06-07 2021-08-06 南开大学 Robustness federated learning algorithm based on partial parameter aggregation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112149160A (en) * 2020-08-28 2020-12-29 山东大学 Homomorphic pseudo-random number-based federated learning privacy protection method and system
CN113221105A (en) * 2021-06-07 2021-08-06 南开大学 Robustness federated learning algorithm based on partial parameter aggregation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FEIYUE WANG, ETC.: "Federated data: Toward new generation of credible and trustable artificial intelligence", 《IEEE》 *
成艺: "联合学习环境下保护隐私的数据聚合技术研究", 《中国优秀硕士学位论文全文数据库（电子期刊）》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI818708B (en) * 2022-09-02 2023-10-11 英業達股份有限公司 Method for verifying model update
CN116052832A (en) * 2023-04-03 2023-05-02 青岛市妇女儿童医院（青岛市妇幼保健院、青岛市残疾儿童医疗康复中心、青岛市新生儿疾病筛查中心） Tamper-proof transmission method based on medical information
CN117811722A (en) * 2024-03-01 2024-04-02 山东云海国创云计算装备产业创新中心有限公司 Global parameter model construction method, secret key generation method, device and server
CN117811722B (en) * 2024-03-01 2024-05-24 山东云海国创云计算装备产业创新中心有限公司 Global parameter model construction method, secret key generation method, device and server

Also Published As

Publication number Publication date
CN114978533B (en) 2023-06-30

Similar Documents

Publication Publication Date Title
Xu et al. VerifyNet: Secure and verifiable federated learning
Hao et al. Efficient and privacy-enhanced federated learning for industrial artificial intelligence
CN114978533B (en) Verifiable security aggregation method based on weighted hierarchical asynchronous federal learning
Li et al. Privacy-preserving machine learning with multiple data providers
CN109951443B (en) Set intersection calculation method and system for privacy protection in cloud environment
CN114219483B (en) Method, equipment and storage medium for sharing block chain data based on LWE-CPBE
CN110912897B (en) Book resource access control method based on ciphertext attribute authentication and threshold function
CN113489591B (en) Traceable comparison attribute encryption method based on multiple authorization centers
CN111581648B (en) Method of federal learning to preserve privacy in irregular users
CN112383550A (en) Dynamic authority access control method based on privacy protection
CN116049897B (en) Verifiable privacy protection federal learning method based on linear homomorphic hash and signcryption
WO2021106077A1 (en) Update method for neural network, terminal device, calculation device, and program
CN117216805A (en) Data integrity audit method suitable for resisting Bayesian and hordeolum attacks in federal learning scene
CN118381600B (en) Federal learning privacy protection method and system
CN113346993A (en) Layered dynamic group key negotiation method based on privacy protection
CN113132104A (en) Active and safe ECDSA (electronic signature SA) digital signature two-party generation method
Yang et al. Efficient and secure federated learning with verifiable weighted average aggregation
CN117421762A (en) Federal learning privacy protection method based on differential privacy and homomorphic encryption
Mahdavi et al. IoT-friendly, pre-computed and outsourced attribute based encryption
Meraouche et al. Learning asymmetric encryption using adversarial neural networks
CN110740034A (en) Method and system for generating QKD network authentication key based on alliance chain
CN113938275B (en) Quantum homomorphic signature method based on d-dimension Bell state
CN111581663B (en) Federal deep learning method for protecting privacy and facing irregular users
CN108011723A (en) Invade the undetachable digital signatures method of rebound
Meraouche et al. Tree Parity Machine-Based Symmetric Encryption: A Hybrid Approach

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant