CN113836556B - Federal learning-oriented decentralized function encryption privacy protection method and system - Google Patents
Federal learning-oriented decentralized function encryption privacy protection method and system Download PDFInfo
- Publication number
- CN113836556B CN113836556B CN202111134122.6A CN202111134122A CN113836556B CN 113836556 B CN113836556 B CN 113836556B CN 202111134122 A CN202111134122 A CN 202111134122A CN 113836556 B CN113836556 B CN 113836556B
- Authority
- CN
- China
- Prior art keywords
- encryption
- model
- parameter
- weight
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 239000013598 vector Substances 0.000 claims abstract description 61
- 238000012549 training Methods 0.000 claims abstract description 51
- 238000012360 testing method Methods 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims description 57
- 239000011159 matrix material Substances 0.000 claims description 34
- 230000002776 aggregation Effects 0.000 claims description 19
- 238000004220 aggregation Methods 0.000 claims description 19
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 14
- 125000004122 cyclic group Chemical group 0.000 claims description 10
- 230000004931 aggregating effect Effects 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 9
- 238000003745 diagnosis Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bioethics (AREA)
- Databases & Information Systems (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a Federal learning-oriented decentralized function encryption privacy protection method and system, wherein the method comprises the steps of obtaining an initial model, a public data set, an encryption tag, an encryption prime number, an encryption weight and a weight vector parameter which are sent by a server; training the initial model according to the local data set to obtain a local model, and testing the local model according to the public data set to obtain model accuracy; generating an encryption private key and a partial decryption key according to the encryption prime number, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model; and sending the encryption model, the partial decryption key and the model accuracy to a server so that the server decrypts and aggregates the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model. The invention ensures that the server can not obtain the local model of the user, effectively prevents the collusion attack of the third party and the server, and improves the privacy protection degree and the service effect.
Description
Technical Field
The invention relates to the technical field of privacy protection of federal learning, in particular to a decentralized function encryption privacy protection method and system for federal learning.
Background
With the wide application of federal learning in the fields of digital image processing, natural language processing, text voice processing and the like, on the basis of breaking a data island and providing more accurate service, further solving the privacy disclosure problem in federal learning gradually becomes a key concern in the implementation and application thereof.
The existing privacy protection methods applied to federal learning mainly include homomorphic encryption, secure multi-party computation and function encryption, such as a distributed selection random gradient descent method combined with homomorphic encryption, a method for aggregating client update models by using a secret sharing technology in secure multi-party computation, and a method for ensuring parameter privacy by adding a trusted third-party entity to generate, manage and distribute keys and using function encryption to execute federal learning secure aggregation. Although the prior art provides privacy protection for federal learning to a certain extent, the prior art also has respective defects, for example, a homomorphic encryption technology can bring performance pressure and communication overhead to a certain extent to equipment with weak computing capability, if the computing overhead brought by model encryption is reduced, the security of a parameter aggregation process can be ignored inevitably, and further the privacy protection effect is influenced, and a credible third-party entity is introduced to be responsible for generating, managing and distributing keys, so that a malicious server and the third-party entity can collude to obtain encryption keys, and further the privacy exposure risk of a user model can be stolen.
Therefore, it is desirable to provide a privacy protection method that overcomes the problem of relying on a trusted third party entity in the prior art, and effectively prevents a trusted third party and a server from performing collusion attack while protecting the privacy of a client model.
Disclosure of Invention
The invention aims to provide a decentralized function encryption privacy protection method facing federal learning, which overcomes the problem that the existing privacy protection method depends on a trusted third party entity to generate, manage and distribute keys by means of interaction between a server and a client while ensuring that a server cannot obtain a specific gradient parameter of a local training model of each user, effectively prevents the risk of collusion attack executed by the trusted third party and the server, and further improves the privacy protection degree and the model service effect of a client model in federal learning.
In order to achieve the above object, it is necessary to provide a method, a system, a computer device and a storage medium for protecting privacy of a decentralization function encryption facing federal learning in view of the above technical problems.
In a first aspect, an embodiment of the present invention provides a federally-learned encryption privacy protection method based on a decentralized function, where the method includes the following steps:
acquiring an initial model, a public data set, an encryption label, an encryption prime number, an encryption weight and a weight vector parameter sent by a server; the initial model is obtained by the server through training according to the public data set;
training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain corresponding model accuracy;
generating an encryption private key and a partial decryption key according to the encryption prime number, the encryption weight and the weight vector parameter, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
and sending the encryption model, the partial decryption key and the model accuracy to the server so that the server decrypts and aggregates the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model.
Further, the step of training the initial model according to the local data set to obtain the local model includes:
setting a privacy budget and a noise parameter according to the distribution characteristics and the privacy protection requirements of the local model;
and according to the privacy budget and the noise parameters, performing localized differential privacy to add noise to the local model.
Further, the step of generating an encryption private key and a part of decryption key according to the encryption prime number, the encryption weight and the weight vector parameter comprises:
generating a key parameter according to the encryption prime number, and taking the key parameter as the encryption private key; the key parameter is represented as:
in which p andrespectively representing encrypted prime numbers and corresponding finite fields;the key parameter of the ith client is represented and is a 2-dimensional vector;
generating a decryption parameter according to the encryption prime number, and generating the partial decryption key according to the decryption parameter, the key parameter, the encryption weight and the weight vector parameter; the partial decryption key is represented as:
in the formula,
wherein,and y i Respectively representing a part of decryption keys and encryption weights of the ith client; t is i Represents the decryption parameter generated by the ith client, and sigma i∈[n] T i =0,n is the total number of participating training clients;representing a hash function;is represented by A 2 × 2 matrix is formed;represents a cyclic group of p-factorial method associated with bilinear pairs, anRespectively representing groups of multiplication cyclesA constructed 2-dimensional vector;andrespectively representing the weight vector composed of all client encryption weights and the corresponding weight vector parameters.
Further, the step of generating a decryption parameter according to the encrypted prime number includes:
initializing a parameter matrix; the parameter matrix is a matrix with all elements being zero;
according to the encrypted prime numbers, negotiating with other clients respectively to determine a corresponding random matrix; the random matrix is represented as:
wherein, T ii Representing a random matrix determined by the negotiation of the ith client and the jth client;representing a finite fieldA constructed 2 × 2 matrix;
generating the decryption parameters according to the parameter matrix and the random matrix; the decryption parameters are expressed as:
T i =T 0 +∑ j∈n,i<j T ij -∑ j∈n,i<i T ij
wherein, T i Representing a decryption parameter corresponding to the ith client; t is 0 A parameter matrix representing the client.
Further, the step of performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model includes:
according to the encryption private key and the encryption label, performing function encryption on the local model by adopting the following encryption algorithm to obtain the encryption model:
in the formula,
wherein x is i And [ c) i ] 1 Respectively representing an ith client local model and an encryption model;a key parameter representing an ith client;representing an encrypted tag;represents a multiplicative cyclic group of the order of the cryptographic prime number associated with the bilinear pair, andrepresenting groups of multiplication cyclesA constructed 2-dimensional vector;representing a hash function.
Further, the server decrypts and aggregates the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model, and the step of obtaining the global model includes:
obtaining a decryption key by adopting a key combination algorithm according to the encryption weight and part of the decryption key; the decryption key is represented as:
in the formula,
wherein,andrespectively representing a weight vector composed of a decryption key and all client encryption weights;a partial decryption key representing the ith client;
decrypting and aggregating encryption models of all clients according to the decryption key and the encryption label to obtain the global model; the global model is represented as:
in the formula,
wherein, [ alpha ] is] T Representing a global model; y is i And [ c i ] 1 Respectively representing the encryption weight and the encryption model of the ith client;representing a decryption key;representing an encrypted tag;represents a multiplicative cyclic group of the order of the cryptographic prime number associated with the bilinear pair, andrepresenting groups of multiplication cyclesA constructed 2-dimensional vector;representing a hash function; e (-) represents a bilinear pair map.
Further, the step of the server performing decryption aggregation on the encryption model according to the partial decryption key, the encryption tag, the encryption weight and the model accuracy to obtain a global model further includes:
testing the global model according to the public data set to obtain the accuracy of the global model;
and judging whether the accuracy of the global model reaches a preset accuracy, if so, stopping iterative training, otherwise, updating the encryption weight and the weight vector parameter of each client according to the model accuracy of all the clients, sending the global model and the updated encryption weight and weight vector parameter to each client, and continuing iterative training.
In a second aspect, an embodiment of the present invention provides a federally-learned encryption privacy protection system with a decentralized function, where the system includes:
the acquisition module is used for acquiring the initial model, the public data set, the encryption label, the encryption prime number, the encryption weight and the weight vector parameter sent by the server; the initial model is obtained by the server through training according to the public data set;
the training module is used for training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain corresponding model accuracy;
the encryption module is used for generating an encryption private key and a part of decryption keys according to the encryption prime number, the encryption weight and the weight vector parameters, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
and the aggregation module is used for sending the encryption model, the partial decryption key and the model accuracy to the server so that the server decrypts and aggregates the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the foregoing method when executing the computer program.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the above method.
The method includes the steps that an encryption label, an encryption prime number and an encryption weight are generated through a server, after an initial model is obtained through public data set training, the initial model, a public data set, the encryption label, the encryption prime number, the encryption weight and weight vector parameters are sent to each client side, each client side trains the initial model according to the local data set to obtain a local model, model accuracy is obtained through testing according to the public data set, an encryption private key and a partial decryption key are generated according to the encryption prime number, the local model is subjected to function encryption according to the encryption private key and the encryption label to obtain an encryption model, the partial decryption key and the model accuracy are sent to the server, so that the server completes decryption aggregation of the encryption models of the client sides, obtains a global model, tests the accuracy of the global model through the public data set, and judges whether iteration is needed or not until an ideal global model is obtained. Compared with the prior art, the Federal learning-oriented decentralized function encryption privacy protection method not only ensures that the server cannot obtain specific gradient parameters of a local training model of each user, but also solves the problem that the existing privacy protection method depends on the generation, management and key distribution of a trusted third party entity through a mode that a server and a client interactively generate keys, effectively prevents the risk that the trusted third party and the server execute collusion attack, and further improves the privacy protection degree and the model service effect of the client model in Federal learning.
Drawings
FIG. 1 is a schematic diagram of a federated learning model framework applied to a federated learning-oriented decentralized function encryption privacy protection method in the embodiment of the present invention;
FIG. 2 is a schematic diagram of a collusion attack between a server and a trusted third party in existing privacy protection for federated learning;
FIG. 3 is a schematic flow chart of a Federal learning-oriented decentralized function encryption privacy protection method in the embodiment of the present invention;
FIG. 4 is a schematic flowchart of the step S12 in FIG. 3 for training the initial model to obtain a local model;
fig. 5 is a schematic flowchart of the process that the server decrypts and aggregates the encryption models of all the clients to obtain the global model in step S14 in fig. 3;
fig. 6 is another schematic flow chart of the server performing decryption and aggregation on the encryption models of all the clients in step S14 in fig. 3 to obtain a global model;
FIG. 7 is a schematic structural diagram of a Federal learning-oriented decentralized function encryption privacy protection system in the embodiment of the present invention;
FIG. 8 is a schematic diagram of a Federal learning-oriented decentralized function encryption privacy protection system applied to a smart medical scenario according to an embodiment of the present invention;
fig. 9 is an internal structural diagram of a computer device in the embodiment of the present invention.
Detailed Description
In order to make the purpose, technical solution and advantages of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments, and it is obvious that the embodiments described below are part of the embodiments of the present invention, and are used for illustrating the present invention only, but not for limiting the scope of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
The method for protecting privacy by using function encryption for decentralization facing to federated learning, provided by the invention, is applied to a model frame for protecting privacy by using function encryption for decentralization facing to federated learning shown in figure 1, and can effectively solve the problem of collusion attack between a server and a trusted third party in the federated learning process shown in figure 2 while ensuring that the server can not obtain a specific gradient parameter of a local training model of each user, thereby further improving the privacy protection degree and the model service effect of a client model in federated learning.
As shown in fig. 2, the malicious server obtains an encryption key and an encryption algorithm that are not owned by the malicious server from the trusted third-party entity through means such as private collusion, even profit exchange, and the like, and then calculates a plaintext model of the user client from an encryption model of the user client by using the encryption algorithm and the encryption key, thereby obtaining local privacy information of the user client. Therefore, the dependence on the generation, management and key distribution of a third party is removed in the privacy protection method of federal learning, and the method is very necessary for further enhancing the client privacy protection strength and improving the client privacy protection effect.
In one embodiment, as shown in fig. 3, a federally-learned decentralized function encryption privacy protection method is provided, which includes the following steps:
s11, acquiring an initial model, a public data set, an encryption label, an encryption prime number, an encryption weight and a weight vector parameter sent by a server; the initial model is obtained by the server through training according to the public data set;
the initial model can be selected according to an actual federal learning task, and the public data set is a data set which is collected by the server and can be shared by all client sides for training, and is not specifically limited here. The encryption label, the encryption prime number, the encryption weight and the weight vector parameter are encryption parameters which are necessary to be used for interactive encryption and decryption of the client and the server, the encryption label is a group of character strings which are formed by any characters and have any length, the encryption prime number is any prime number, the encryption weight is a one-to-one corresponding weight numerical value which is distributed to each client by the server according to application scenes and requirements, the weight vector parameter is an encryption parameter which is obtained by the server based on the weight of each client by adopting a Hash algorithm, for example, the server can distribute the corresponding encryption weight to each client according to the data quantity of each client and other conditions during initial iteration, based on the encryption weights of all clients, the weight vector parameter which is fused with the encryption weights of all the clients is obtained by adopting the Hash algorithm, the encryption weight is updated according to the accuracy of a training model of each round of iterative clients during subsequent iteration, and the corresponding weight vector parameter is also updated, so that the model aggregation effect is ensured.
S12, training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain corresponding model accuracy;
the specific loss function and the training mode adopted by the local model are determined based on the actual application requirement and the selected initial model type, and this embodiment is not particularly limited. After each client side obtains a local model through local data set training, the client side uses a public data set sent by the server to carry out testing to obtain corresponding model accuracy, and the model accuracy is sent to the server together when the local model is sent to the server to be aggregated in the follow-up iteration. In addition, in order to enhance the privacy of the client model, after each client obtains a local model by training with a local data set, each client may perform localized differential privacy, add noise to the model, and then encrypt the model, as shown in fig. 4, specifically including:
s121, setting a privacy budget and a noise parameter according to the distribution characteristics of the local model and privacy protection requirements;
and S122, according to the privacy budget and the noise parameter, performing localized differential privacy to add noise to the local model.
The setting of the privacy budget and the noise parameter can be adjusted according to the actual application scenario and the application requirement, which is not limited herein. Meanwhile, in order to improve the efficiency of the whole federal learning, the present embodiment preferably performs model compression by using the existing model compression technology before or after the local model is noisy by performing the localized differential privacy, which is not described herein again.
S13, generating an encryption private key and a partial decryption key according to the encryption prime number, the encryption weight and the weight vector parameter, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
the method comprises the following steps that an encryption private key is used for each client to encrypt a local model to obtain an encryption model, part of decryption keys are sent to a server to be used for decrypting and aggregating the encryption models of all the clients, and the generation of the encryption private key and the part of decryption keys specifically comprises the following steps of:
generating a key parameter according to the encryption prime number, and taking the key parameter as the encryption private key; the key parameter is represented as:
in which p andrespectively representing encrypted prime numbers and corresponding finite fields;the key parameter of the ith client is represented and is a 2-dimensional vector;
generating a decryption parameter according to the encryption prime number, and generating the partial decryption key according to the decryption parameter, the key parameter, the encryption weight and the weight vector parameter; the partial decryption key is represented as:
in the formula,
wherein,and y i Respectively representing a partial decryption key and an encryption weight of the ith client; t is i Represents the decryption parameter generated by the ith client, and sigma i∈[n] T i =0,n is the total number of participating training clients;representing a hash function;is represented byA 2 × 2 matrix is formed;represents a cyclic group of p-factorial method associated with bilinear pairs, anRespectively representing groups of multiplication cyclesA constructed 2-dimensional vector;andrespectively representing the weight vector composed of all client encryption weights and the corresponding weight vector parameters.
The step of generating the decryption parameter by each client according to the encryption prime number comprises the following steps:
initializing a parameter matrix; the parameter matrix is a matrix with all elements being zero;
according to the encrypted prime numbers, negotiating with other clients respectively to determine a corresponding random matrix; the random matrix is represented as:
wherein, T ij Representing a random matrix determined by the negotiation of the ith client and the jth client;representing by finite fieldsFormed 2 x 2 matrix;
Generating the decryption parameters according to the parameter matrix and the random matrix; the decryption parameters are expressed as:
T i =T 0 +∑ j∈n,i<j T ij -∑ j∈n,i<i T ij
wherein, T i Representing a decryption parameter corresponding to the ith client; t is a unit of 0 A parameter matrix representing the client.
It should be noted that the parameter matrix initialized by each client may also be a non-zero matrix, and for convenience of calculation, all the parameter matrices are set as zero matrices in this embodiment. In addition, the method for mutually negotiating and determining the corresponding random matrix between the clients may be implemented by using Diffie-Hellman key exchange protocol, or may also use other similar negotiation techniques, which is not limited herein. After the client side obtains the encryption private key by adopting the steps of the method, the local model can be subjected to function encryption by adopting the following encryption algorithm according to the encryption private key and the encryption label to obtain a corresponding encryption model:
in the formula,
wherein x is i And [ c) i ] 1 Respectively representing an ith client local model and an encryption model;a key parameter representing an ith client;representing an encrypted tag;represents a multiplicative cyclic group of the order of the cryptographic prime number associated with the bilinear pair, andrepresenting groups of multiplication cyclesA constructed 2-dimensional vector;representing a hash function.
In the embodiment, after the client adds noise to the local model (or further adds model compression), the noise-added model is encrypted by the encryption parameter sent by the server according to the encryption algorithm, so that the problem of generating, managing and distributing keys depending on a trusted third party entity is effectively solved, and the reliable guarantee is provided for the privacy protection of the client model.
And S14, sending the encryption model, the partial decryption key and the model accuracy to the server so that the server can decrypt and aggregate the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model.
The global model is obtained by decrypting and aggregating encryption models which are locally trained and uploaded by all the clients through the server, and the aggregation method can be selected according to actual requirements. In this embodiment, the model is obtained by a method of performing weighted average aggregation on local models uploaded by all clients based on client weights, and is used as a model which is sent to all clients for training in subsequent iterations. When a part of decryption keys are used for the server to aggregate the encryption models of all the clients, the decryption keys need to be combined with the weight vectors corresponding to the encryption weights of all the clients, the corresponding decryption keys are calculated through a key combination algorithm, and then the decryption keys and the encryption labels are used to decrypt and aggregate the encryption models of all the clients through a decryption algorithm, as shown in fig. 5, the method specifically comprises the following steps:
s141, obtaining a decryption key by adopting a key combination algorithm according to the encryption weight and part of the decryption key; the decryption key is represented as:
in the formula,
wherein,andrespectively representing a weight vector composed of a decryption key and all client encryption weights;a partial decryption key representing the ith client;
s142, carrying out decryption aggregation on the encryption models of all the clients according to the decryption key and the encryption label to obtain the global model; the global model is represented as:
in the formula,
wherein, [ alpha ] is] T Representing a global model; y is i And [ c) i ] 1 Respectively representing the encryption weight and the encryption model of the ith client;representing a decryption key;representing an encrypted tag;represents a multiplicative cyclic group of order cryptographic prime number associated with a bilinear pair, andrepresenting groups of cycles by multiplicationA constructed 2-dimensional vector;representing a hash function; e (-) represents a bilinear pairwise map.
After the server obtains the global model of the current round of iterative training through the decryption aggregation method, the server can directly issue the global model to all the clients to continue subsequent iterative training in principle under the condition that the preset iteration times or other preset iteration convergence standards are not met. In order to ensure that the encryption weights subsequently distributed to the clients are more reasonable and effective, in this embodiment, after the server obtains the global model, the server uses a public data set to test the global model, and when the accuracy rate does not meet the requirement, the server re-adjusts the weights of all the clients according to the accuracy rate of the local model obtained by the current round of iteration sent by each client, that is, the server distributes higher encryption weights to the clients with high accuracy rate of the current round of training model for encryption and aggregation of the next round of training model, and each round of iterative training executes the same method steps. Specifically, in addition to the above S141-S142, as shown in fig. 6, the step of performing decryption aggregation on the encryption model by the server according to the partial decryption key, the encryption tag, the encryption weight, and the model accuracy to obtain a global model further includes:
s143, testing the global model according to the public data set to obtain the accuracy of the global model;
s144, judging whether the accuracy of the global model reaches a preset accuracy, if so, stopping iterative training, otherwise, updating the encryption weight and the weight vector parameter of each client according to the model accuracy of all the clients, sending the global model and the updated encryption weight and weight vector parameter to each client, and continuing iterative training.
The weight vector parameter is generated by the server according to the encryption weight of each client, is updated along with the change of the encryption weight of each client, and can be expressed as:
wherein,representing a weight vector parameter;representing a hash function;represents a cyclic group of p-factorial method relating to bilinear pairs, anda 2-dimensional vector composed of a multiplication loop group is represented.
In the embodiment of the application, based on the problem that the privacy protection of the existing federal learning relies on the generation, management and distribution of keys of a trusted third party entity, the consideration of collusion attack risk of the trusted third party and a server is easy to bring, a decentralized function encryption privacy protection method facing the federal learning is designed, the method achieves that an encryption label, an encryption prime number and an encryption weight are generated by the server in advance, an initial model is obtained according to public data set training, then the initial model, the public data set, the encryption label, the encryption prime number, the encryption weight and weight vector parameters are sent to each client, each client trains the initial model according to the local data set to obtain a local model, tests according to the public data set to obtain model accuracy, an encryption private key and a partial decryption key are generated according to the encryption prime number, local model is subjected to noise adding processing according to local differential privacy, the local model is subjected to function encryption according to the encryption private key and the encryption label to obtain an encryption model, the partial decryption key and the model accuracy are sent to the server, the server is judged, and whether the global model is required to be subjected to iteration, and whether the global model is judged, and whether the global data is required to be tested continuously. The method is applied to actual federal learning training, a method of combining a method of generating a secret key by interaction of a server and a client with local differential privacy is adopted, the server is guaranteed not to obtain specific gradient parameters of a local training model of each user, the problem that the existing privacy protection method depends on a trusted third party entity is solved, the risk that the trusted third party and the server execute collusion attack is effectively prevented, and the privacy protection degree and the model service effect of the client model in the federal learning are further improved.
It should be noted that, although the steps in the above-mentioned flowcharts are shown in sequence as indicated by arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders.
In one embodiment, as shown in fig. 7, there is provided a federally-learned decentralized function encryption privacy protection system, the system comprising:
the acquisition module 1 is used for acquiring an initial model, a public data set, an encryption tag, an encryption prime number, an encryption weight and a weight vector parameter which are sent by a server; the initial model is obtained by the server through training according to the public data set;
the training module 2 is used for training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain the corresponding model accuracy;
the encryption module 3 is used for generating an encryption private key and a part of decryption keys according to the encryption prime number, the encryption weight and the weight vector parameters, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
and the aggregation module 4 is configured to send the encryption model, the partial decryption key, and the model accuracy to the server, so that the server performs decryption aggregation on the encryption model according to the partial decryption key, the encryption tag, the encryption weight, and the model accuracy, to obtain a global model.
For specific limitations on the encryption privacy protection of the decentralization function for federal learning, refer to the above limitations on the encryption privacy protection method of the decentralization function for federal learning, which are not described herein again. The modules in the Federal learning-oriented decentralized function encryption privacy protection system can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
As shown in fig. 8, in an intelligent medical scenario including hospitals and diagnosis servers, the system is used to complete safe federal learning of each hospital database, so as to implement accurate diagnosis of online diseases, and the specific application is as follows: the online diagnosis server obtains an initial model according to the training of the public data set, and simultaneously generates an encryption label l, an encryption prime number P and an encryption weight y i And weight vector parametersAnd simultaneously sending the initial model and the initial model to each hospital; each hospital receives the initial model, the public data set, the encryption label l, the encryption prime number P and the encryption weight y sent by the server i And weight vector parametersThen, training the initial model according to the local data set to obtain a local model, testing the local model according to the public data set to obtain corresponding model accuracy, executing local differential privacy according to the distribution characteristics and privacy protection requirements of the local model to add noise to the local model, and generating an encryption private key according to an encryption prime number PAnd partial decryption keyAnd based on the encrypted private keyAnd the encryption label l performs function encryption on the local model to obtain an encryption model, and the encryption model and a part of decryption key are usedAnd the model accuracy is sent to an online diagnosis server; the on-line diagnosis server decrypts the secret key according to the part transmitted by each hospitalAnd an encryption weight y i Obtaining a decryption key using a key combination algorithmBased on the decryption keyAnd the encryption label l is used for decrypting and aggregating the encryption models of all the clients to obtain a global model, testing the global model according to the public data set to obtain the accuracy of the global model, updating the encryption weight and the weight vector parameter of each client according to the model accuracy of all the clients when the accuracy of the global model is judged not to reach the preset accuracy, and updating the encryption weight and the weight vector parameter of each client according to the model accuracy of all the clientsSending the global model, the updated encryption weight and the updated weight vector parameters to each client, and continuing iterative training until an ideal global model is obtained; the online diagnosis server provides service by using the model trained by federal learning, and after the personal health data is uploaded to the online diagnosis server by an individual user, the online diagnosis server inputs the personal data of the user into the global model for disease matching and feeds back the online diagnosis result to the individual user in time.
Fig. 9 shows an internal structure diagram of a computer device in one embodiment, and the computer device may be specifically a terminal or a server. As shown in fig. 9, the computer apparatus includes a processor, a memory, a network interface, a display, and an input device, which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a federated learning-oriented decentralized function privacy protection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 9 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown in its own right, or may combine certain components, or have an arrangement of components in common.
In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the steps of the above method being performed when the computer program is executed by the processor.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method.
To sum up, the federal learning oriented decentralized function encryption privacy protection method and system provided by the embodiments of the present invention generate an encryption tag, an encryption prime number and an encryption weight by a server, train according to a public data set to obtain an initial model, then send the initial model, the public data set, the encryption tag, the encryption prime number, the encryption weight and weight vector parameters to each client, train the initial model according to the local data set by each client to obtain a local model, test according to the public data set to obtain model accuracy, generate an encryption private key and a partial decryption key according to the encryption prime number, perform function encryption on the local model according to the encryption private key and the encryption tag to obtain an encryption model, then send the encryption model, the partial decryption key and the model accuracy to the server, so that the server completes decryption aggregation of the encryption models of each client to obtain a global model, test the accuracy of the global model by using the public data set, and determine whether to continue training until an ideal global model meeting iterative service requirements is obtained. The decentralization function encryption privacy protection method for federal learning ensures that a server cannot obtain specific gradient parameters of a local training model of each user, overcomes the problem that the existing privacy protection method depends on a trusted third party entity to generate, manage and distribute keys in a mode that a server and a client interact to generate the keys, effectively prevents the trusted third party and the server from executing collusion attack risk, and further improves the privacy protection degree and the model service effect of the client model in federal learning.
The embodiments in this specification are described in a progressive manner, and all the same or similar parts of the embodiments are directly referred to each other, and each embodiment is described with emphasis on differences from other embodiments. In particular, as for the system embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points. It should be noted that, the technical features of the embodiments may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several preferred embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for those skilled in the art, various modifications and substitutions can be made without departing from the technical principle of the present invention, and these should be construed as the protection scope of the present application. Therefore, the protection scope of the present patent application shall be subject to the protection scope of the claims.
Claims (8)
1. A Federal learning-oriented decentralized function encryption privacy protection method is characterized by comprising the following steps:
acquiring an initial model, a public data set, an encryption label, an encryption prime number, an encryption weight and a weight vector parameter which are sent by a server; the initial model is obtained by the server through training according to the public data set;
training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain corresponding model accuracy;
generating an encryption private key and a partial decryption key according to the encryption prime number, the encryption weight and the weight vector parameter, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
sending the encryption model, the partial decryption key and the model accuracy to the server, so that the server decrypts and aggregates the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model;
wherein, the step of generating an encryption private key and a part of decryption keys according to the encryption prime number, the encryption weight and the weight vector parameters comprises:
generating a key parameter according to the encryption prime number, and taking the key parameter as the encryption private key; the key parameter is represented as:
in the formula, p andrespectively representing encrypted prime numbers and corresponding finite fields;the key parameter of the ith client is represented and is a 2-dimensional vector;
generating a decryption parameter according to the encryption prime number, and generating the partial decryption key according to the decryption parameter, the key parameter, the encryption weight and the weight vector parameter; the partial decryption key is represented as:
in the formula,
wherein,and y i Respectively representing a part of decryption keys and encryption weights of the ith client; t is a unit of i Represents the decryption parameter generated by the ith client, and sigma i∈[n] T i =0,n is the total number of participating training clients;representing a hash function;represents a cyclic group of p-factorial method associated with bilinear pairs, anRepresenting groups of multiplication cyclesA constructed 2-dimensional vector;andrespectively representing weight vectors formed by all client encryption weights and corresponding weight vector parameters;
the step of generating the decryption parameter according to the encrypted prime number comprises the following steps:
initializing a parameter matrix; the parameter matrix is a matrix with all elements being zero;
according to the encrypted prime numbers, negotiating with other clients respectively to determine a corresponding random matrix; the random matrix is represented as:
wherein, T ij Representing the random moment determined by the negotiation between the ith client and the jth clientArraying;representing a finite fieldA constructed 2 × 2 matrix;
generating the decryption parameters according to the parameter matrix and the random matrix; the decryption parameters are expressed as:
T i =T 0 +Σ j∈n,i<j T ij -Σ j∈n,j<i T ij
wherein, T i Representing a decryption parameter corresponding to the ith client; t is a unit of 0 A parameter matrix representing the client.
2. The decentralized function encryption privacy protection method according to claim 1, wherein the step of training the initial model according to the local data set to obtain the local model comprises:
setting a privacy budget and a noise parameter according to the distribution characteristics and privacy protection requirements of the local model;
and according to the privacy budget and the noise parameter, performing localized differential privacy to add noise to the local model.
3. The decentralized function encryption privacy protection method according to claim 1, wherein the step of performing function encryption on the local model according to the encryption private key and the encryption tag to obtain an encryption model comprises:
according to the encryption private key and the encryption label, performing function encryption on the local model by adopting the following encryption algorithm to obtain the encryption model:
in the formula,
wherein x is i And [ c) i ] 1 Respectively representing an ith client local model and an encryption model;a key parameter representing an ith client; l represents an encryption tag;represents a multiplicative cyclic group of the order of the cryptographic prime number associated with the bilinear pair, andrepresenting groups of cycles by multiplicationA constructed 2-dimensional vector;representing a hash function.
4. The decentralized function encryption privacy protection method according to claim 1, wherein the step of the server performing decryption aggregation on the encryption model according to the partial decryption key, the encryption tag, the encryption weight and the model accuracy to obtain a global model comprises:
obtaining a decryption key by adopting a key combination algorithm according to the encryption weight and part of the decryption key; the decryption key is represented as:
in the formula,
wherein,andrespectively representing a weight vector composed of a decryption key and all client encryption weights;a partial decryption key representing the ith client;
decrypting and aggregating encryption models of all clients according to the decryption key and the encryption label to obtain the global model; the global model is represented as:
in the formula,
wherein, [ alpha ] is] T Representing a global model; y is i And [ c) i ] 1 Respectively representing the encryption weight and the encryption model of the ith client;representing a decryption key; l represents an encryption tag;represents a multiplicative cyclic group of the order of the cryptographic prime number associated with the bilinear pair, andrepresenting groups of multiplication cyclesA constructed 2-dimensional vector;representing a hash function; e (-) represents a bilinear pair map.
5. The decentralized function encryption privacy protection method according to claim 1, wherein the step of the server performing decryption aggregation on the encryption model according to the partial decryption key, the encryption tag, the encryption weight and the model accuracy to obtain the global model further comprises:
testing the global model according to the public data set to obtain the accuracy of the global model;
and judging whether the accuracy of the global model reaches a preset accuracy, if so, stopping iterative training, otherwise, updating the encryption weight and weight vector parameter of each client according to the model accuracy of all clients, sending the global model and the updated encryption weight and weight vector parameter to each client, and continuing iterative training.
6. A federally-learned-oriented decentralized function privacy protection system capable of executing the federally-learned-oriented decentralized function privacy protection method according to any one of claims 1 to 5, the system comprising:
the acquisition module is used for acquiring the initial model, the public data set, the encryption label, the encryption prime number, the encryption weight and the weight vector parameter sent by the server; the initial model is obtained by the server through training according to the public data set; the encryption parameters comprise encryption labels, encryption prime numbers and encryption weights;
the training module is used for training the initial model according to a local data set to obtain a local model, and testing the local model according to the public data set to obtain corresponding model accuracy;
the encryption module is used for generating an encryption private key and a part of decryption keys according to the encryption prime number, the encryption weight and the weight vector parameters, and performing function encryption on the local model according to the encryption private key and the encryption label to obtain an encryption model;
and the aggregation module is used for sending the encryption model, the partial decryption key and the model accuracy to the server so that the server can decrypt and aggregate the encryption model according to the partial decryption key, the encryption label, the encryption weight and the model accuracy to obtain a global model.
7. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method as claimed in any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111134122.6A CN113836556B (en) | 2021-09-26 | 2021-09-26 | Federal learning-oriented decentralized function encryption privacy protection method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111134122.6A CN113836556B (en) | 2021-09-26 | 2021-09-26 | Federal learning-oriented decentralized function encryption privacy protection method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113836556A CN113836556A (en) | 2021-12-24 |
CN113836556B true CN113836556B (en) | 2022-11-04 |
Family
ID=78970549
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111134122.6A Active CN113836556B (en) | 2021-09-26 | 2021-09-26 | Federal learning-oriented decentralized function encryption privacy protection method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113836556B (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114362940B (en) * | 2021-12-29 | 2023-09-29 | 华东师范大学 | Server-free asynchronous federation learning method for protecting data privacy |
CN114726496B (en) * | 2022-03-07 | 2023-10-03 | 电子科技大学 | Safe feature selection method applied to longitudinal federal learning |
CN115130814B (en) * | 2022-05-10 | 2023-05-02 | 中南大学 | Privacy computing method and system for longitudinal data fusion |
CN115828287B (en) * | 2023-01-10 | 2023-05-23 | 湖州丽天智能科技有限公司 | Model encryption method, model decryption method, computer and integrated chip |
CN116010944B (en) * | 2023-03-24 | 2023-06-20 | 北京邮电大学 | Federal computing network protection method and related equipment |
CN116168789B (en) * | 2023-04-26 | 2023-07-18 | 之江实验室 | Multi-center medical data generation system and method |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110572253B (en) * | 2019-09-16 | 2023-03-24 | 济南大学 | Method and system for enhancing privacy of federated learning training data |
CN111600707B (en) * | 2020-05-15 | 2023-04-14 | 华南师范大学 | Decentralized federal machine learning method under privacy protection |
CN113434873A (en) * | 2021-06-01 | 2021-09-24 | 内蒙古大学 | Federal learning privacy protection method based on homomorphic encryption |
-
2021
- 2021-09-26 CN CN202111134122.6A patent/CN113836556B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN113836556A (en) | 2021-12-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113836556B (en) | Federal learning-oriented decentralized function encryption privacy protection method and system | |
Wang et al. | Privacy-preserving federated learning for internet of medical things under edge computing | |
CN108712260B (en) | Multi-party deep learning computing agent method for protecting privacy in cloud environment | |
Zhang et al. | Homomorphic encryption-based privacy-preserving federated learning in IoT-enabled healthcare system | |
Xu et al. | VerifyNet: Secure and verifiable federated learning | |
CN110245510B (en) | Method and apparatus for predicting information | |
Liu et al. | Privacy-preserving aggregation in federated learning: A survey | |
US20210143987A1 (en) | Privacy-preserving federated learning | |
US20230108682A1 (en) | Data processing method and apparatus, device, and computer-readable storage medium | |
CN112966298B (en) | Composite privacy protection method, system, computer equipment and storage medium | |
JP2020515087A5 (en) | ||
CN112347500B (en) | Machine learning method, device, system, equipment and storage medium of distributed system | |
Lyu et al. | Towards fair and decentralized privacy-preserving deep learning with blockchain | |
CN112364376A (en) | Attribute agent re-encryption medical data sharing method | |
CN111027981A (en) | Method and device for multi-party joint training of risk assessment model for IoT (Internet of things) machine | |
CN113747426B (en) | Data auditing method and system, electronic equipment and storage medium | |
CN114003950A (en) | Federal machine learning method, device, equipment and medium based on safety calculation | |
CN110752024A (en) | Online medical diagnosis service system based on privacy protection | |
Sun et al. | A privacy-aware and traceable fine-grained data delivery system in cloud-assisted healthcare IIoT | |
Chang et al. | Secure medical data management with privacy-preservation and authentication properties in smart healthcare system | |
CN113849828B (en) | Anonymous generation and attestation of processed data | |
CN116502732B (en) | Federal learning method and system based on trusted execution environment | |
CN116996215A (en) | Key negotiation system and method under medical digital twin environment | |
Zhou et al. | VDFChain: Secure and verifiable decentralized federated learning via committee-based blockchain | |
Alowolodu et al. | Medical image security using quantum cryptography |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |