CN116451275B - Privacy protection method based on federal learning and computing equipment - Google Patents

Privacy protection method based on federal learning and computing equipment Download PDF

Info

Publication number
CN116451275B
CN116451275B CN202310706551.9A CN202310706551A CN116451275B CN 116451275 B CN116451275 B CN 116451275B CN 202310706551 A CN202310706551 A CN 202310706551A CN 116451275 B CN116451275 B CN 116451275B
Authority
CN
China
Prior art keywords
model
aggregation
local
update
protection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310706551.9A
Other languages
Chinese (zh)
Other versions
CN116451275A (en
Inventor
王志强
于欣月
薛培阳
罗乐琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING ELECTRONIC SCIENCE AND TECHNOLOGY INSTITUTE
Original Assignee
BEIJING ELECTRONIC SCIENCE AND TECHNOLOGY INSTITUTE
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING ELECTRONIC SCIENCE AND TECHNOLOGY INSTITUTE filed Critical BEIJING ELECTRONIC SCIENCE AND TECHNOLOGY INSTITUTE
Priority to CN202310706551.9A priority Critical patent/CN116451275B/en
Publication of CN116451275A publication Critical patent/CN116451275A/en
Application granted granted Critical
Publication of CN116451275B publication Critical patent/CN116451275B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/007—Encryption, En-/decode, En-/decipher, En-/decypher, Scramble, (De-)compress
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

The invention provides a privacy protection method and computing equipment based on federal learning, and relates to the technical field of computer application. The method comprises the following steps: acquiring a local model update and a global model; determining a direction consistency coefficient according to the local model update and the global model; respectively comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value, and determining a marker of local model update and a protection mode of uploading the local model update to a server according to a comparison result; receiving a first aggregation model and a second aggregation model issued by a server; the first aggregation model and the second aggregation model are determined according to the marker, the first aggregation model needs to be decrypted, and the second aggregation model does not need to be decrypted; and carrying out decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating. The beneficial effects of this technical scheme are: the federal learning training time and the traffic are reduced on the premise of ensuring the training safety and the model performance.

Description

Privacy protection method based on federal learning and computing equipment
Technical Field
The invention relates to the technical field of computer application, in particular to a privacy protection method and computing equipment based on federal learning.
Background
Federal learning (FL, federated Learning), which is a distributed machine learning and model training technique, in the distributed learning process, each role involved in learning does not exchange its own individual sample and data field, and only completes the whole model training process by exchanging model parameters or intermediate results, thereby completing a series of business model creation and optimization requirements through sample or feature expansion.
Privacy protection of federal learning is one of the key research problems of federal learning, and gradient data and the like in the exchange process are protected by using privacy protection methods such as differential privacy, homomorphic encryption, multi-party security calculation and the like. In the prior art, the adoption of differential privacy to add noise easily causes the reduction of model performance, the adoption of homomorphic encryption calculation amount is large, the training time is long, and the adoption of multiparty security calculation requires a plurality of servers, thereby easily causing a large amount of communication overhead.
Disclosure of Invention
The invention solves the problem of how to reduce the federal learning training time and the traffic on the premise of ensuring the training safety and the model performance.
In order to solve the above problems, the present invention provides a privacy protection method based on federal learning, including:
acquiring a local model update and a global model;
determining a direction consistency coefficient according to the local model update and the global model;
comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value respectively, and determining a marker of the local model update and a protection mode of uploading the local model update to a server according to a comparison result; wherein the protection threshold is greater than the discard threshold;
receiving a first aggregation model and a second aggregation model issued by the server; wherein the first aggregate model and the second aggregate model are determined according to the marker, the first aggregate model needs decryption, and the second aggregate model does not need decryption;
and carrying out decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating.
The beneficial effects of the invention are as follows: updating and globally modeling by acquiring a local model; and determining a direction consistency coefficient according to the local model update and the global model. The direction consistency coefficient is used for judging the degree that the updating direction of the local model meets the requirement. The way in which the local model updates the upload server can be determined from the direction consistency coefficients. Comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value respectively, and determining a marker of the local model update and a protection mode of uploading the local model update to a server according to a comparison result; wherein the protection threshold is greater than the discard threshold. And comparing the direction consistency coefficient with the discarding threshold value and the protection threshold value to determine a protection mode of uploading the local model update to the server, wherein the local model update can be uploaded in a mode with better protection performance when the direction consistency coefficient is higher, and when the direction consistency coefficient belongs to a medium interval range, the efficiency of uploading the local model update to the server can be improved by using a method which is easier to realize on the premise of ensuring the safety of model data. Receiving a first aggregation model and a second aggregation model issued by the server; wherein the first aggregate model and the second aggregate model are determined according to the marker, the first aggregate model requires decryption, and the second aggregate model does not require decryption. And in the process of issuing the models, a classification aggregation mode is adopted, the aggregation mode is determined to be a first aggregation model or a second aggregation model according to the marker, and the first aggregation model and the second aggregation model are issued to the client together, so that hierarchical aggregation is realized, and server resources can be saved. And carrying out decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating. Because the first aggregation model needs to be decrypted and the second aggregation model does not need to be decrypted, the global model update obtained through classified uploading and classified aggregation can reduce training time and traffic on the premise of guaranteeing training safety and model performance.
Optionally, the determining the direction consistency coefficient according to the local model update and the global model includes:
determining the direction consistency coefficient through a first formula; wherein the first formula comprises:
in the method, in the process of the invention,for the number of identical symbols between the local model update and the global model,and the number of all symbols of the global model.
Optionally, the comparing the direction consistency coefficient with a preset discard threshold and a preset protection threshold respectively, and determining the marker of the local model update and the protection mode of uploading the local model update to the server according to the comparison result includes:
discarding the local model update and resetting its option to zero if the direction consistency coefficient is less than the discard threshold, without setting the marker;
if the direction consistency coefficient is greater than or equal to the protection threshold, the local model update is protected through homomorphic encryption and uploaded to a server, and the marker is set to be 1;
and if the direction consistency coefficient is larger than or equal to the discarding threshold value and smaller than the protection threshold value, updating the local model to a server through differential privacy protection, and setting the marker as 2.
Optionally, the protecting the local model update by homomorphic encryption includes:
protecting the local model update according to a second formula, wherein the second formula comprises:
in the method, in the process of the invention,updating protection for local model, +.>For the key used in said homomorphic encryption, -/-, for the encryption>For a single weighted scalar variable, +.>Updating,/for the local model of the ith client>For the direction consistency coefficient of the ith client,/th client>Is the protection threshold.
Optionally, the protecting the local model update by differential privacy includes protecting the local model update according to a third formula, wherein the third formula includes:
in the method, in the process of the invention,updating protection for local model, +.>Updating for the local model of the ith client,algorithm for adding noise using Laplace mechanism, +.>For the direction consistency coefficient of the ith client,/th client>For the discard threshold, +_>Is the protection threshold.
Optionally, the first aggregation model and the second aggregation model are determined according to the marker, including:
if the marker is 1, aggregating the local model update of the homomorphic encryption protection in a ciphertext model form to obtain the first aggregation model;
and if the marker is 2, aggregating the local model update of the differential privacy protection in the form of added noise to obtain the second aggregation model.
Optionally, the aggregating the local model updates of the homomorphic encryption protection in the form of a ciphertext model to obtain the first aggregate model includes:
obtaining the first aggregation model according to a fourth formula, wherein the fourth formula comprises:
in the method, in the process of the invention,for the first aggregation model, +.>Protecting the number of clients updated for the local model using the homomorphic encryption,/->For the key used in said homomorphic encryption, -/-, for the encryption>For a single weighted scalar variable, +.>Updating for said local model of the z-th client,/and>is the marker;
said aggregating said local model updates of said differential privacy protection in the form of added noise to obtain said second aggregate model, comprising:
obtaining the second aggregate model according to a fifth formula, wherein the fifth formula comprises:
in the method, in the process of the invention,for the second aggregation model, +.>Updating,/for the local model of the jth client>-number of clients updated for protecting the local model using the differential privacy +.>Algorithm for adding noise using Laplace mechanism, +.>Is the marker.
Optionally, the performing decryption aggregation on the first aggregation model and the second aggregation model to obtain global model update includes:
decrypting the first aggregation model according to the decryption algorithm corresponding to homomorphic encryption to obtain a decryption model;
and aggregating the decryption model and the second aggregation model to obtain the global model update.
Optionally, the decrypting the first aggregation model according to the decryption algorithm corresponding to the homomorphic encryption to obtain a decryption model includes:
obtaining the decryption model according to a sixth formula, wherein the sixth formula comprises:
in the method, in the process of the invention,for the decryption model->-number of clients updated for protecting the local model using the differential privacy +.>For the first aggregation model, +.>For a single weighted scalar variable;
the step of aggregating the decryption model and the second aggregation model to obtain the global model update includes:
obtaining the global model update according to a seventh formula, wherein the seventh formula comprises:
in the method, in the process of the invention,updating for said global model,/->For the decryption model->For the second aggregation model, +.>The number of clients participating in training.
The invention also provides a computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor implements a federal learning-based privacy protection method as described above.
The computing device of the present invention has the same advantages as the federal learning-based privacy protection method described above relative to the prior art, and is not described in detail herein.
Drawings
Fig. 1 is a schematic flow chart of a privacy protection method based on federal learning according to an embodiment of the present invention;
fig. 2 is a second flowchart of a federal learning-based privacy protection method according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
It is noted that the terms "first," "second," and the like in the description and claims of the invention and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein.
In the description of the present specification, reference to the terms "embodiment," "some embodiments," and "optionally embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or implementation is included in at least one embodiment or illustrated implementation of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same examples or implementations. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or implementations.
Referring to fig. 1 and 2, an embodiment of the present invention provides a privacy protection method based on federal learning, including the steps of:
s1, acquiring local model updating and a global model;
the local model update refers to the update of a single client to the global model in one training, and the global model can be an initial global model or a global model update obtained by the client in the last training, and is applicable according to the condition whether the initial training is performed or not. And if the client is in primary training, applying an initial global model, and if the client is not in primary training, applying global model updating obtained in the last training.
S2, determining a direction consistency coefficient according to the local model update and the global model;
and determining a direction consistency coefficient according to the local model update and the global model, namely judging the problem of the direction consistency of the local model update and the previous global model in the training process, and avoiding great update deviation.
S3, comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value respectively, and determining a marker of the local model update and a protection mode of uploading the local model update to a server according to a comparison result; wherein the protection threshold is greater than the discard threshold;
the discarding threshold and the protecting threshold are manually set according to actual needs, wherein the protecting threshold is larger than the discarding threshold.
S4, receiving a first aggregation model and a second aggregation model which are issued by the server; wherein the first aggregate model and the second aggregate model are determined according to the marker, the first aggregate model needs decryption, and the second aggregate model does not need decryption;
the server updates the local models uploaded by a plurality of clients, determines to aggregate into a first aggregation model or a second aggregation model according to the markers, and transmits all the first aggregation model and the second aggregation model obtained by the round of aggregation to the clients for training in the next round, namely, each client for training in the next round can receive the complete first aggregation model and second aggregation model.
S5, performing decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating.
And the client side needing the next round of training receives the first aggregation model and the second aggregation model and then carries out decryption aggregation to obtain global model updating of the round.
It should be noted that, the local model update refers to the global model update or the initial global model update performed by the single client of the previous training round. The global model update refers to a new global model obtained by the client-side performing decryption aggregation on the first aggregation model and the second aggregation model issued by the server.
It should be further noted that, as can be appreciated in connection with the basic knowledge of federal learning, the number of clients in federal learning is plural, the plural clients are in communication with the server, the plural clients upload the local model updates to the server, and the server issues the first aggregate model and the second aggregate model to the plural clients. The privacy protection method based on federal learning in the embodiment of the present invention only describes one round of uploading of the local model update and issuing of the first aggregation model and the second aggregation model, and in practical application, the uploading and issuing processes need to be repeatedly trained for multiple times until the models reach convergence conditions.
In this embodiment, local model updates and global models are obtained; and determining a direction consistency coefficient according to the local model update and the global model. The direction consistency coefficient is used for judging the degree that the updating direction of the local model meets the requirement. The way in which the local model updates the upload server can be determined from the direction consistency coefficients. Comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value respectively, and determining a marker of the local model update and a protection mode of uploading the local model update to a server according to a comparison result; wherein the protection threshold is greater than the discard threshold. And comparing the direction consistency coefficient with the discarding threshold value and the protection threshold value to determine a protection mode of uploading the local model update to the server, wherein the local model update can be uploaded in a mode with better protection performance when the direction consistency coefficient is higher, and when the direction consistency coefficient belongs to a medium interval range, the efficiency of uploading the local model update to the server can be improved by using a method which is easier to realize on the premise of ensuring the safety of model data. Receiving a first aggregation model and a second aggregation model issued by the server; wherein the first aggregate model and the second aggregate model are determined according to the marker, the first aggregate model requires decryption, and the second aggregate model does not require decryption. And in the process of issuing the models, a classification aggregation mode is adopted, the aggregation mode is determined to be a first aggregation model or a second aggregation model according to the marker, and the first aggregation model and the second aggregation model are issued to the client together, so that hierarchical aggregation is realized, and server resources can be saved. And carrying out decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating. Because the first aggregation model needs to be decrypted and the second aggregation model does not need to be decrypted, the global model update obtained through classified uploading and classified aggregation can reduce training time and traffic on the premise of guaranteeing training safety and model performance.
In another alternative embodiment of the present invention, the determining the direction consistency coefficient according to the local model update and the global model includes:
determining the direction consistency coefficient through a first formula; wherein the first formula comprises:
in the method, in the process of the invention,for the number of identical symbols between the local model update and the global model,and the number of all symbols of the global model.
Specifically, in this embodiment, the symbols refer to the signs of the vector space coordinates in the local model update and the global model. Illustratively, the local model update is (-1, 2, -3), the global model is (1, 2, -1), then the sign of the local model update is (-, +, -), and the global model sign is (+, +, -). The symbols of the last two bits are the same, namely the number of the same symbols between the local model update and the global model is 2, the number of all symbols of the global model is 3, and the direction consistency coefficient is 2/3.
In this embodiment, the local model updates and the global model updates may be determined to adopt different uploading manners for different local model updates according to the same proportion of the local model updates and the global model symbols.
In another alternative embodiment of the present invention, as shown in fig. 2, the comparing the direction consistency coefficient with a preset discard threshold and a preset protection threshold respectively, and determining the marker of the local model update and the protection manner of uploading the local model update to the server according to the comparison result includes:
discarding the local model update and resetting its option to zero if the direction consistency coefficient is less than the discard threshold, without setting the marker;
if the direction consistency coefficient is greater than or equal to the protection threshold, the local model update is protected through homomorphic encryption and uploaded to a server, and the marker is set to be 1;
and if the direction consistency coefficient is larger than or equal to the discarding threshold value and smaller than the protection threshold value, updating the local model to a server through differential privacy protection, and setting the marker as 2.
In this embodiment, if the direction consistency coefficient is smaller than the discard threshold, the local model update is not transmitted, so that the security of the local model update in the transmission process is directly ensured.
If the direction consistency coefficient is greater than the protection threshold, the local model update is protected in a homomorphic encryption mode, and under the premise that the secret key is not transmitted, an attacker cannot acquire the secret key in the transmission process, and the local model update transmitted in the round is difficult to decrypt, so that the safety of the local model update in the transmission process is ensured.
And if the direction consistency coefficient is larger than or equal to the discarding threshold value and smaller than the protection threshold value, the local model update is protected by adopting a differential privacy mode. The manner of differential privacy may include, but is not limited to: laplace mechanism, exponential mechanism, gaussian mechanism, etc. The laplace mechanism is preferably used in this embodiment. Satisfaction using the Laplace noise mechanismDifferential privacy mechanisms, i.e. adding noise, enhance the protection of local model updates during transmission.
According to the judging conditions, three conditions are independent, so that the safety of the local model update is protected in the process of uploading the server.
It should be noted that, in one training, the client that the client uploads the local model update and the client that receives and issues the first aggregation model and the second aggregation model are different clients. The option is reset to zero, which means that in the next round of training, the client with the option reset to zero is not selected. The method does not set a marker, namely, the marker is not added in the model. For example, the clients uploading the local model updates are numbered 1,2, 3, 4 and 5, if the selection right of the client numbered 3 is reset to zero, when the first aggregation model and the second aggregation model are issued, the client numbered 3 is not selected, and the first aggregation model and the second aggregation model are issued to the clients numbered 1,2, 4 and 5.
Optionally, as shown in fig. 2, if the direction consistency coefficient is greater than or equal to the protection threshold, the local model update is protected by homomorphic encryption and uploaded to a server, and the selection weight of the corresponding client is increased; and if the direction consistency coefficient is larger than or equal to the discarding threshold value and smaller than the protection threshold value, protecting the local model from updating to a server through differential privacy, and reducing the selection weight of the corresponding client.
The increasing selection weight of the corresponding client indicates an increase in the probability that the client is selected in the next round of training, and the decreasing selection weight of the corresponding client indicates a decrease in the probability that the client is selected in the next round of training.
In another alternative embodiment of the present invention, said protecting said local model update by homomorphic encryption comprises:
protecting the local model update according to a second formula, wherein the second formula comprises:
in the method, in the process of the invention,updating protection for local model, +.>For the key used in said homomorphic encryption, -/-, for the encryption>For a single weighted scalar variable, +.>Updating,/for the local model of the ith client>For the direction consistency coefficient of the ith client,/th client>Is the protection threshold. The single weighted scalar variable is a fixed value in the homomorphic encryption algorithm.
It should be noted that, the local model update protection refers to data for uploading to a server, which adds protection to the local model update.
In the present embodiment, the key is passed throughAnd encrypting the local model update, so that the safety of the transmission process is protected.
In another alternative embodiment of the present invention, said protecting said local model updates by differential privacy comprises: protecting the local model update according to a third formula, wherein the third formula comprises:
in the method, in the process of the invention,updating protection for local model, +.>Updating for the local model of the ith client,algorithm for adding noise using Laplace mechanism, +.>For the direction consistency coefficient of the ith client,/th client>For the discard threshold, +_>Is the protection threshold.
It should be noted that, the local model update protection refers to data for uploading to a server, which adds protection to the local model update.
In this embodiment, noise is introduced by using an algorithm of adding noise by using a laplace mechanism, so that the original local model update is protected.
In another alternative embodiment of the present invention, as shown in connection with fig. 2, the first aggregation model and the second aggregation model are determined according to the marker, comprising:
if the marker is 1, aggregating the local model update of the homomorphic encryption protection in a ciphertext model form to obtain the first aggregation model;
and if the marker is 2, aggregating the local model update of the differential privacy protection in the form of added noise to obtain the second aggregation model.
Specifically, in the present embodiment, the global model is secure in the sort aggregation and the decryption aggregation. And in the process of transmitting the first aggregation model and the second aggregation model by the server, an attacker can only acquire the transmission data of the first aggregation model and the second aggregation model. When an attacker cannot acquire the secret key, the first aggregation model cannot be decrypted, so that the first aggregation model is safe; the addition of noise causes the second polymeric pattern to satisfyDifferential privacy mechanisms and therefore also secure. Therefore, the safety of issuing the first aggregation model and the second aggregation model to the client is ensured.
In another optional embodiment of the present invention, the aggregating the local model updates of the homomorphic encryption protection in the form of a ciphertext model, to obtain the first aggregate model, includes:
obtaining the first aggregation model according to a fourth formula, wherein the fourth formula comprises:
in the method, in the process of the invention,for the first aggregation model, +.>Protecting the number of clients updated for the local model using the homomorphic encryption,/->For the key used in said homomorphic encryption, -/-, for the encryption>Representing the inverse of the matrix->For a single weighted scalar variable, +.>Updating for said local model of the z-th client,/and>for the marker->Representing the local model update uploaded with the homomorphic encryption protection;
said aggregating said local model updates of said differential privacy protection in the form of added noise to obtain said second aggregate model, comprising:
obtaining the second aggregate model according to a fifth formula, wherein the fifth formula comprises:
in the method, in the process of the invention,for the second aggregation model, +.>Updating,/for the local model of the jth client>-number of clients updated for protecting the local model using the differential privacy +.>Adding noise for using Laplace mechanismAcoustic algorithm->For the marker->Representing the local model update uploaded with the differential privacy protection.
In another alternative embodiment of the present invention, as shown in fig. 2, the performing decryption aggregation on the first aggregation model and the second aggregation model to obtain global model update includes:
decrypting the first aggregation model according to the decryption algorithm corresponding to homomorphic encryption to obtain a decryption model;
and aggregating the decryption model and the second aggregation model to obtain the global model update.
Specifically, in this embodiment, when the aggregation is decrypted, the global model update is obtained by decrypting the first aggregation model and then aggregating with the second aggregation model, and if the key cannot be obtained, the first aggregation model cannot be decrypted to obtain the correct global model update. The attacker cannot acquire the secret key, and the secret key only exists in the client side, and the attacker cannot acquire the global model update by intercepting the transmission data of the first aggregation model and the second aggregation model, so that the safety of the global model update is ensured.
In another optional embodiment of the present invention, the decrypting the first aggregation model according to the decryption algorithm corresponding to the homomorphic encryption, to obtain a decryption model, includes:
obtaining the decryption model according to a sixth formula, wherein the sixth formula comprises:
in the method, in the process of the invention,for the decryption model->-number of clients updated for protecting the local model using the differential privacy +.>For the first aggregation model, +.>For a single weighted scalar variable;
the step of aggregating the decryption model and the second aggregation model to obtain the global model update includes:
obtaining the global model update according to a seventh formula, wherein the seventh formula comprises:
in the method, in the process of the invention,updating for said global model,/->For the decryption model->For the second aggregation model, +.>The number of clients participating in training.
The embodiment of the invention also provides a computing device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the privacy protection method based on federal learning.
The computing device according to the embodiment of the present invention has the same advantages as the privacy protection method based on federal learning, compared with the prior art, and is not described in detail herein.
Although the invention is disclosed above, the scope of the invention is not limited thereto. Various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications will fall within the scope of the invention.

Claims (8)

1. A federal learning-based privacy preserving method, comprising:
acquiring a local model update and a global model;
determining a direction consistency coefficient according to the local model update and the global model;
determining the direction consistency coefficient through a first formula; wherein the first formula comprises:
in the method, in the process of the invention,updating the number of identical symbols between the local model and the global model, +.>The number of all symbols of the global model;
comparing the direction consistency coefficient with a preset discarding threshold value and a preset protection threshold value respectively, and determining a marker of the local model update and a protection mode of uploading the local model update to a server according to a comparison result; wherein the protection threshold is greater than the discard threshold;
discarding the local model update and resetting its option to zero if the direction consistency coefficient is less than the discard threshold, without setting the marker;
if the direction consistency coefficient is greater than or equal to the protection threshold, the local model update is protected through homomorphic encryption and uploaded to a server, and the marker is set to be 1;
if the direction consistency coefficient is larger than or equal to the discarding threshold value and smaller than the protection threshold value, the local model is protected to be updated to a server through differential privacy, and the marker is set to be 2;
receiving a first aggregation model and a second aggregation model issued by the server; wherein the first aggregate model and the second aggregate model are determined according to the marker, the first aggregate model needs decryption, and the second aggregate model does not need decryption;
and carrying out decryption aggregation on the first aggregation model and the second aggregation model to obtain global model updating.
2. The federal learning-based privacy protection method according to claim 1, wherein the protecting the local model update by homomorphic encryption comprises:
protecting the local model update according to a second formula, wherein the second formula comprises:
in the method, in the process of the invention,updating protection for local model, +.>For the key used in said homomorphic encryption, -/-, for the encryption>For a single weighted scalar variable, +.>Updating,/for the local model of the ith client>For the direction consistency coefficient of the ith client,/th client>Is the protection threshold.
3. The federal learning-based privacy preserving method of claim 1, wherein the preserving the local model updates by differential privacy comprises preserving the local model updates according to a third formula, wherein the third formula comprises:
in the method, in the process of the invention,updating protection for local model, +.>Updating,/for the local model of the ith client>Algorithm for adding noise using Laplace mechanism, +.>For the direction consistency coefficient of the ith client,/th client>For the discard threshold, +_>Is the protection threshold.
4. The federal learning-based privacy preserving method of claim 1, wherein the first aggregation model and the second aggregation model are determined from the markers, comprising:
if the marker is 1, aggregating the local model update of the homomorphic encryption protection in a ciphertext model form to obtain the first aggregation model;
and if the marker is 2, aggregating the local model update of the differential privacy protection in the form of added noise to obtain the second aggregation model.
5. The federal learning-based privacy protection method of claim 4,
and aggregating the local model updates of the homomorphic encryption protection in the form of a ciphertext model to obtain the first aggregation model, wherein the aggregation model comprises the following components:
obtaining the first aggregation model according to a fourth formula, wherein the fourth formula comprises:
in the method, in the process of the invention,for the first aggregation model, +.>Protecting the number of clients updated for the local model using the homomorphic encryption,/->For the key used in said homomorphic encryption, -/-, for the encryption>For a single weighted scalar variable, +.>Updating for said local model of the z-th client,/and>is the marker;
said aggregating said local model updates of said differential privacy protection in the form of added noise to obtain said second aggregate model, comprising:
obtaining the second aggregate model according to a fifth formula, wherein the fifth formula comprises:
in the method, in the process of the invention,for the second aggregation model, +.>Updating,/for the local model of the jth client>-number of clients updated for protecting the local model using the differential privacy +.>To use the algorithm of the Laplace mechanism to add noise, g is the marker.
6. The federal learning-based privacy preserving method of claim 1, wherein the performing decryption aggregation on the first aggregate model and the second aggregate model results in global model updates, comprising:
decrypting the first aggregation model according to the decryption algorithm corresponding to homomorphic encryption to obtain a decryption model;
and aggregating the decryption model and the second aggregation model to obtain the global model update.
7. The federal learning-based privacy protection method of claim 6,
the decrypting the first aggregation model according to the decryption algorithm corresponding to homomorphic encryption to obtain a decryption model, including:
obtaining the decryption model according to a sixth formula, wherein the sixth formula comprises:
in the method, in the process of the invention,for the decryption model->-number of clients updated for protecting the local model using the differential privacy +.>For the first aggregation model, +.>For a single weighted scalar variable;
the step of aggregating the decryption model and the second aggregation model to obtain the global model update includes:
obtaining the global model update according to a seventh formula, wherein the seventh formula comprises:
in the method, in the process of the invention,updating for said global model,/->For the decryption model->For the second aggregation model, +.>The number of clients participating in training.
8. A computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, which when executed by the processor, implements the federal learning-based privacy protection method of any of claims 1-7.
CN202310706551.9A 2023-06-15 2023-06-15 Privacy protection method based on federal learning and computing equipment Active CN116451275B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310706551.9A CN116451275B (en) 2023-06-15 2023-06-15 Privacy protection method based on federal learning and computing equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310706551.9A CN116451275B (en) 2023-06-15 2023-06-15 Privacy protection method based on federal learning and computing equipment

Publications (2)

Publication Number Publication Date
CN116451275A CN116451275A (en) 2023-07-18
CN116451275B true CN116451275B (en) 2023-08-22

Family

ID=87128819

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310706551.9A Active CN116451275B (en) 2023-06-15 2023-06-15 Privacy protection method based on federal learning and computing equipment

Country Status (1)

Country Link
CN (1) CN116451275B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113434873A (en) * 2021-06-01 2021-09-24 内蒙古大学 Federal learning privacy protection method based on homomorphic encryption
CN113468521A (en) * 2021-07-01 2021-10-01 哈尔滨工程大学 Data protection method for federal learning intrusion detection based on GAN
CN114372046A (en) * 2021-05-13 2022-04-19 青岛亿联信息科技股份有限公司 Parking flow prediction model training method based on federal learning
CN114547643A (en) * 2022-01-20 2022-05-27 华东师范大学 Linear regression longitudinal federated learning method based on homomorphic encryption
WO2022110720A1 (en) * 2020-11-24 2022-06-02 平安科技（深圳）有限公司 Selective gradient updating-based federated modeling method and related device
CN115150068A (en) * 2022-06-10 2022-10-04 上海大学 Safe federal learning system and method in quantum automatic driving car networking
WO2023045503A1 (en) * 2021-09-27 2023-03-30 支付宝(杭州)信息技术有限公司 Feature processing method and device based on differential privacy
CN115965093A (en) * 2021-10-09 2023-04-14 北京字节跳动网络技术有限公司 Model training method and device, storage medium and electronic equipment
WO2023092792A1 (en) * 2021-11-29 2023-06-01 深圳前海微众银行股份有限公司 Optimization method for modeling based on federated learning, and electronic device, storage medium and program product

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3970074A1 (en) * 2019-05-16 2022-03-23 FRAUNHOFER-GESELLSCHAFT zur Förderung der angewandten Forschung e.V. Concepts for federated learning, client classification and training data similarity measurement

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022110720A1 (en) * 2020-11-24 2022-06-02 平安科技（深圳）有限公司 Selective gradient updating-based federated modeling method and related device
CN114372046A (en) * 2021-05-13 2022-04-19 青岛亿联信息科技股份有限公司 Parking flow prediction model training method based on federal learning
CN113434873A (en) * 2021-06-01 2021-09-24 内蒙古大学 Federal learning privacy protection method based on homomorphic encryption
CN113468521A (en) * 2021-07-01 2021-10-01 哈尔滨工程大学 Data protection method for federal learning intrusion detection based on GAN
WO2023045503A1 (en) * 2021-09-27 2023-03-30 支付宝(杭州)信息技术有限公司 Feature processing method and device based on differential privacy
CN115965093A (en) * 2021-10-09 2023-04-14 北京字节跳动网络技术有限公司 Model training method and device, storage medium and electronic equipment
WO2023092792A1 (en) * 2021-11-29 2023-06-01 深圳前海微众银行股份有限公司 Optimization method for modeling based on federated learning, and electronic device, storage medium and program product
CN114547643A (en) * 2022-01-20 2022-05-27 华东师范大学 Linear regression longitudinal federated learning method based on homomorphic encryption
CN115150068A (en) * 2022-06-10 2022-10-04 上海大学 Safe federal learning system and method in quantum automatic driving car networking

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Towards Efficient and Privacy-preserving Federated Deep Learning;Meng Hao等;《IEEE》;第1-6页 *

Also Published As

Publication number Publication date
CN116451275A (en) 2023-07-18

Similar Documents

Publication Publication Date Title
CN111611610B (en) Federal learning information processing method, system, storage medium, program, and terminal
CN110572253B (en) Method and system for enhancing privacy of federated learning training data
Gong et al. Quantum image encryption algorithm based on quantum image XOR operations
Ye A block image encryption algorithm based on wave transmission and chaotic systems
US11599832B2 (en) Systems, circuits and computer program products providing a framework for secured collaborative training using hyper-dimensional vector based data encoding/decoding and related methods
CN113077060A (en) Federal learning system and method aiming at edge cloud cooperation
DE112019001441T5 (en) FORGETTABLE PSEUDO ACCIDENT FUNCTION IN A KEY MANAGEMENT SYSTEM
Erkin et al. Privacy-preserving distributed clustering
CN114363043B (en) Asynchronous federal learning method based on verifiable aggregation and differential privacy in peer-to-peer network
CN111581648B (en) Method of federal learning to preserve privacy in irregular users
Jin et al. FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System
Alghafis et al. A novel digital contents privacy scheme based on quantum harmonic oscillator and schrodinger paradox
CN116484415A (en) Privacy decision tree reasoning method based on isomorphic encryption
Li et al. An adaptive communication-efficient federated learning to resist gradient-based reconstruction attacks
Wang et al. Protecting data privacy in federated learning combining differential privacy and weak encryption
CN112560059B (en) Vertical federal model stealing defense method based on neural pathway feature extraction
CN111865581B (en) Quantum secret sharing method based on tensor network and quantum communication system
CN116451275B (en) Privacy protection method based on federal learning and computing equipment
Yi et al. An Algorithm of Image Encryption based on AES & Rossler Hyperchaotic Modeling
Yang et al. Efficient and secure federated learning with verifiable weighted average aggregation
CN111581663B (en) Federal deep learning method for protecting privacy and facing irregular users
CN116681141A (en) Federal learning method, terminal and storage medium for privacy protection
CN115643105A (en) Federal learning method and device based on homomorphic encryption and depth gradient compression
CN108900294A (en) It is related to the neural network model encryption protection system and method for designated frequency band encryption
CN116415267A (en) Iterative updating method, device and system for joint learning model and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant