CN113282960A - Privacy calculation method, device, system and equipment based on federal learning - Google Patents
Privacy calculation method, device, system and equipment based on federal learning Download PDFInfo
- Publication number
- CN113282960A CN113282960A CN202110654316.2A CN202110654316A CN113282960A CN 113282960 A CN113282960 A CN 113282960A CN 202110654316 A CN202110654316 A CN 202110654316A CN 113282960 A CN113282960 A CN 113282960A
- Authority
- CN
- China
- Prior art keywords
- target
- parameter
- model
- sub
- aggregation model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention provides a privacy calculation method, a device, a system and equipment based on federal learning, wherein the method is applied to a server side and comprises the following steps: determining each target client participating in the iterative training; sending preset cutting parameters to each target client so that each target client cuts the sub-model parameter change values; receiving the parameter change value of the clipped sub-model and carrying out polymerization treatment to obtain a parameter change value of a polymerization model; updating the parameters of the target aggregation model based on the parameter change value of the aggregation model obtained after the noise processing and the parameters of the target aggregation model of the iteration; when the current iteration meets a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem; and if not, returning to execute the step of determining each target client participating in the iterative training. According to the embodiment of the invention, the model privacy parameters based on the federal learning can be calculated.
Description
Technical Field
The invention relates to the technical field of federal learning, in particular to a privacy calculation method, a device, a system and equipment based on federal learning.
Background
With the development of science and technology, the international society pays more and more attention to the problems of information safety and personal privacy, various related laws are issued in succession, and the management, supervision and protection of private data are more comprehensive, strict and intensive. Companies pay more and more attention to respective data, and the data cannot be taken out and shared as assets, so that a data isolated island phenomenon is caused. In order to solve the data island problem among equipment, Google provides a federal learning method aiming at a mobile phone terminal. The federated learning is to jointly train the models by combining data of different terminals, different companies, different merchants and the like, so that the problem of data islands can be effectively solved, participators can jointly model on the basis of not sharing data, and the data islands can be technically broken.
In the related technology, based on a federal learning model, in order to protect the privacy of user data of each terminal, disturbance noise is added to the local user data at each terminal in the model training process, and then terminal sub-model parameters obtained by adding disturbance are uploaded to a server for joint training, so that the privacy protection of user-level data is realized. In the process, the calculation of the model privacy parameters obtained by joint training becomes a technical problem to be solved urgently.
Disclosure of Invention
The embodiment of the invention aims to provide a privacy calculation method, a device, a system and equipment based on federal learning, so as to calculate model privacy parameters based on federal learning. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a privacy calculation method based on federal learning, which is applied to a server side, and the method includes:
determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
sending a preset clipping parameter to each target client, so that each target client clips a sub-model parameter variation value based on the preset clipping parameter, and obtaining a clipped sub-model parameter variation value, wherein the sub-model parameter variation value is as follows: each target client side trains a local submodel by using local sample data to obtain a variation value of a submodel parameter before and after training;
receiving the clipped sub-model parameter change values sent by each target client;
performing aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value;
carrying out noise adding processing on the aggregation model parameter change value by using a differential privacy mechanism, updating a target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client;
under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem;
and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the step of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
Optionally, the step of sending a preset clipping parameter to each target client, so that each target client clips the sub-model parameter variation value based on the preset clipping parameter, to obtain a clipped sub-model parameter variation value includes:
sending preset cutting parameters to each target client so that each target client cuts the parameter change value of each layer of the sub-model based on the preset cutting parameters to obtain the parameter change value of the sub-model after cutting; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
Optionally, the step of performing noise processing on the aggregation model parameter variation value by using a differential privacy mechanism, and updating the target aggregation model parameter based on the aggregation model parameter variation value obtained after the noise processing and the target aggregation model parameter of the current iteration includes:
and by utilizing a differential privacy mechanism, performing noise addition processing on the parameter change value of the aggregation model by using a first preset expression as follows, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing the variation value of the parameters of the aggregation model, m representing the number of target clients, N (0, z)2·S2) Representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
Optionally, the step of calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to the central limit theorem includes:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
In a second aspect, an embodiment of the present invention provides a privacy computing apparatus based on federal learning, which is applied to a server side, and the apparatus includes:
the determining module is used for determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
a sending module, configured to send a preset clipping parameter to each target client, so that each target client clips a sub-model parameter variation value based on the preset clipping parameter, to obtain a clipped sub-model parameter variation value, where the sub-model parameter variation value is: each target client side trains a local submodel by using local sample data to obtain a variation value of a submodel parameter before and after training;
the receiving module is used for receiving the clipped sub-model parameter change values sent by the target clients;
the aggregation module is used for aggregating the parameter change values of the clipped sub-models to obtain the parameter change values of the aggregation models;
the noise adding module is used for performing noise adding processing on the aggregation model parameter change value by using a differential privacy mechanism, updating a target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client;
the calculation module is used for calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem under the condition that the current iteration number meets a preset convergence condition; and under the condition that the current iteration times do not meet the preset convergence condition, triggering a determining module to execute the steps of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
Optionally, the sending module is specifically configured to:
sending preset cutting parameters to each target client so that each target client cuts the parameter change value of each layer of the sub-model based on the preset cutting parameters to obtain the parameter change value of the sub-model after cutting; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
Optionally, the noise adding module is specifically configured to:
and by utilizing a differential privacy mechanism, performing noise addition processing on the parameter change value of the aggregation model by using a first preset expression as follows, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing the variation value of the parameters of the aggregation model, m representing the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
Optionally, the calculation module is specifically configured to:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
In a third aspect, an embodiment of the present invention provides a privacy computing system based on federal learning, where the system includes a server and a target client;
the server is used for determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data, and sending preset cutting parameters to each target client;
the target client is used for training a local sub-model by using local sample data to obtain change values of sub-model parameters before and after training, receiving the preset cutting parameters sent by the server, cutting the sub-model parameter change values based on the preset cutting parameters to obtain cut sub-model parameter change values, and sending the cut sub-model parameter change values to the server;
the server is used for receiving the clipped sub-model parameter change values sent by the target clients, performing aggregation processing on the clipped sub-model parameter change values to obtain aggregation model parameter change values, performing noise processing on the aggregation model parameter change values by using a differential privacy mechanism, updating target aggregation model parameters based on the aggregation model parameter change values obtained after the noise processing and the current iteration target aggregation model parameters, and sending the updated target aggregation model parameters to the target clients; under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem; and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the process of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
In a fourth aspect, an embodiment of the present invention provides a server device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
a processor configured to implement the method steps of the first aspect when executing the program stored in the memory.
In a fifth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps described in the first aspect.
Embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any of the above-mentioned steps of the privacy calculation method based on federated learning.
The embodiment of the invention has the following beneficial effects:
according to the privacy calculation method, device, system and equipment based on federal learning, provided by the embodiment of the invention, the server side can determine each target client participating in the iterative training from a plurality of clients respectively storing different sample data; sending preset cutting parameters to each target client so that each target client cuts the sub-model parameter change value based on the preset cutting parameters to obtain the cut sub-model parameter change value; receiving the clipped sub-model parameter change values sent by each target client; carrying out aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value; carrying out noise adding processing on the parameter change value of the aggregation model by using a differential privacy mechanism, and updating the parameters of the target aggregation model based on the parameter change value of the aggregation model obtained after the noise adding processing and the parameters of the target aggregation model of the iteration; and under the condition that the current iteration times meet a preset convergence condition, calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem. According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.
Fig. 1 is a schematic flowchart of a privacy calculation method based on federal learning according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a privacy computing apparatus based on federal learning according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a privacy computing system based on federated learning according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a server device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.
In order to realize the calculation of the model privacy parameters based on the federal learning, the embodiment of the invention provides a privacy calculation method based on the federal learning, which is applied to a server side and comprises the following steps:
determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
sending a preset clipping parameter to each target client, so that each target client clips a sub-model parameter variation value based on the preset clipping parameter, and obtaining a clipped sub-model parameter variation value, wherein the sub-model parameter variation value is as follows: each target client side trains a local submodel by using local sample data to obtain a variation value of a submodel parameter before and after training;
receiving the clipped sub-model parameter change values sent by each target client;
performing aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value;
carrying out noise adding processing on the aggregation model parameter change value by using a differential privacy mechanism, updating a target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client;
under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem;
and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the step of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
According to the privacy calculation method based on federal learning, provided by the embodiment of the invention, a server side can determine each target client participating in the iterative training from a plurality of clients respectively storing different sample data; sending preset cutting parameters to each target client so that each target client cuts the sub-model parameter change value based on the preset cutting parameters to obtain the cut sub-model parameter change value; receiving the clipped sub-model parameter change values sent by each target client; carrying out aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value; carrying out noise adding processing on the parameter change value of the aggregation model by using a differential privacy mechanism, and updating the parameters of the target aggregation model based on the parameter change value of the aggregation model obtained after the noise adding processing and the parameters of the target aggregation model of the iteration; and under the condition that the current iteration times meet a preset convergence condition, calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem. According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
Specifically, referring to fig. 1, fig. 1 is a schematic flow chart of a privacy calculation method based on federal learning according to an embodiment of the present invention, which is applied to a server, and the method may include:
s101, determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data.
In the federal learning process, the server side can independently select the client side participating in training in each round, different sample data is stored in different client sides, and the sample data can be user data of the client side. The server side and different clients can agree in advance on the type of a jointly trained target aggregation model, such as a neural network model, a random forest model and the like.
After entering the iteration, the server can randomly select the client participating in the iteration training as a target client. The server determines that the target clients participating in each iterative training can be the same or different, the number of the determined target clients can be the same or different, and the client selection of each round is independent of the total iteration number.
And S102, sending the preset cutting parameters to each target client, so that each target client cuts the sub-model parameter change values based on the preset cutting parameters, and obtaining the cut sub-model parameter change values. Wherein, the parameter variation value of the sub-model is as follows: and each target client respectively trains the local submodel by using local sample data to obtain the change value of the submodel parameter before and after training.
The server side and different target clients agree in advance on the type of the jointly trained target aggregation model, namely, the server side trains the overall model (namely, the target aggregation model), and each target client side trains the sub-model. And each target client determined by the server trains the local submodel by using the local sample data to obtain submodel parameters and further obtains the change values of the submodel parameters before and after training. After the server sends the preset cutting parameters to each target client, each target client cuts the parameter change value of each sub-model based on the preset cutting parameters to obtain the parameter change value of the sub-model after cutting.
The preset clipping parameters may be determined by the server side according to the number of clients participating in training in each round.
As an optional implementation manner of the embodiment of the present invention, an implementation manner in which the server sends the preset clipping parameters to each target client, so that each target client clips the sub-model parameter variation value based on the preset clipping parameters, and obtains the clipped sub-model parameter variation value includes:
sending preset cutting parameters to each target client side so that each target client side cuts parameter change values of each layer of the sub-model respectively based on the preset cutting parameters to obtain cut sub-model parameter change values; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
Under federal learning, each target client can train a local submodel by using local sample data, the local sample data of the target client can be a non-independent and uniformly distributed data set, and the purpose of the target client for training the local submodel is as follows: a more excellent federated model trained from more data is obtained without exposing local user data.
The process that the target client side trains the local submodel by using the local sample data can be a complete deep learning training process, specifically, the local sample data can be divided into a plurality of small batches of data, each small batch of data comprises a plurality of sample data, and then the small batches of data are used for training the submodel instead of the whole local training data set, so that the training process of each time is very quick. In the training process of the sub-model, the parameters of the sub-model can be trained by using the following expression:
wherein w represents the parameters (or weighting parameters) of the submodel, η represents the learning rate,
and b, representing a loss function of the sub-model, and b representing a small batch of data sets participating in training, wherein the size of the data sets can be adjusted according to actual conditions. After the target client finishes the training of the submodel, the parameter variation values of the submodel before and after the training of the submodel can be obtained, and the parameter variation value obtaining expression of the submodel can be expressed as: Δ w ═ w-wnWhere Δ w represents a parameter change value of the before-and-after-training submodel, w represents a parameter of the before-training submodel, and w representsnAnd representing parameters obtained after the sub-model is iteratively trained.
When the submodel has multiple layers, the target client can train the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training, and then after receiving the preset cutting parameters sent by the server, the preset cutting parameters are divided based on the layer number of the submodel, so that the square sum of the cutting parameters of all the layer numbers of the submodel is added to be the square sum of the preset cutting parameters, and then single-layer cutting is carried out on the parameter change value of each layer of the submodel. Specifically, the following expressions may be used to respectively clip the parameter variation values of each layer of the sub-model, so as to obtain the clipped sub-model parameter variation values:
And the target client cuts the parameter change values of each layer of the sub-model respectively to obtain the cut sub-model parameter change values, and then sends the cut sub-model parameter change values to the server.
Referring to fig. 1, S103, receives the clipped sub-model parameter variation value sent by each target client.
And S104, carrying out aggregation processing on the parameter change values of the clipped sub-models to obtain the parameter change values of the aggregation models.
After receiving the clipped sub-model parameter change values sent by each target client, the server may perform aggregation processing on the clipped sub-model parameter change values by using the following expression to obtain an aggregation model parameter change value:
Δwt+1representing the parameter variation value of the aggregation model, m representing the number of target clients, k representing the kth target client, mtAn index representing the target client corresponding to the t-th iteration,
and showing the parameter change value of the clipped sub-model corresponding to the kth target client.
And S105, utilizing a differential privacy mechanism to perform noise adding processing on the aggregation model parameter change value, updating the target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client.
As an optional implementation manner in the embodiment of the present invention, an implementation manner in which a difference privacy mechanism is used to perform denoising processing on a aggregation model parameter variation value, and based on the aggregation model parameter variation value obtained after the denoising processing and a target aggregation model parameter of this iteration, a target aggregation model parameter is updated includes:
by utilizing a differential privacy mechanism, carrying out noise addition processing on the parameter change value of the aggregation model by using a following first preset expression, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing the variation value of the parameters of the aggregation model, m representing the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
After the server updates the target aggregation model parameters, the updated target aggregation model parameters are sent to each target client participating in the current round of training, so that each target client participating in the training can update the sub-model parameters by using the updated target aggregation model parameters, and subsequent joint training is facilitated.
And S106, under the condition that the current iteration number meets the preset convergence condition, calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to the central limit theorem.
And under the condition that the current iteration number meets the preset convergence condition, the target aggregation model is shown to be completed in the joint training, and at the moment, the privacy parameter value of the target aggregation model can be calculated by utilizing an f-difference privacy mechanism according to the central limit theorem so as to obtain the privacy parameter of the target aggregation model of the joint training under the current precision.
Optionally, the privacy parameter value of the target aggregation model may also be calculated by using an f-difference privacy mechanism according to a central limit theorem when the accuracy of the currently trained target aggregation model reaches a preset value.
As an optional implementation manner in the embodiment of the present invention, an implementation manner of calculating a privacy parameter value of a target aggregation model by using an f-difference privacy mechanism according to a central limit theorem includes:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
Optionally, according to the central limit theorem, the embodiment of calculating the privacy parameter value of the target aggregation model by using the f-difference privacy mechanism may further include:
calculating the privacy parameter value of the target aggregation model by using a third preset expression as follows:
the method comprises the steps that mu represents a privacy parameter value of a target aggregation model, p represents sampling probability of a target client, T represents preset iteration times of federal learning, and z represents a noise scale parameter.
And S107, under the condition that the current iteration times do not meet the preset convergence condition, returning to the step of executing S101 to determine each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
And under the condition that the current iteration times do not meet the preset convergence condition, the target aggregation model joint training is not finished, at the moment, the step of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data can be returned to execute S101, and the training of the model is continued.
According to the privacy calculation method based on federal learning, provided by the embodiment of the invention, a server side can determine each target client participating in the iterative training from a plurality of clients respectively storing different sample data; sending preset cutting parameters to each target client so that each target client cuts the sub-model parameter change value based on the preset cutting parameters to obtain the cut sub-model parameter change value; receiving the clipped sub-model parameter change values sent by each target client; carrying out aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value; carrying out noise adding processing on the parameter change value of the aggregation model by using a differential privacy mechanism, and updating the parameters of the target aggregation model based on the parameter change value of the aggregation model obtained after the noise adding processing and the parameters of the target aggregation model of the iteration; and under the condition that the current iteration times meet a preset convergence condition, calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem. According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
For example, according to the central limit theorem, the above-mentioned embodiment of calculating the privacy parameter value of the target aggregation model by using the f-differential privacy mechanism is obtained by redefining the differential privacy through an angle of hypothesis testing in federal learning, so as to adopt a more reasonable privacy measurement method for a specific environment of federal learning.
Specifically, assuming that an attacker first defines data levels under federal learning in the process of deducing whether any single data is in a data set formed by clients, from the viewpoint of hypothesis testing, a target client selected by each round of training server is used as one data set, and each data in the data set represents local data of each target client. Let H0Assume that: the data set deduced by the attacker is real client data, otherwise, the data set is H1Suppose, wherein H0Hypothesis represents type I hypothesis testing in the hypothesis testing problem, H1The hypothesis represents type II hypothesis testing in the hypothesis testing problem. Then alpha is H under certain rejection rules0Deny H if it is correct0Is the probability of (b) is H under a certain rejection rule1Receive H in error1Where α represents the type I error rate in the hypothesis testing problem and β represents the type II error rate in the hypothesis testing problem.
In the process of one iteration, a weighing function is established to combine the two types of errors to weigh the attack strength of an attacker, namely privacy disclosure under differential privacy protection. The trade-off function is the following expression:
where F (P, Q) represents a trade-off function, P and Q represent the distributions adopted by the differential privacy mechanism for two neighboring data sets, respectively,
represents H0Hypothesis sum H1Given the rules of rejection between the hypotheses,
indicating rules for rejection in case of applying Q-distribution
The resulting expectation is that, as a result,
indicating rules for rejection in case of application P distribution
The resulting expectation, a, indicates a type I error probability,
indicating the infimum bound for the rejection rule phi. The contiguous data sets represent two data sets that differ by 1 for the number of data/elements selected by the server in the iteration.
The privacy disclosure problem in the Differential privacy is measured by a weighting function belonging to F-DP (F-Differential privacy), and the measurement function F (P, Q) is abbreviated as function F, so the privacy mechanism under the federal learning framework can be expressed as:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresenting target aggregation model parameters under the t-th iteration, f representing a balance function, m representing the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
The sensitivity of the function f is S/m, and P and Q are normally distributed, so that:
Gμ(α)＝Φ(Φ-1(1-α)-μ)
wherein G isμRepresenting a trade-off function for noise addition using Gaussian noise with only a single parameter μ, μ representing a privacy parameter value of the target aggregation model, Gμ(α) represents a trade-off function for noising using gaussian noise with a single parameter μ, and Φ () represents a cumulative distribution function for normal distribution.
According to the above expression
It can be seen that the closer μ is to 0, the more consistent the two distributions in the trade-off function are, the less readily the hypothetical attacker can see hypothetical H0And hypothesis H1。
The server communicates with the target client T times (i.e. iterates T times), in this process, the differential privacy mechanism may be set as:
wherein the content of the first and second substances,
is the T folds of the mechanism M,
represents a mechanism formed by combining T times of sampling (the probability of each sampling is p) on a target client, M represents a mechanism formed by combining sampling with the probability of p on the target client, Samplep(d) This means that a data set d is sampled with a probability p, in this case, a data set d represented by a target client is sampled with a probability p. By the combinatorial theorem of f-DP, we obtain:
d and d' are neighbor user data sets, then there are some according to the trade-off function property
The two mechanisms mentioned above can be obtained at this time
And
quilt
This is not a trade-off function, however, and is also non-convex, using conjugate for further analysis
Is also the best convex optimization choice from experience and is also a trade-off function. It can thus be obtained that the calculation of the privacy parameter value satisfies:
DP is the Differential privacy, and according to the property of f-DP, the calculation of the privacy parameter value also meets GDP (Gaussian Differential privacy), and meets the requirement of single iteration
Then, in the calculation process of the privacy parameter value, when the number of times that the target client communicates with the server (i.e. iterates) tends to infinity, and the square root of the server sampling probability and the number of times of communication tends to a normal amount, by the combined nature of f-DP, it can be obtained that:
wherein the content of the first and second substances,
the target client is sampled for T times, the target client is selected once for one iteration, the sampling probability of the target client is p,
represents the conjugation of G μ. The calculation of the temporarily set privacy parameter value satisfies μ -GDP using the Central limit theorem, and the value is solved by CLT (Central limit theorem).
If the server adopts uniform and consistent sampling to the target client, calculating according to chi-square distribution of f-DP to obtain:
similarly, if Poisson sampling is adopted, a calculation result is obtained through f-DP property and chi-square distribution:
in order to exhibit the privacy-preserving effect designed based on hypothesis testing, f-DP is converted into (ε, δ (ε; μ)) -DP (ε ≧ 0) by the duality of f-DP, and can be mutually converted by the following expression:
wherein, delta (epsilon; mu) represents a traditional differential privacy balance function, epsilon represents a privacy parameter under the traditional differential privacy definition, and phi represents a cumulative distribution function of normal distribution. The effect of f-DP was judged by a tradeoff function of α and β, based on Gμ(α)＝Φ(Φ-1(1- α) - μ) obtaining a relationship between α and β, corresponding to the privacy parameters ε and μ under the definition of traditional differential privacy, where the differential privacy is changed from traditional two parameters to a single parameter μ by
The single parameter may be converted to a conventional differential privacy parameter. And the f-DP can be proved to have privacy guarantee in the calculation of the privacy parameter value through the above.
Corresponding to the foregoing method embodiment, an embodiment of the present invention provides a privacy computing apparatus based on federal learning, as shown in fig. 2, applied to a server, where the apparatus may include:
a determining module 201, configured to determine, from multiple clients that respectively store different sample data, each target client that participates in the current iterative training;
a sending module 202, configured to send a preset clipping parameter to each target client, so that each target client clips the sub-model parameter variation value based on the preset clipping parameter, to obtain a clipped sub-model parameter variation value, where the sub-model parameter variation value is: each target client side trains the local submodel by using local sample data respectively to obtain the change values of the parameters of the submodel before and after training;
the receiving module 203 is used for receiving the clipped sub-model parameter change values sent by each target client;
the aggregation module 204 is configured to aggregate the parameter variation values of the clipped sub-models to obtain an aggregate model parameter variation value;
the denoising module 205 is configured to perform denoising processing on the aggregation model parameter change value by using a differential privacy mechanism, update a target aggregation model parameter based on the aggregation model parameter change value obtained after the denoising processing and the target aggregation model parameter of the current iteration, and send the updated target aggregation model parameter to each target client;
the calculating module 206 is configured to calculate a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem when the current iteration number satisfies a preset convergence condition; and under the condition that the current iteration times do not meet the preset convergence condition, triggering a determining module to execute the steps of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
According to the privacy calculation device based on the federal learning, provided by the embodiment of the invention, the server side can determine each target client participating in the iterative training from a plurality of clients respectively storing different sample data; sending preset cutting parameters to each target client so that each target client cuts the sub-model parameter change value based on the preset cutting parameters to obtain the cut sub-model parameter change value; receiving the clipped sub-model parameter change values sent by each target client; carrying out aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value; carrying out noise adding processing on the parameter change value of the aggregation model by using a differential privacy mechanism, and updating the parameters of the target aggregation model based on the parameter change value of the aggregation model obtained after the noise adding processing and the parameters of the target aggregation model of the iteration; and under the condition that the current iteration times meet a preset convergence condition, calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem. According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
Optionally, the sending module 202 is specifically configured to:
sending preset cutting parameters to each target client side so that each target client side cuts parameter change values of each layer of the sub-model respectively based on the preset cutting parameters to obtain cut sub-model parameter change values; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
Optionally, the noise adding module 205 is specifically configured to:
by utilizing a differential privacy mechanism, carrying out noise addition processing on the parameter change value of the aggregation model by using a following first preset expression, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing the variation value of the parameters of the aggregation model, m representing the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
Optionally, the calculating module 206 is specifically configured to:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
Corresponding to the method embodiment, the embodiment of the invention also provides a privacy computing system based on federal learning.
As shown in fig. 3, the privacy computing system 300 based on federal learning includes: a server 301, a target client 302;
the server 301 is configured to determine, from a plurality of clients that respectively store different sample data, each target client that participates in the iterative training, and send a preset clipping parameter to each target client;
the target client 302 is used for training the local submodel by using local sample data to obtain change values of parameters of the submodel before and after training and receiving preset cutting parameters sent by the server, cutting the change values of the parameters of the submodel based on the preset cutting parameters to obtain change values of the parameters of the clipped submodel, and sending the change values of the parameters of the clipped submodel to the server;
the server 301 is configured to receive the clipped sub-model parameter change values sent by each target client, perform aggregation processing on the clipped sub-model parameter change values to obtain aggregation model parameter change values, perform noise addition processing on the aggregation model parameter change values by using a differential privacy mechanism, update a target aggregation model parameter based on the aggregation model parameter change values obtained after the noise addition processing and the target aggregation model parameter of the iteration, and send the updated target aggregation model parameter to each target client; under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem; and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the process of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
Optionally, the target client 302 is specifically configured to: the method comprises the steps of training a local submodel by using local sample data, obtaining parameter change values of each layer of the submodel before and after training, receiving preset cutting parameters sent by a server terminal 301, cutting the parameter change values of each layer of the submodel based on the preset cutting parameters to obtain the parameter change values of the clipped submodel, and sending the parameter change values of the clipped submodel to the server terminal 301.
Optionally, the server 301 is specifically configured to:
by utilizing a differential privacy mechanism, carrying out noise addition processing on the parameter change value of the aggregation model by using a following first preset expression, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing the variation value of the parameters of the aggregation model, m representing the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
Optionally, the server 301 is specifically configured to:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
The embodiment of the present invention further provides a server device, as shown in fig. 4, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, where the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401, when executing the program stored in the memory 403, implements the following steps:
determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
sending the preset cutting parameters to each target client so that each target client cuts the sub-model parameter variation values based on the preset cutting parameters to obtain the sub-model parameter variation values after cutting, wherein the sub-model parameter variation values are as follows: each target client side trains the local submodel by using local sample data respectively to obtain the change values of the parameters of the submodel before and after training;
receiving the clipped sub-model parameter change values sent by each target client;
carrying out aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value;
the method comprises the steps of utilizing a differential privacy mechanism to conduct noise adding processing on a parameter change value of a polymerization model, updating parameters of a target polymerization model based on the parameter change value of the polymerization model obtained after the noise adding processing and parameters of a target polymerization model of the iteration, and sending the updated parameters of the target polymerization model to target clients;
under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem; and under the condition that the current iteration times do not meet the preset convergence condition, determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
According to the embodiment of the invention, when the training of the server-side target aggregation model is completed, the privacy parameter values of the target aggregation model can be calculated by using the f-difference privacy mechanism according to the central limit theorem, and the privacy parameter values of the server-side target aggregation model under different convergence conditions can be calculated, so that privacy leakage values corresponding to user data under different target aggregation model accuracies can be obtained.
The communication bus mentioned in the above server device may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a RAM (Random Access Memory) or an NVM (Non-Volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and when executed by a processor, the computer program implements the steps of any one of the above-mentioned privacy calculation methods based on federal learning, so as to achieve the same technical effect.
In yet another embodiment of the present invention, a computer program product containing instructions is also provided, which when run on a computer, causes the computer to perform the steps of any one of the above-mentioned embodiments of the method for privacy calculation based on federal learning, so as to achieve the same technical effect.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber, DSL (Digital Subscriber Line)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a DVD (Digital Versatile Disk)), or a semiconductor medium (e.g., an SSD (Solid State Disk)), etc.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device/system/server apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, refer to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A privacy calculation method based on federal learning is characterized by being applied to a server side, and the method comprises the following steps:
determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
sending a preset clipping parameter to each target client, so that each target client clips a sub-model parameter variation value based on the preset clipping parameter, and obtaining a clipped sub-model parameter variation value, wherein the sub-model parameter variation value is as follows: each target client side trains a local submodel by using local sample data to obtain a variation value of a submodel parameter before and after training;
receiving the clipped sub-model parameter change values sent by each target client;
performing aggregation processing on each clipped sub-model parameter change value to obtain an aggregation model parameter change value;
carrying out noise adding processing on the aggregation model parameter change value by using a differential privacy mechanism, updating a target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client;
under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem;
and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the step of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
2. The method according to claim 1, wherein the step of sending a preset clipping parameter to each of the target clients so that each of the target clients clips a sub-model parameter variation value based on the preset clipping parameter to obtain a clipped sub-model parameter variation value includes:
sending preset cutting parameters to each target client so that each target client cuts the parameter change value of each layer of the sub-model based on the preset cutting parameters to obtain the parameter change value of the sub-model after cutting; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
3. The method according to claim 1, wherein the step of performing noise processing on the aggregation model parameter variation value by using a differential privacy mechanism, and updating the target aggregation model parameter based on the aggregation model parameter variation value obtained after the noise processing and the target aggregation model parameter of the current iteration includes:
and by utilizing a differential privacy mechanism, performing noise addition processing on the parameter change value of the aggregation model by using a first preset expression as follows, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target aggregation model parameter under the t-th iteration, the delta wt +1 represents the aggregation model parameter variation value, the m represents the number of the target clients,
representing a gaussian distribution function with an expectation of 0 and a variance of z2 · S2, z representing a noise scale parameter and S representing a preset clipping parameter.
4. The method of claim 1, wherein the step of calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to the central limit theorem comprises:
calculating the privacy parameter value of the target aggregation model by using a second preset expression as follows:
wherein μ represents a privacy parameter value of the target aggregation model, p represents a sampling probability for the target client, T represents a preset iteration number of federal learning, z represents a noise scale parameter, and Φ () represents a cumulative distribution function of normal distribution.
5. A privacy computing apparatus based on federal learning, applied to a server, the apparatus comprising:
the determining module is used for determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data;
a sending module, configured to send a preset clipping parameter to each target client, so that each target client clips a sub-model parameter variation value based on the preset clipping parameter, to obtain a clipped sub-model parameter variation value, where the sub-model parameter variation value is: each target client side trains a local submodel by using local sample data to obtain a variation value of a submodel parameter before and after training;
the receiving module is used for receiving the clipped sub-model parameter change values sent by the target clients;
the aggregation module is used for aggregating the parameter change values of the clipped sub-models to obtain the parameter change values of the aggregation models;
the noise adding module is used for performing noise adding processing on the aggregation model parameter change value by using a differential privacy mechanism, updating a target aggregation model parameter based on the aggregation model parameter change value obtained after the noise adding processing and the target aggregation model parameter of the iteration, and sending the updated target aggregation model parameter to each target client;
the calculation module is used for calculating the privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem under the condition that the current iteration number meets a preset convergence condition; and under the condition that the current iteration times do not meet the preset convergence condition, triggering a determining module to execute the steps of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
6. The apparatus of claim 5, wherein the sending module is specifically configured to:
sending preset cutting parameters to each target client so that each target client cuts the parameter change value of each layer of the sub-model based on the preset cutting parameters to obtain the parameter change value of the sub-model after cutting; wherein, the parameter variation value of each layer of the submodel is as follows: and each target client respectively trains the local submodel by using local sample data to obtain parameter change values of each layer of the submodel before and after training.
7. The apparatus of claim 5, wherein the noise adding module is specifically configured to:
and by utilizing a differential privacy mechanism, performing noise addition processing on the parameter change value of the aggregation model by using a first preset expression as follows, and updating the target aggregation model parameter based on the parameter change value of the aggregation model obtained after the noise addition processing and the target aggregation model parameter of the iteration:
wherein, wt+1Representing updated target aggregate model parameters, wtRepresents the target polymerization model parameter, Δ w, at the t-th iterationt+1Representing aggregate model parametersA variance value, m represents the number of target clients,
representing an expectation of 0, variance z2·S2Z represents a noise scale parameter and S represents a preset clipping parameter.
8. The privacy computing system based on the federal learning is characterized by comprising a server side and a target client side;
the server is used for determining each target client participating in the iterative training from a plurality of clients respectively storing different sample data, and sending preset cutting parameters to each target client;
the target client is used for training a local sub-model by using local sample data to obtain change values of sub-model parameters before and after training, receiving the preset cutting parameters sent by the server, cutting the sub-model parameter change values based on the preset cutting parameters to obtain cut sub-model parameter change values, and sending the cut sub-model parameter change values to the server;
the server is used for receiving the clipped sub-model parameter change values sent by the target clients, performing aggregation processing on the clipped sub-model parameter change values to obtain aggregation model parameter change values, performing noise processing on the aggregation model parameter change values by using a differential privacy mechanism, updating target aggregation model parameters based on the aggregation model parameter change values obtained after the noise processing and the current iteration target aggregation model parameters, and sending the updated target aggregation model parameters to the target clients; under the condition that the current iteration times meet a preset convergence condition, calculating a privacy parameter value of the target aggregation model by using an f-difference privacy mechanism according to a central limit theorem; and under the condition that the current iteration times do not meet the preset convergence condition, returning to execute the process of determining each target client participating in the current iteration training from a plurality of clients respectively storing different sample data.
9. The server equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 4 when executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654316.2A CN113282960B (en) | 2021-06-11 | 2021-06-11 | Privacy calculation method, device, system and equipment based on federal learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110654316.2A CN113282960B (en) | 2021-06-11 | 2021-06-11 | Privacy calculation method, device, system and equipment based on federal learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113282960A true CN113282960A (en) | 2021-08-20 |
| CN113282960B CN113282960B (en) | 2023-02-17 |
Family
ID=77284389
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110654316.2A Active CN113282960B (en) | 2021-06-11 | 2021-06-11 | Privacy calculation method, device, system and equipment based on federal learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113282960B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114021734A (en) * | 2021-10-14 | 2022-02-08 | 深圳致星科技有限公司 | Parameter calculation device, system and method for federal learning and privacy calculation |
| CN114331540A (en) * | 2021-12-29 | 2022-04-12 | 北京百度网讯科技有限公司 | Method for training model, method and device for determining asset valuation |
| CN114841145A (en) * | 2022-05-10 | 2022-08-02 | 平安科技（深圳）有限公司 | Text abstract model training method and device, computer equipment and storage medium |
| CN114863499A (en) * | 2022-06-30 | 2022-08-05 | 广州脉泽科技有限公司 | Finger vein and palm vein identification method based on federal learning |
| CN115510472A (en) * | 2022-11-23 | 2022-12-23 | 南京邮电大学 | Cloud edge aggregation system-oriented multiple differential privacy protection method and system |
| CN115664839A (en) * | 2022-11-15 | 2023-01-31 | 富算科技(上海)有限公司 | Security monitoring method, device, equipment and medium for privacy computing process |
| WO2023109294A1 (en) * | 2021-12-13 | 2023-06-22 | 支付宝(杭州)信息技术有限公司 | Method and apparatus for jointly training natural language processing model on basis of privacy protection |
| CN116432781A (en) * | 2023-04-23 | 2023-07-14 | 中国工商银行股份有限公司 | Federal learning defense method, federal learning defense device, federal learning defense computer device, and federal learning defense storage medium |
| CN117390448A (en) * | 2023-10-25 | 2024-01-12 | 西安交通大学 | Client model aggregation method and related system for inter-cloud federal learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190227980A1 (en) * | 2018-01-22 | 2019-07-25 | Google Llc | Training User-Level Differentially Private Machine-Learned Models |
| CN110084365A (en) * | 2019-03-13 | 2019-08-02 | 西安电子科技大学 | A kind of service provider system and method based on deep learning |
| CN112052480A (en) * | 2020-09-11 | 2020-12-08 | 哈尔滨工业大学（深圳） | Privacy protection method, system and related equipment in model training process |
| CN112818394A (en) * | 2021-01-29 | 2021-05-18 | 西安交通大学 | Self-adaptive asynchronous federal learning method with local privacy protection |
-
2021
- 2021-06-11 CN CN202110654316.2A patent/CN113282960B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190227980A1 (en) * | 2018-01-22 | 2019-07-25 | Google Llc | Training User-Level Differentially Private Machine-Learned Models |
| CN110084365A (en) * | 2019-03-13 | 2019-08-02 | 西安电子科技大学 | A kind of service provider system and method based on deep learning |
| CN112052480A (en) * | 2020-09-11 | 2020-12-08 | 哈尔滨工业大学（深圳） | Privacy protection method, system and related equipment in model training process |
| CN112818394A (en) * | 2021-01-29 | 2021-05-18 | 西安交通大学 | Self-adaptive asynchronous federal learning method with local privacy protection |
Non-Patent Citations (3)
| Title |
|---|
| JEROME P. REITER: "Differential Privacy and Federal Data Releases", 《ANNUAL REVIEWS》 * |
| 程俊宏: "基于联邦学习的差分隐私保护方法", 《中国优秀硕士学位论文全文数据库》 * |
| 黄茜茜: "基于差分隐私保护的不均衡数据联邦学习方法", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114021734B (en) * | 2021-10-14 | 2022-04-12 | 深圳致星科技有限公司 | Parameter calculation device, system and method for federal learning and privacy calculation |
| CN114021734A (en) * | 2021-10-14 | 2022-02-08 | 深圳致星科技有限公司 | Parameter calculation device, system and method for federal learning and privacy calculation |
| WO2023109294A1 (en) * | 2021-12-13 | 2023-06-22 | 支付宝(杭州)信息技术有限公司 | Method and apparatus for jointly training natural language processing model on basis of privacy protection |
| CN114331540A (en) * | 2021-12-29 | 2022-04-12 | 北京百度网讯科技有限公司 | Method for training model, method and device for determining asset valuation |
| CN114331540B (en) * | 2021-12-29 | 2023-09-15 | 北京百度网讯科技有限公司 | Method for training model, method and device for determining asset valuation |
| CN114841145B (en) * | 2022-05-10 | 2023-07-11 | 平安科技（深圳）有限公司 | Text abstract model training method, device, computer equipment and storage medium |
| CN114841145A (en) * | 2022-05-10 | 2022-08-02 | 平安科技（深圳）有限公司 | Text abstract model training method and device, computer equipment and storage medium |
| CN114863499A (en) * | 2022-06-30 | 2022-08-05 | 广州脉泽科技有限公司 | Finger vein and palm vein identification method based on federal learning |
| CN115664839A (en) * | 2022-11-15 | 2023-01-31 | 富算科技(上海)有限公司 | Security monitoring method, device, equipment and medium for privacy computing process |
| CN115664839B (en) * | 2022-11-15 | 2023-04-11 | 富算科技(上海)有限公司 | Security monitoring method, device, equipment and medium for privacy computing process |
| CN115510472A (en) * | 2022-11-23 | 2022-12-23 | 南京邮电大学 | Cloud edge aggregation system-oriented multiple differential privacy protection method and system |
| CN116432781A (en) * | 2023-04-23 | 2023-07-14 | 中国工商银行股份有限公司 | Federal learning defense method, federal learning defense device, federal learning defense computer device, and federal learning defense storage medium |
| CN117390448A (en) * | 2023-10-25 | 2024-01-12 | 西安交通大学 | Client model aggregation method and related system for inter-cloud federal learning |
| CN117390448B (en) * | 2023-10-25 | 2024-04-26 | 西安交通大学 | Client model aggregation method and related system for inter-cloud federal learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113282960B (en) | 2023-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113282960B (en) | Privacy calculation method, device, system and equipment based on federal learning | |
| CN110544155B (en) | User credit score acquisition method, acquisition device, server and storage medium | |
| Meinshausen et al. | Monte Carlo methods for the valuation of multiple‐exercise options | |
| Yu et al. | MCMC estimation of Lévy jump models using stock and option prices | |
| CN112163963A (en) | Service recommendation method and device, computer equipment and storage medium | |
| CN110351299B (en) | Network connection detection method and device | |
| CN110162692B (en) | User label determination method and device, computer equipment and storage medium | |
| CN113379042A (en) | Business prediction model training method and device for protecting data privacy | |
| US11042880B1 (en) | Authenticating users in the presence of small transaction volumes | |
| CN111861119B (en) | Enterprise risk data processing method and device based on enterprise risk association graph | |
| CN111355725B (en) | Method and device for detecting network intrusion data | |
| CN112613658A (en) | Method and device for predicting rainfall day by day, electronic equipment and storage medium | |
| Fathurahman et al. | Geographically weighted multivariate logistic regression model and its application | |
| CN116383708A (en) | Transaction account identification method and device | |
| CN115758271A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN116108286A (en) | False information detection method, device and equipment based on propagation reconstruction | |
| TWI811574B (en) | Violation detection system, violation detection method and program product | |
| CN110458707B (en) | Behavior evaluation method and device based on classification model and terminal equipment | |
| CN113657996A (en) | Method and device for determining feature contribution degree in federated learning and electronic equipment | |
| Ramos et al. | Objective Bayesian analysis for the differential entropy of the Gamma distribution | |
| CN112259239B (en) | Parameter processing method and device, electronic equipment and storage medium | |
| TWI759702B (en) | Real estate appraisal system | |
| Karavarsamis et al. | The score test for the two‐sample occupancy model | |
| CN116308721B (en) | Information supervision and management method and device, electronic equipment and storage medium | |
| CN114510704B (en) | Risk detection method and device, computing equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |