WO2022199480A1 - 实现隐私保护的多方协同更新模型的方法、装置及系统 - Google Patents

实现隐私保护的多方协同更新模型的方法、装置及系统 Download PDF

Info

Publication number
WO2022199480A1
WO2022199480A1 PCT/CN2022/081672 CN2022081672W WO2022199480A1 WO 2022199480 A1 WO2022199480 A1 WO 2022199480A1 CN 2022081672 W CN2022081672 W CN 2022081672W WO 2022199480 A1 WO2022199480 A1 WO 2022199480A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
round
server
participant
public
Prior art date
Application number
PCT/CN2022/081672
Other languages
English (en)
French (fr)
Inventor
吕灵娟
王维强
漆远
Original Assignee
支付宝(杭州)信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 支付宝(杭州)信息技术有限公司 filed Critical 支付宝(杭州)信息技术有限公司
Priority to US18/283,343 priority Critical patent/US20240176906A1/en
Publication of WO2022199480A1 publication Critical patent/WO2022199480A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6209—Protecting access to data via a platform, e.g. using keys or access control rules to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • One or more embodiments of this specification relate to the field of computer technology, and in particular, to a method, apparatus, and system for implementing a multi-party collaborative update model for privacy protection.
  • federated learning also known as federated learning
  • federated learning has revolutionized traditional centralized machine learning, allowing participants to collaboratively build more accurate models without uploading local data.
  • federated learning is often realized by sharing model parameters or gradients among the participants.
  • model parameters or gradients are usually high-dimensional private data
  • traditional federated learning is accompanied by high communication overhead and privacy to a certain extent. leaks, etc.
  • One or more embodiments of this specification describe a method, device and system for implementing a multi-party collaborative updating model for privacy protection, which can effectively reduce the consumption of communication resources caused by multi-party collaborative modeling, and at the same time play a role in privacy protection.
  • a first aspect provides a method for implementing a multi-party collaborative update model for privacy protection, including: the server delivering the aggregated result of the t-th round of public samples to each participant i; wherein the aggregated result is all
  • the server aggregates the n first prediction results output by the n participants based on their respective local models for the t-th round of public samples; each participant i is based on the t-th round of public samples and the Aggregate the results, and perform a first update on its local i-th model; each participant i performs a second update on the i-th model after the first update based on the fixed first private sample and its sample label in the local sample set; each Each participant i will use the t+1th round of common samples for the next round of iterations, input the second updated i-th model, and send the second output prediction result to the server; the server aggregates the The n second prediction results sent by n participants are used for the next round of iterations; after the multiple rounds of iterations, each
  • a method for implementing a multi-party collaborative update model for privacy protection including: receiving an aggregated result of the t-th round of public samples sent by the server; wherein the aggregated result is the result of the server on all
  • the n participants are obtained by aggregating the n first prediction results output by the t-th round of public samples based on their respective local models; according to the t-th round of public samples and the aggregation results, their local i-th
  • the model is first updated; based on the fixed first private sample and its sample label in the local sample set, the second update is performed on the ith model after the first update; the t+1th round of public samples will be used for the next round of iterations , input the second updated i-th model, and send the output second prediction result to the server, so that the server can aggregate the second prediction result and other prediction results sent by other participants for use in The next round of iteration; after the multiple rounds of iterations, the i-th model after the second update is used as
  • a system for implementing a multi-party collaborative update model for privacy protection including: the server, configured to deliver the aggregated result of the t-th round of public samples to each participant i; wherein, the aggregated result , which is obtained by the server aggregating the n first prediction results output by the n participants based on their respective local models for the t-th round of public samples; The round of public samples and the aggregated results, the first update is performed on its local i-th model; each participant i is also used to update the first update based on the fixed first private sample and its sample label in the local sample set.
  • the i-th model is updated for the second time; each participant i is also used to input the second-updated i-th model for the t+1th round of common samples used in the next round of iterations, and output the second
  • the prediction result is sent to the server; the server is used for aggregating the n second prediction results sent by the n participants for use in the next round of iterations; each participant i is also used in the After multiple rounds of iterations, the i-th model after the second update is used as the model for collaborative updating with other participants.
  • an apparatus for implementing a multi-party collaborative update model for privacy protection comprising: a receiving unit configured to receive the aggregation result of the t-th round of public samples sent by the server; wherein the aggregation result is The server obtains by aggregating the n first prediction results output by the n participants based on their respective local models for the t-th round of public samples; the updating unit is configured to obtain according to the t-th round of public samples and all the first prediction results; The aggregation result is to perform the first update on its local i-th model; the updating unit is also used to perform the first update on the i-th model after the first update based on the fixed first private sample and its sample label in the local sample set Second update; the input unit is used to input the t+1th round of public samples used for the next round of iterations into the second updated ith model, and send the second output prediction result to the server for The server aggregates the second prediction result and other prediction results sent by other participants for the next round
  • a computer storage medium is provided on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute the method of the first aspect or the second aspect.
  • a computing device including a memory and a processor, the memory stores executable code, and when the processor executes the executable code, the method of the first aspect or the second aspect is implemented.
  • One or more embodiments of this specification provide a method, device, and system for a multi-party collaborative update model for privacy protection. Only the prediction results are transmitted between each participant and the server, because the dimensions of the prediction results are usually far lower than the model parameters or gradient, so this scheme can reduce the consumption of communication resources. Furthermore, only the prediction results are transmitted between the participants and the server, so that the participants can cooperate to build models with different network structures, thus greatly improving the general adaptability of federated learning. Finally, when each participant performs the second update to their local model, they only use the fixed private samples in their local sample set, which can reduce the cost of privacy protection.
  • Figure 1 is a schematic diagram of federated learning based on centralized differential privacy
  • Figure 2 is a schematic diagram of federated learning based on local differential privacy
  • Figure 3 is a schematic diagram of federated learning based on shared prediction results
  • FIG. 4 is a schematic diagram of an implementation scenario of an embodiment provided in this specification.
  • FIG. 5 is an interaction diagram of a method for implementing a multi-party collaborative update model for privacy protection provided by an embodiment of this specification
  • FIG. 6 is a schematic diagram of a system for implementing a multi-party collaborative update model for privacy protection provided by an embodiment of this specification
  • FIG. 7 is a schematic diagram of an apparatus for implementing a multi-party collaborative update model for privacy protection according to an embodiment of the present specification.
  • CDP Central Differential Privacy
  • LDP Local Differential Privacy
  • FIG. 1 is a schematic diagram of federated learning based on centralized differential privacy.
  • each participating direction uploads its own model gradients ⁇ w 1 , ⁇ w 2 , . . . , ⁇ w n to a trusted third-party server (hereinafter referred to as a server).
  • the server aggregates the model gradients uploaded by each participant:
  • FIG. 2 is a schematic diagram of federated learning based on local differential privacy.
  • each participant before uploading, each participant first performs local differential privacy through the differential privacy mechanism M on their respective model gradients, and then uses the model gradients (M( ⁇ w 1 ), M( ⁇ w 2 ), M( ⁇ w 2 ), ..., M( ⁇ w n )) are uploaded to the server.
  • the server aggregates the locally differentially private model gradients of each participant:
  • the aggregated model gradient w' is sent to each participant for each participant to update their local model based on it.
  • the model built with this method has poor performance.
  • each participant first pre-trains their local models based on their own local sample sets, then uses the pre-trained model to predict the common sample x p , and uploads their prediction results for the public sample x p to the server, respectively.
  • Y p 1 , Y p 2 , ..., Y p n The server aggregates the prediction results uploaded by each participant:
  • Y p aggregate(Y p 1 +Y p 2 +...+Y p n ),
  • the aggregated result Yp is issued to each participant, so that each participant can update their own pre-training model based on it. It should be noted that although this method can solve the problem that traditional federated learning can only synergistically build models of the same structure, and can reduce the consumption of communication resources (the dimension of the prediction result is much smaller than the model parameters or gradients). However, since the prediction results of each participant are also output by the model, sensitive information of the local sample set will also be leaked to a certain extent. For example, an attacker can conduct member inference attacks based on the prediction results.
  • the prediction result also leaks the sensitive information of the local sample set: Assuming that there are two parties, one of them (hereinafter referred to as the attacker) obtains the aggregated result Yp , according to the aggregated result and its own Prediction results, the prediction results of the other party can be calculated. After that, based on the calculated prediction results and public samples, the attacker can further infer the parameter information of the pre-trained model of another participant. Since the pre-training model of each participant is trained based on its local sample set, that is to say, the pre-training model of each participant incorporates its privatization information, so when an attacker infers the parameter information of the pre-training model, to a certain extent Sensitive information about the participant's local sample set is also leaked.
  • one method is to add random noise that satisfies differential privacy to the prediction results of each participant.
  • this alleviates the privacy problem, it also introduces another new problem: the guarantee of model performance usually requires a very large privacy budget to trade off.
  • the present application proposes a multi-party collaborative model building method for privacy protection.
  • Each participant performs two local model updates, one of which is based on the public sample and the aggregated result of each participant's prediction result. In this way, the consensus of all participants on the public sample can be achieved.
  • the other time is based on the fixed sampled private samples (ie, the first private samples) and their sample labels in the local sample set, so that individualized training of the respective local models of each participant can be realized.
  • the solutions provided by the embodiments of this specification can solve the privacy protection problem in the process of collaboratively constructing different structural models by the participants, and can balance the communication efficiency and model performance at the same time.
  • FIG. 4 is a schematic diagram of an implementation scenario of an embodiment provided in this specification.
  • the scenario of multi-party collaborative updating of the model involves a server and n participants, where n is a positive integer.
  • each participant can be implemented as any device, platform, server or device cluster with computing and processing capabilities.
  • the various parties may be institutions with sample sets of different sizes.
  • the model here may be a business forecasting model for performing forecasting tasks for business objects.
  • the business objects therein may be, for example, pictures, audios, or texts.
  • any participant i maintains the i-th model wi locally and owns the i-th local sample set D i , and the private sample X i in the local sample set D i has the sample label Yi .
  • the server may maintain the public sample set D p , or it may only maintain the description information (including the index and other information of the samples) of each public sample X p in the public sample set D p , while the real public sample set D p is provided by a third party supply.
  • the server may deliver the aggregated result Y p [t] of the t-th round of public samples X p [t] to each participant i.
  • the aggregation result Y p [t] is obtained by the server aggregating n first prediction results output by the n participants based on their respective local models for the t-th round of public samples X p [t].
  • the n first prediction results here can be represented as Y p 1 [t], Y p 2 [t], . . . , Y p n [t], respectively.
  • Each participant i performs the first update to the local i-th model according to the t-th round of public samples X p [t] and the aggregated result Y p [t]. After that, each participant i performs a second update on the i-th model after the first update based on the fixed first private sample X 1 and its sample label Y 1 in the local sample set. Finally, each participant i will use the t+1th round common sample Xp [t+1] for the next iteration, input the second updated ith model, and send the output second prediction result to server, for the server to aggregate n second prediction results corresponding to n participants (i.e.
  • each participant i can use its second updated i-th model as the model updated in collaboration with other participants.
  • the following takes the implementation scenario shown in FIG. 4 as an example to describe the method for implementing the multi-party collaborative update model for privacy protection provided in this specification.
  • FIG. 5 is an interaction diagram of a method for implementing a multi-party collaborative update model for privacy protection provided by an embodiment of the present specification. It should be noted that this method involves multiple rounds of iterations, and FIG. 5 shows the interaction steps included in the t-th (t is a positive integer) round of iteration, and because the interaction process between each participant participating in the t-th round of iteration and the server is similar, Therefore, Fig. 5 mainly shows the interaction steps between any participant participating in the t-th round of iteration (for ease of description, referred to as the first participant) and the server, and the interaction steps between other participants participating in this round of iteration and the server, Please refer to the interaction steps between the first participant and the server.
  • Step 502 the server delivers the aggregated result of the t-th round of public samples to each participant i.
  • the aggregation result here may be obtained by the server aggregating the n first prediction results output by the n participants based on their respective local models for the t-th round of common samples.
  • the above i is a positive integer not exceeding n.
  • the t-th round of common samples here can be one of the following: pictures, texts, and audios.
  • any one of the above n first prediction results may be the identification result of the picture, for example, the picture corresponds to the score of each predetermined category.
  • the above-mentioned t-th round of common samples may be pre-selected by the server before starting multiple rounds of iterations.
  • the above-mentioned t-th round of common samples may be selected by the server in advance in the t-1 round of iterations. The following describes the method for the server to select the t-th round of public samples in two cases.
  • a public sample set is maintained in the server.
  • the server can randomly select several public samples from the public sample set it maintains as the t-th round of public samples, and send them to each Participant i; or, the server may randomly select several public samples from the public sample set maintained by it as the t-th round of public samples, and send an indication of the t-th round of public samples to each participant i. After that, each participant i obtains the t-th round of public samples from the server according to the received instructions.
  • the third party maintains a public sample set
  • the server maintains description information of each public sample in the public sample set.
  • the description information here includes, but is not limited to, information such as the index of each public sample.
  • the server may randomly select the description information of several public samples from the description information of each public sample maintained by the server as the description information of the t-th round of public samples, and deliver it to each participant i. After that, each participant i obtains the t-th round of public samples from the third party according to the received description information.
  • the server selects the t-th round of public samples, and each participant i obtains the t-th round of public samples, it can output the corresponding first prediction result for the t-th round of public samples based on the local i-th model .
  • the network structures of the local models of each participant may be the same or different.
  • the difference here can be divided into the following two situations: First, the types of models are different.
  • the model maintained by one party may be an Artificial Neural Network (ANN), and the model maintained by the other party may be eXtreme Gradient Boosting, XGBoost).
  • the types of models are the same, but the specific expressions are inconsistent.
  • the models maintained by each participant are Convolutional Neural Networks (CNN), but the number of network layers is inconsistent.
  • CNN Convolutional Neural Networks
  • each participant i can obtain the initial model of the corresponding network structure (for different network structures situation); or, the initial model uniformly distributed by the server (for the same network structure) can be received. Then, based on the randomly selected sample subset (hereinafter referred to as the first private sample) and its sample label from the local sample set, the corresponding initial model is updated to obtain its local i-th model.
  • the first private sample the randomly selected sample subset
  • the corresponding initial model is updated to obtain its local i-th model.
  • the local ith model of each participant i may refer to the ith model after the second update in the t-1th iteration.
  • the two update processes of the i-th model after the second update will be described later.
  • each participant i updates the corresponding initial model based on the first private sample and its sample label randomly selected from the local sample set, so as to avoid the leakage of sensitive information of each participant.
  • the reason is that each participant i uses the first private sample randomly selected from its local sample set to update the corresponding initial model, which is equivalent to adding random noise while personalizing the training model, so that the obtained i-th model With differential privacy function.
  • the subsequent data output based on the i-th model meets the requirements of differential privacy.
  • differential privacy protection of the private data of each participant can be ensured.
  • Step 504 each participant i performs a first update of its local i-th model according to the t-th round of public samples and the aggregated results.
  • each participant i can input the t-th round of public samples into its local i-th model to obtain the first local prediction result.
  • the aggregated result is used as the sample label of the t-th round of common samples, and the first prediction loss is determined based on the sample label and the first local prediction result.
  • the first prediction loss may be determined based on the difference between the sample label and the local prediction result.
  • a first update is made to its local i-th model.
  • the back-propagation method can be used to first calculate the update gradient corresponding to the model parameters of the ith model based on the first prediction loss, and then perform the first update on the ith model based on the update gradient. More specifically, the model parameters of the ith model are subtracted from the product of the corresponding update gradient and the learning step size (which is a hyperparameter) to obtain the updated model parameters of the ith model, and it is determined as the first The updated i-th model.
  • the learning step size which is a hyperparameter
  • the first update of the respective local models by each participant based on the public sample can realize the consensus of each participant on the public sample.
  • Step 506 each participant i performs a second update on the i-th model after the first update based on the fixed first private sample and its sample label in the local sample set.
  • the private samples in the local sample set of each participant i may be one of the following: pictures, texts, and audios. It should be understood that when the i-th round of public samples and the first private samples are both pictures, the second updated i-th model here may be a picture recognition model. When the i-th round of public samples and the first private samples are both texts, the i-th model after the second update here may be a text recognition model. And, when the i-th round of public samples and the first private samples are both audio, the second updated i-th model here may be an audio recognition model.
  • the second update in the above step 506 may specifically include: each participant i may input the fixed first private sample in the local sample set into the first updated i-th model to obtain a second local prediction result. Afterwards, a second prediction loss is determined based on the sample label of the first private sample and the second local prediction result. For example, the second prediction loss may be determined based on the difference between the sample label of the first private sample and the second local prediction result. Finally, a second update is performed on the i-th model after the first update according to the second prediction loss. For the step of performing the second update based on the second prediction loss here, reference may be made to the above-mentioned first update, which will not be repeated in this application.
  • the first private sample used in step 506 is the first private sample randomly selected locally and used by each participant i when updating the corresponding initial model. That is, after multiple rounds of iterations start, any participant i performs a second update based on the fixed first private sample in each round of iterations.
  • any participant i performs the second update based on the fixed first private sample in each round of iteration, which can reduce the cost of privacy protection.
  • This step uses a randomly selected private sample, then it is equivalent to adding new random noise in each iteration after the start of multiple iterations, which makes as the number of iterations increases, the added The random noise is getting bigger and bigger.
  • the added random noise usually needs to be balanced by the privacy budget. Specifically, the larger the random noise added, the larger the privacy budget required to ensure the availability of data, which will greatly consume the cost of privacy protection.
  • any participant i uses a fixed first private sample in each round of iterations, so as to ensure that no new random noise will be added after the start of multiple rounds of iterations, thereby ensuring privacy
  • the budget can be fixed. It should be understood that due to the post-processing feature of differential privacy, even in each subsequent iteration, no new random noise will be added. It also meets the requirements of differential privacy.
  • each participant performs a second update on their respective local models based on their respective fixed first private samples, so that individualized training of the respective local models of each participant can be realized.
  • Step 508 each participant i inputs the second updated i-th model for the t+1th round of common samples used in the next round of iteration, and sends the second output prediction result to the server.
  • the t+1th round of common samples here may be selected by the server in the t round of iterations before step 508 is executed.
  • the specific selection method is similar to the above-mentioned selection method of the t-th round of public samples, and will not be repeated here.
  • the second prediction result here is similar to the above-mentioned first prediction result, for example, it may be an image recognition result, a text recognition result, an audio recognition result, or the like.
  • the local i-th model of each participant i has a differential privacy function, so when this step outputs the second prediction result based on it, the outputted second prediction result satisfies the requirement of differential privacy. That is to say, when each participant i sends the corresponding second prediction result to the server, the sensitive information of each participant will not be leaked.
  • Step 510 the server aggregates n copies of the second prediction results sent by the n participants for the next round of iteration.
  • the server may perform a summation, a weighted summation, or a weighted average calculation on the n second prediction results, so as to obtain an aggregated result of the t+1th round of common samples. Afterwards, after entering the t+1th round of iteration, the server may deliver the aggregated result to each participant, and each participant then performs the first and second updates on their local models again, and so on.
  • the above-mentioned steps 502 to 510 are repeatedly performed for multiple times, so that multiple rounds of iterative updating of the respective local models of each participant can be implemented.
  • the model parameters used in each iteration are the parameters updated in the previous round.
  • the termination condition of the iteration may be that the number of iterations reaches a predetermined round or the model parameters converge.
  • each participant i uses its second updated i-th model as the model updated in collaboration with other participants.
  • the model updated collaboratively with other participants may be a picture recognition model.
  • the model that is updated collaboratively with other participants may be an audio recognition model.
  • the model updated in collaboration with other participants may be a text recognition model or the like.
  • this solution can reduce the consumption of communication resources.
  • only the prediction results are transmitted between the participants and the server, so that the participants can cooperate to build models with different network structures, thus greatly improving the general adaptability of federated learning.
  • each participant updates their respective initial models based on the private samples randomly selected from the local sample set to obtain their own local models, which can ensure the differential privacy protection of each participant's data.
  • each participant performs the second update to their respective local models, only the fixed private samples in their respective local sample sets are used, thereby reducing the cost of privacy protection.
  • an embodiment of this specification also provides a system for implementing a multi-party collaborative update model for privacy protection.
  • the system may include servers 602 and n Participants 604 .
  • the server 602 is configured to deliver the aggregated result of the t-th round of public samples to each participant i.
  • the aggregation result is obtained by the server aggregating the n first prediction results output by the n participants based on their respective local models for the t-th round of common samples.
  • Each participant 604 is configured to perform a first update of its local i-th model according to the t-th round of public samples and the aggregated results.
  • the network structures of the respective local models of the n participants are different.
  • Each participant 604 is further configured to perform a second update on the i-th model after the first update based on the fixed first private sample and its sample label in the local sample set.
  • Each participant 604 is further configured to input the t+1th round of common samples used for the next round of iterations into the second updated ith model, and send the second output prediction result to the server 602 .
  • the server 602 is configured to aggregate the n second prediction results sent by the n participants for use in the next round of iteration.
  • the server 602 is specifically configured to: perform a summation, a weighted summation, or a weighted average of the n second prediction results.
  • Each participant 604 is further configured to use its second updated i-th model as the model to be updated collaboratively with other participants after multiple rounds of iterations.
  • the samples in the local sample set of any participant i are pictures, and the model updated in collaboration with other participants is the image recognition model; or, the samples in the local sample set of any participant i are audio, which cooperates with other participants
  • the updated model is an audio recognition model; or, the samples in the local sample set of any participant i are text, and the model updated in collaboration with other participants is a text recognition model.
  • the server 602 maintains a public sample set; the server 602 is further configured to randomly select several public samples from the public sample set maintained by the server 602 as the t+1th round of public samples, and send them to each participant 604 , or; the server 602 is further configured to randomly select a number of public samples from the public sample set maintained by it as the t+1th round of public samples, and send the indication of the t+1th round of public samples to each participant 604; The participating parties 604 are further configured to obtain the t+1th round of public samples from the server 602 according to the received instruction.
  • the third party maintains a public sample set
  • the server 602 maintains description information of each public sample in the public sample set
  • the server 602 is also used to randomly select the description information of several public samples as the t+1th round of public samples.
  • the description information is sent to each participant 604; each participant 604 is also used to obtain the t+1th round of public samples from a third party according to the received description information.
  • An embodiment of this specification provides a system for implementing a multi-party collaborative update model for privacy protection, which can solve the privacy protection problem of each participant in the process of collaboratively building different structural models, and can balance communication efficiency and model performance.
  • an embodiment of the present specification further provides an apparatus for implementing a multi-party collaborative update model for privacy protection.
  • the multi-party here includes the server and n participating parties.
  • the device is set at any participant i among the n participants, and is used to perform multiple rounds of iterations.
  • the apparatus executes any t-th round of iterations through the following units: a receiving unit 702, configured to receive the aggregated results of the t-th round of public samples delivered by the server.
  • the aggregation result is obtained by the server aggregating the n first prediction results output by the n participants based on their respective local models for the t-th round of common samples.
  • the updating unit 704 is configured to perform a first update of the local i-th model according to the t-th round of public samples and the aggregation result.
  • the updating unit 704 is specifically used to: input the t-th round of public samples into its local i-th model to obtain a local prediction result; take the aggregated result as the sample label of the t-th round of public samples, and determine the prediction loss based on it and the local prediction result ; According to the prediction loss, make the first update of its local i-th model.
  • the updating unit 704 is further configured to perform a second update on the i-th model after the first update based on the fixed first private sample and its sample label in the local sample set.
  • the input unit 706 is used for inputting the t+1th round public samples used for the next round of iterations into the second updated ith model, and sending the output second prediction result to the server for the server to aggregate the second Predictions and other predictions sent by other parties for use in the next iteration.
  • the determining unit 708 is configured to, after multiple rounds of iterations, use the second updated i-th model as the model to be updated collaboratively with other participants.
  • the receiving unit 702 is further configured to receive the initial model delivered by the server; the updating unit 704 is further configured to update the initial model based on the fixed first private sample and its sample label in the local sample set to obtain the i-th model.
  • An apparatus for implementing a multi-party collaborative update model for privacy protection provided by an embodiment of this specification can solve the privacy protection problem of each participant in the process of collaboratively building different structural models, and can balance communication efficiency and model performance.
  • a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in conjunction with FIG. 5 .
  • a computing device including a memory and a processor, where executable codes are stored in the memory, and when the processor executes the executable codes, the processing described in conjunction with FIG. 5 is implemented. method.
  • the steps of the method or algorithm described in conjunction with the disclosure of this specification may be implemented in a hardware manner, or may be implemented in a manner in which a processor executes software instructions.
  • the software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable hard disk, CD-ROM or any other form of storage well known in the art in the medium.
  • An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and storage medium may reside in an ASIC.
  • the ASIC may be located in a server.
  • the processor and storage medium may also exist in the server as discrete components.
  • the functions described in the present invention may be implemented in hardware, software, firmware, or any combination thereof.
  • the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
  • Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Information Transfer Between Computers (AREA)
  • Telephonic Communication Services (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本说明书实施例提供一种实现隐私保护的多方协同更新模型的方法、装置及系统，服务器可以向每个参与方i下发第t轮公共样本的聚合结果。每个参与方i根据第t轮公共样本和聚合结果，对本地的第i模型进行第一更新。每个参与方i基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新。每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给服务器，以供服务器聚合对应于n个参与方的n份第二预测结果，并在下一轮迭代开始之后使用。在多轮迭代结束之后，每个参与方i可以将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。

Description

实现隐私保护的多方协同更新模型的方法、装置及系统 技术领域
本说明书一个或多个实施例涉及计算机技术领域，尤其涉及一种实现隐私保护的多方协同更新模型的方法、装置及系统。
背景技术
联邦学习(也称联合学习)的出现革新了传统的集中式机器学习，使得参与方在不需要上传本地数据的情况下，就可以协同构建更精确的模型。
目前，联邦学习往往是通过在各参与方之间共享模型参数或梯度来实现，然而由于模型参数或梯度通常为高维度的隐私数据，因此传统的联邦学习一定程度的伴随着通信开销大、隐私泄露等问题。
发明内容
本说明书一个或多个实施例描述了一种实现隐私保护的多方协同更新模型的方法、装置及系统，可有效降低多方协同建模引起的通信资源消耗，同时起到隐私保护作用。
第一方面，提供了一种实现隐私保护的多方协同更新模型的方法，包括：所述服务器向每个参与方i下发第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；每个参与方i根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；每个参与方i基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器；所述服务器聚合所述n个参与方发送的n份第二预测结果，以用于下一轮迭代；在所述多轮迭代后，每个参与方i将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
第二方面，提供了一种实现隐私保护的多方协同更新模型的方法，包括：接收所述服务器下发的第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器，以供所述服务器聚合所述第二预测结果以及其它参与方发送的其它预测结果，以用于下一轮迭代；在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
第三方面，提供了一种实现隐私保护的多方协同更新模型的系统，包括：所述服务器，用于向每个参与方i下发第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；每个参与方i，用于根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；每个参与方i，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；每个参与方i，还用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器；所述服务器，用于聚合所述n个参与方发送的n份 第二预测结果，以用于下一轮迭代；每个参与方i，还用于在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
第四方面，提供了一种实现隐私保护的多方协同更新模型的装置，包括：接收单元，用于接收所述服务器下发的第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；更新单元，用于根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；所述更新单元，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；输入单元，用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器，以供所述服务器聚合所述第二预测结果以及其它参与方发送的其它预测结果，以用于下一轮迭代；确定单元，用于在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
第五方面，提供了一种计算机存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行第一方面或第二方面的方法。
第六方面，提供了一种计算设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现第一方面或第二方面的方法。
本说明书一个或多个实施例提供的实现隐私保护的多方协同更新模型的方法、装置及系统，各参与方与服务器之间只传输预测结果，由于预测结果的维度通常远远低于模型参数或梯度，从而本方案可以降低通信资源消耗。再者，各参与方与服务器之间只传输预测结果，使得各参与方可以协同构建不同网络结构的模型，由此大大提升了联邦学习的普遍适应性。最后，各参与方对各自本地的模型进行第二更新时，只使用各自本地样本集中固定的私有样本，可以降低隐私保护成本。
附图说明
为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本说明书的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为基于集中式差分隐私的联邦学习示意图；
图2为基于本地差分隐私的联邦学习示意图；
图3为基于共享预测结果的联邦学习示意图；
图4为本说明书提供的一个实施例的实施场景示意图；
图5为本说明书一个实施例提供的实现隐私保护的多方协同更新模型的方法交互图；
图6为本说明书一个实施例提供的实现隐私保护的多方协同更新模型的系统示意图；
图7为本说明书一个实施例提供的实现隐私保护的多方协同更新模型的装置示意图。
具体实施方式
下面结合附图，对本说明书提供的方案进行描述。
如前所述，传统的联邦学习通过在各参与方之间共享模型参数或梯度来实现。其中，主流的方案主要分为两种：第一种，基于集中式差分隐私(Central Differential Privacy， CDP)的联邦学习；第二种，基于本地差分隐私(Local Differential Privacy，LDP)的联邦学习。以下结合附图对该两种方法进行说明。
图1为基于集中式差分隐私的联邦学习示意图。图1中，首先，各参与方向可信的第三方服务器(以下简称服务器)上传各自的模型梯度Δw 1、Δw 2、…、Δw n。之后，服务器聚合各参与方上传的模型梯度：
aggregate(Δw 1+Δw 2+…+Δw n)，
并在聚合的模型梯度中通过差分隐私机制M添加噪声M(aggregate(…))，最后，服务器将添加噪声后的模型梯度w’下发给各参与方，以供各参与方基于其更新各自本地的模型。然而，由于目前受信任的第三方在实际场景中很少见，而且容易遭受窃听者的攻击，因此，该方法的适用性较差。此外，该种联邦学习只适应于参与方数目较多的场景，在只有少数参与方的情况下，所构建的模型性能较差。
图2为基于本地差分隐私的联邦学习示意图。图2中，各参与方在上传之前，先在各自的模型梯度上通过差分隐私机制M执行本地差分隐私，之后将经过本地差分隐私的模型梯度(M(Δw 1)、M(Δw 2)、…、M(Δw n))上传至服务器。最后，服务器聚合各参与方的经过本地差分隐私的模型梯度：
aggregate(M(Δw 1)+M(Δw 2)+…+M(Δw n))，
并将聚合的模型梯度w’下发至各参与方，以供各参与方基于其更新各自本地的模型。然而，由于本地差分隐私会导致较大的性能损失，因此，采用该方法构建的模型性能较差。
可见，上述两种联邦学习均存在各自的缺陷。此外，由于该两种联邦学习均通过共享模型参数或梯度来实现，因此该两种方法只适用于多方协同构建相同网络结构模型(简称同结构模型)的场景。
为了克服传统的联邦学习的种种缺陷，部分改进方法提出通过在各参与方之间共享模型预测结果(以下简称预测结果)来实现联邦学习，具体可以参见图3所示。
图3中，各参与方首先基于各自的本地样本集预训练各自本地的模型，之后利用该预训练模型针对公共样本x p进行预测，并向服务器上传各自针对公共样本x p的预测结果，分别表示为Y p 1、Y p 2、…、Y p n。服务器聚合各参与方上传的预测结果：
Y p＝aggregate(Y p 1+Y p 2+…+Y p n)，
并将聚合结果Y p下发给各参与方，以供各参与方基于其更新各自的预训练模型。需要说明，采用该方法虽然能够解决传统的联邦学习只能协同构建同结构模型的问题，且能够降低通信资源消耗(预测结果的维度远远小于模型参数或梯度)。但是，由于各参与方的预测结果本身也是由模型输出的结果，一定程度上也会泄露本地样本集的敏感信息，比如，攻击者可以基于预测结果进行成员推理攻击。
以下对预测结果也会泄露本地样本集的敏感信息进行说明：假设有两个参与方，其中的一个参与方(以下称攻击者)在获取到聚合结果Y p之后，根据该聚合结果以及自身的预测结果，可以计算出另一个参与方的预测结果。之后，攻击者基于计算出的预测结果以及公共样本，可以进一步推测出另一个参与方的预训练模型的参数信息。由于各参与方的预训练模型基于其本地样本集训练得到，也就是说各参与方的预训练模型融入了其私有化信息，从而当攻击者推测出预训练模型的参数信息时，一定程度上也泄露了参与方的本地样本集的敏感信息。
为解决上述敏感信息泄露的问题，一种方法是将满足差分隐私的随机噪声添加到各 参与方的预测结果中。尽管这样可以缓解隐私问题，但是同时带来了另一个新的问题：模型性能的保障通常需要非常大的隐私预算才能进行权衡。
基于此，本申请提出了一种实现隐私保护的多方协同构建模型的方法，各参与方执行两次本地模型更新，其中一次是基于公共样本以及各参与方针对其的预测结果的聚合结果执行，由此可以实现各参与方对公共样本的共识。另一次是基于本地样本集中固定的采样私有样本(即第一私有样本)及其样本标签执行，由此可以实现对各参与方各自本地的模型进行个性化训练。
总而言之，本说明书实施例提供的方案可以解决各参与方在协同构建不同结构模型的过程中的隐私保护问题，同时可以权衡通信效率和和模型性能。
图4为本说明书提供的一个实施例的实施场景示意图。图4中，多方协同更新模型的场景涉及服务器和n个参与方，其中，n为正整数。其中，各参与方可以实现为任何具有计算、处理能力的设备、平台、服务器或设备集群。在一个具体例子中，各参与方可以为具有不同规模样本集的机构。需要说明，服务器和各参与方要在保护数据隐私的情况下，协同更新各参与方各自本地的模型。这里的模型可以是用于执行针对业务对象的预测任务的业务预测模型。其中的业务对象例如可以为图片、音频或文本等。
图4中，任意的参与方i在本地维护有第i模型w i，并拥有第i本地样本集D i，该本地样本集D i中的私有样本X i具有样本标签Y i。服务器可以维护有公共样本集D p，或者也可以只维护有公共样本集D p中各公共样本X p的描述信息(包括样本的索引等信息)，而真正的公共样本集D p由第三方提供。
具体地，在第t轮迭代中，服务器可以向每个参与方i下发第t轮公共样本X p[t]的聚合结果Y p[t]。其中，该聚合结果Y p[t]，是服务器对n个参与方基于各自本地的模型针对第t轮公共样本X p[t]输出的n份第一预测结果进行聚合得到。这里的n份第一预测结果可以分别表示为Y p 1[t]、Y p 2[t]、…、Y p n[t]。每个参与方i根据第t轮公共样本X p[t]和聚合结果Y p[t]，对本地的第i模型进行第一更新。之后，每个参与方i基于本地样本集中固定的第一私有样本X 1及其样本标签Y 1，对第一更新后的第i模型进行第二更新。最后，每个参与方i将用于下一轮迭代的第t+1轮公共样本X p[t+1]，输入第二更新后的第i模型，并将输出的第二预测结果发送给服务器，以供服务器聚合对应于n个参与方的n份第二预测结果(即Y p 1[t+1]、Y p 2[t+1]、…、Y p n[t+1])，并在下一轮迭代开始之后，将聚合得到的聚合结果Y P[t+1]下发至各参与方。应理解，在多轮迭代结束之后，每个参与方i可以将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
以下以图4示出的实施场景为例，对本说明书提供的实现隐私保护的多方协同更新模型的方法进行说明。
图5为本说明书一个实施例提供的实现隐私保护的多方协同更新模型的方法交互图。需要说明，该方法涉及多轮迭代，图5中示出其中第t(t为正整数)轮迭代包括的交互步骤，并且，因参与第t轮迭代的各个参与方与服务器的交互过程相近，所以图5中主要示出参与该第t轮迭代的任意一个参与方(为便于描述，称作第一参与方)与服务器的交互步骤，参与该轮迭代的其它参与方与服务器的交互步骤，可以参见该第一参与方与服务器的交互步骤。可以理解，通过重复执行其中示出的交互步骤，可以实现对各参与方各自维护的模型的多轮迭代更新，进而将最后一轮迭代更新得到的模型，作为各自最终使用的模型。如图5所示，该方法可以包括如下步骤：步骤502，服务器向每个参与方i下发第t轮公共样本的聚合结果。
这里的聚合结果可以是服务器对n个参与方基于各自本地的模型针对第t轮公共样本输出的n份第一预测结果进行聚合得到。上述i为不超过n的正整数。
这里的第t轮公共样本可以为以下中的一种：图片、文本以及音频等。以第t轮公共样本为图片为例来说，上述n份第一预测结果中任一份第一预测结果可以为图片的识别结果，比如，图片对应于各预定分类的打分。
需要说明，当第t轮迭代为首轮迭代时，上述第t轮公共样本可以是由服务器在多轮迭代开始之前预先选取得到。当第t轮迭代为非首轮迭代时，上述第t轮公共样本可以是由服务器提前在t-1轮迭代选取得到。以下分两种情况，对服务器选取第t轮公共样本的方法进行说明。
第一种，服务器中维护有公共样本集。在该种情况下，在多轮迭代开始之前或者在第t-1轮迭代，服务器可以从其维护的公共样本集中，随机选取若干公共样本作为第t轮公共样本，并将其发送给每个参与方i；或者，服务器可以从其维护的公共样本集中，随机选取若干公共样本作为第t轮公共样本，并将第t轮公共样本的指示发送给每个参与方i。之后，每个参与方i根据接收的指示，从服务器获取第t轮公共样本。
第二种，第三方中维护有公共样本集，服务器中维护有公共样本集中各公共样本的描述信息。这里的描述信息包括但不限于各公共样本的索引等信息。在该种情况下，服务器可以从其维护的各公共样本的描述信息中，随机选取若干公共样本的描述信息作为第t轮公共样本的描述信息，并将其下发至每个参与方i。之后，每个参与方i根据接收的描述信息，从第三方获取第t轮公共样本。
需要说明，在服务器选取出第t轮公共样本，且每个参与方i在获取到第t轮公共样本之后，其可以基于本地的第i模型针对第t轮公共样本输出对应的第一预测结果。
在本说明书中，各参与方本地的模型的网络结构可以相同，也可不同。其中，这里的不同可分为如下两种情况：第一种，模型的类型不同。比如，在有两个参与方的情况下，其中的一个参与方维护的模型可以是人工神经网络(Artificial Neural Network，ANN)，另一个参与方维护的模型可以是极端梯度提升(eXtreme Gradient Boosting，XGBoost)。第二种，模型的类型相同，但具体表现形式不一致，比如，各参与方维护的模型均为卷积神经网络(Convolutional Neural Networks，CNN)，但其网络层数不一致。
具体地，当第t轮迭代为首轮迭代时，每个参与方i本地的第i模型经过以下的预训练步骤得到：每个参与方i可以获取对应网络结构的初始模型(针对不同网络结构的情况)；或者，可以接收服务器统一下发的初始模型(针对相同网络结构的情况)。然后基于本地样本集中随机选取的样本子集(以下称第一私有样本)及其样本标签，更新对应的初始模型，得到其本地的第i模型。
而当第t轮迭代为非首轮迭代时，每个参与方i本地的第i模型可以是指第t-1轮迭代中第二更新后的第i模型。其中，关于第二更新后的第i模型的两次更新过程后续说明。
需要说明，本说明书实施例中，每个参与方i基于本地样本集中随机选取的第一私有样本及其样本标签，更新对应的初始模型可以避免各参与方的敏感信息泄露问题。原因在于：每个参与方i使用从其本地样本集中随机选取的第一私有样本，更新对应的初始模型时，相当于在个性化训练模型的同时添加了随机噪声，从而所得到的第i模型具有了差分隐私功能。而根据差分隐私的后处理特性，后续基于该第i模型输出的数据均满足差分隐私的要求。由此，可以确保各参与方隐私数据的差分隐私保护。
步骤504，每个参与方i根据第t轮公共样本和聚合结果，对其本地的第i模型进行第一更新。
具体地，每个参与方i可将第t轮公共样本输入其本地的第i模型，得到第一本地预 测结果。之后，将聚合结果作为第t轮公共样本的样本标签，并基于该样本标签以及第一本地预测结果，确定第一预测损失。比如，可基于该样本标签与本地预测结果的差值，确定第一预测损失。最后，根据第一预测损失，对其本地的第i模型进行第一更新。
比如，可以利用反向传播法，基于第一预测损失，先计算出第i模型的模型参数对应的更新梯度，再基于该更新梯度对第i模型进行第一更新。更具体地，将第i模型的模型参数减去其所对应的更新梯度与学习步长(为超参)之间的乘积，得到第i模型更新后的模型参数，并将其确定为第一更新后的第i模型。
应理解，该步骤中，各参与方基于公共样本，对各自本地的模型进行的第一更新，可以实现各参与方对公共样本的共识。
步骤506，每个参与方i基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新。
其中，每个参与方i本地样本集中的私有样本可以为以下中的一种：图片、文本以及音频等。应理解，在上述第i轮公共样本、第一私有样本均为图片时，这里第二更新后的第i模型可以为图片识别模型。在上述第i轮公共样本、第一私有样本均为文本时，这里第二更新后的第i模型可以为文本识别模型。以及，在上述第i轮公共样本、第一私有样本均为音频时，这里第二更新后的第i模型可以为音频识别模型。
上述步骤506中的第二更新具体可以包括：每个参与方i可以将本地样本集中固定的第一私有样本输入第一更新后的第i模型，得到第二本地预测结果。之后，基于第一私有样本的样本标签以及第二本地预测结果，确定第二预测损失。比如，可以基于第一私有样本的样本标签与第二本地预测结果的差值，确定第二预测损失。最后，根据第二预测损失，对第一更新后的第i模型进行第二更新。这里的基于第二预测损失进行第二更新的步骤可参见上述第一更新，本申请在此不复赘述。
需要说明，步骤506中使用的第一私有样本即为每个参与方i在更新对应的初始模型时所使用的从本地随机选取的第一私有样本。也就是说，在多轮迭代开始之后，任意的参与方i在每轮迭代中都基于固定的第一私有样本进行第二更新。
在此说明，本说明书实施例中，任意的参与方i在每轮迭代中都基于固定的第一私有样本进行第二更新，可以降低隐私保护成本。原因如下：试想如果该步骤使用的是随机选取的私有样本，那么相当于在多轮迭代开始之后，在每轮迭代中都会添加新的随机噪声，这使得随着迭代轮次的增加，所添加的随机噪声越来越大。而根据差分隐私的特性，添加的随机噪声通常需要通过隐私预算来平衡。具体而言，添加的随机噪声越大，那么为了保证数据的可用性，所需要的隐私预算就越大，这将会极大地耗费隐私保护成本。为此，在本说明书实施例中，任意的参与方i在每轮迭代中都使用固定的第一私有样本，这样就可以确保在多轮迭代开始之后不会再添加新的随机噪声，从而隐私预算就可以固定下来。应理解，由于差分隐私具有后处理特征，因此即便在后续的每轮迭代中，不会再添加新的随机噪声，通过每个参与方i本地的具有差分隐私功能的第i模型输出的数据，也满足差分隐私的要求。
总而言之，本说明书实施例提供的方案可以保证在比较小的隐私预算下，对各参与方的数据进行差分隐私保护，且该差分隐私保护，不会影响到各参与方的模型性能。
此外，该步骤中，各参与方基于各自的固定的第一私有样本，对各自本地的模型进行第二更新，可以实现对各参与方各自本地的模型的个性化训练。
步骤508，每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给服务器。
这里的第t+1轮公共样本可以是在t轮迭代中，在执行步骤508之前，由服务器选取得到。具体的选取方法同上述第t轮公共样本的选取方法相类似，在此不复赘述。此外，这里的第二预测结果与上述第一预测结果相类似，比如，其可以为图像的识别结果、文本的识别结果或者音频的识别结果等。
需要说明，如前所述，每个参与方i本地的第i模型具有差分隐私功能，从而该步骤在基于其输出第二预测结果时，所输出的第二预测结果满足差分隐私的要求。也就是说，每个参与方i在向服务器发送对应的第二预测结果时，不会造成各参与方敏感信息的泄露。
步骤510，服务器聚合n个参与方发送的n份第二预测结果，以用于下一轮迭代。
比如，服务器可对n份第二预测结果进行求和、加权求和或者求加权平均等，以获得第t+1轮公共样本的聚合结果。之后，在进入第t+1轮迭代之后，服务器可将该聚合结果下发给各参与方，然后各参与方再次针对各自本地的模型进行第一和第二更新等等。
换句话说，在本说明书实施例中，上述步骤502-步骤510是重复多次执行的，由此可以实现对各参与方各自本地的模型多轮迭代更新。且每次迭代所使用的模型参数是上一轮更新后的参数。该迭代的终止条件可以为迭代次数达到预定轮次或者模型参数收敛。
在多轮迭代后，每个参与方i将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
以任意的参与方i为例来说，在前述的公共样本以及其本地样本集中的样本为图片的情况下，那么其与其它参与方协同更新的模型可以为图片识别模型。在前述的公共样本以及其本地样本集中的样本为音频的情况下，那么其与其它参与方协同更新的模型可以为音频识别模型。在前述的公共样本以及其本地样本集中的样本为文本的情况下，那么其与其它参与方协同更新的模型可以为文本识别模型等等。
应理解，在各参与方本地的模型网络结构不相同的情况下，拥有大规模本地样本集的参与方可以更新得到一个复杂的模型，而拥有少量本地样本集的参与方可以更新得到一个比较简单的模型，从而本方案可以实现拥有不同规模样本集参与方的无障碍合作。
综合以上，本说明书实施例中，由于各参与方与服务器之间只传输预测结果，而预测结果的维度通常远远低于模型参数或梯度，从而本方案可以降低通信资源消耗。此外，各参与方与服务器之间只传输预测结果，使得各参与方可以协同构建不同网络结构的模型，由此大大提升了联邦学习的普遍适应性。此外，各参与方基于本地样本集中随机选取的私有样本，更新各自的初始模型，以获取各自本地的模型，可以确保各参与方数据的差分隐私保护。最后，各参与方对各自本地的模型进行第二更新时，只使用各自本地样本集中固定的私有样本，由此可以降低隐私保护成本。
与上述实现隐私保护的多方协同更新模型的方法对应地，本说明书一个实施例还提供的一种实现隐私保护的多方协同更新模型的系统，如图6所示，该系统可以包括服务器602和n个参与方604。
服务器602，用于向每个参与方i下发第t轮公共样本的聚合结果。其中，该聚合结果，是服务器对n个参与方基于各自本地的模型针对第t轮公共样本输出的n份第一预测结果进行聚合得到。
每个参与方604，用于根据第t轮公共样本和聚合结果，对其本地的第i模型进行第一更新。
其中，n个参与方各自本地的模型的网络结构不同。
每个参与方604，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新。
每个参与方604，还用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给服务器602。
服务器602，用于聚合n个参与方发送的n份第二预测结果，以用于下一轮迭代。
服务器602具体用于：对n份第二预测结果进行求和、加权求和或者求加权平均。
每个参与方604，还用于在多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
其中，任意的参与方i本地样本集中的样本为图片，其与其它参与方协同更新的模型为图片识别模型；或者，任意的参与方i本地样本集中的样本为音频，其与其它参与方协同更新的模型为音频识别模型；或者，任意的参与方i本地样本集中的样本为文本，其与其它参与方协同更新的模型为文本识别模型。
可选地，服务器602中维护有公共样本集；服务器602，还用于从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将其发送给每个参与方604，或者；服务器602，还用于从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将第t+1轮公共样本的指示发送给每个参与方604；每个参与方604，还用于根据接收的指示，从服务器602获取第t+1轮公共样本。
可选地，第三方中维护有公共样本集，服务器602中维护有公共样本集中各公共样本的描述信息；服务器602，还用于随机选取若干公共样本的描述信息作为第t+1轮公共样本的描述信息，并将其下发至每个参与方604；每个参与方604，还用于根据接收的描述信息，从第三方获取第t+1轮公共样本。
说明书上述实施例系统的各功能模块的功能，可以通过上述方法实施例的各步骤来实现，因此，本说明书一个实施例提供的系统的具体工作过程，在此不复赘述。
本说明书一个实施例提供的实现隐私保护的多方协同更新模型的系统，可以解决各参与方在协同构建不同结构模型的过程中的隐私保护问题，同时可以权衡通信效率和和模型性能。
与上述实现隐私保护的多方协同更新模型的方法对应地，本说明书一个实施例还提供的一种实现隐私保护的多方协同更新模型的装置。这里的多方包括服务器和n个参与方。该装置设置于n个参与方中任意的参与方i，用于执行多轮迭代。如图7所示，该装置通过其包括的以下单元执行其中任意的第t轮迭代：接收单元702，用于接收服务器下发的第t轮公共样本的聚合结果。其中，该聚合结果，是服务器对n个参与方基于各自本地的模型针对第t轮公共样本输出的n份第一预测结果进行聚合得到。
更新单元704，用于根据第t轮公共样本和聚合结果，对其本地的第i模型进行第一更新。
更新单元704具体用于：将第t轮公共样本输入其本地的第i模型，得到本地预测结果；将聚合结果作为第t轮公共样本的样本标签，并基于其以及本地预测结果，确定预测损失；根据预测损失，对其本地的第i模型进行第一更新。
更新单元704，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新。
输入单元706，用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给服务器，以供服务器聚合第二预测结果以及其它 参与方发送的其它预测结果，以用于下一轮迭代。
确定单元708，用于在多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
可选地，接收单元702，还用于接收服务器下发的初始模型；更新单元704，还用于基于本地样本集中固定的第一私有样本及其样本标签，更新初始模型，得到第i模型。
本说明书上述实施例装置的各功能模块的功能，可以通过上述方法实施例的各步骤来实现，因此，本说明书一个实施例提供的装置的具体工作过程，在此不复赘述。
本说明书一个实施例提供的实现隐私保护的多方协同更新模型的装置，可以解决各参与方在协同构建不同结构模型的过程中的隐私保护问题，同时可以权衡通信效率和和模型性能。
根据另一方面的实施例，还提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行结合图5所描述的方法。
根据再一方面的实施例，还提供一种计算设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现结合图5所述的方法。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于设备实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
结合本说明书公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于RAM存储器、闪存、ROM存储器、EPROM存储器、EEPROM存储器、寄存器、硬盘、移动硬盘、CD-ROM或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于服务器中。当然，处理器和存储介质也可以作为分立组件存在于服务器中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本发明所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
以上所述的具体实施方式，对本说明书的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本说明书的具体实施方式而已，并不用于限定本说明书的保护范围，凡在本说明书的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本说明书的保护范围之内。

Claims (20)

  1. 一种实现隐私保护的多方协同更新模型的方法，所述多方包括服务器和n个参与方；所述方法包括多轮迭代，其中任意的第t轮迭代包括：
    所述服务器向每个参与方i下发第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；
    每个参与方i根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；
    每个参与方i基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；
    每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器；
    所述服务器聚合所述n个参与方发送的n份第二预测结果，以用于下一轮迭代；
    在所述多轮迭代后，每个参与方i将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
  2. 根据权利要求1所述的方法，所述服务器中维护有公共样本集；
    在所述每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型之前，所述方法还包括：
    所述服务器从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将其发送给每个参与方i，或者；
    所述服务器从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将所述第t+1轮公共样本的指示发送给每个参与方i；
    每个参与方i根据所述指示，从所述服务器获取所述第t+1轮公共样本。
  3. 根据权利要求1所述的方法，第三方中维护有公共样本集，所述服务器中维护有所述公共样本集中各公共样本的描述信息；
    在所述每个参与方i将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型之前，所述方法还包括：
    所述服务器随机选取若干公共样本的描述信息作为所述第t+1轮公共样本的描述信息，并将其下发至每个参与方i；
    每个参与方i根据所述描述信息，从第三方获取所述第t+1轮公共样本。
  4. 根据权利要求1所述的方法，
    任意的参与方i本地样本集中的样本为图片，其与其它参与方协同更新的模型为图片识别模型；或者，
    任意的参与方i本地样本集中的样本为音频，其与其它参与方协同更新的模型为音频识别模型；或者，
    任意的参与方i本地样本集中的样本为文本，其与其它参与方协同更新的模型为文本识别模型。
  5. 根据权利要求1所述的方法，所述n个参与方各自本地的模型的网络结构不同。
  6. 根据权利要求1所述的方法，所述服务器聚合所述n个参与方发送的n份第二预测结果，包括：
    所述服务器对所述n份第二预测结果进行求和、加权求和或者求加权平均。
  7. 一种实现隐私保护的多方协同更新模型的方法，所述多方包括服务器和n个参与方；所述方法通过所述n个参与方中任意的参与方i执行；所述方法包括多轮迭代，其中任意的第t轮迭代包括：
    接收所述服务器下发的第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一 预测结果进行聚合得到；
    根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；
    基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；
    将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器，以供所述服务器聚合所述第二预测结果以及其它参与方发送的其它预测结果，以用于下一轮迭代；
    在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
  8. 根据权利要求7所述的方法，所述第i模型通过以下步骤获得：
    接收所述服务器下发的初始模型；
    基于本地样本集中固定的第一私有样本及其样本标签，更新所述初始模型，得到所述第i模型。
  9. 根据权利要求7所述的方法，所述根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新，包括：
    将所述第t轮公共样本输入其本地的第i模型，得到本地预测结果；
    将所述聚合结果作为所述第t轮公共样本的样本标签，并基于其以及所述本地预测结果，确定预测损失；
    根据所述预测损失，对其本地的第i模型进行第一更新。
  10. 一种实现隐私保护的多方协同更新模型的系统，包括服务器和n个参与方；
    所述服务器，用于向每个参与方i下发第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；
    每个参与方i，用于根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；
    每个参与方i，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；
    每个参与方i，还用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器；
    所述服务器，用于聚合所述n个参与方发送的n份第二预测结果，以用于下一轮迭代；
    每个参与方i，还用于在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
  11. 根据权利要求10所述的系统，所述服务器中维护有公共样本集；
    所述服务器，还用于从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将其发送给每个参与方i，或者；
    所述服务器，还用于从其维护的公共样本集中随机选取若干公共样本作为第t+1轮公共样本，并将所述第t+1轮公共样本的指示发送给每个参与方i；
    每个参与方i，还用于根据所述指示，从所述服务器获取所述第t+1轮公共样本。
  12. 根据权利要求10所述的系统，第三方中维护有公共样本集，所述服务器中维护有所述公共样本集中各公共样本的描述信息；
    所述服务器，还用于随机选取若干公共样本的描述信息作为所述第t+1轮公共样本的描述信息，并将其下发至每个参与方i；
    每个参与方i，还用于根据所述描述信息，从第三方获取所述第t+1轮公共样本。
  13. 根据权利要求10所述的系统，
    任意的参与方i本地样本集中的样本为图片，其与其它参与方协同更新的模型为图 片识别模型；或者，
    任意的参与方i本地样本集中的样本为音频，其与其它参与方协同更新的模型为音频识别模型；或者，
    任意的参与方i本地样本集中的样本为文本，其与其它参与方协同更新的模型为文本识别模型。
  14. 根据权利要求10所述的系统，所述n个参与方各自本地的模型的网络结构不同。
  15. 根据权利要求10所述的系统，所述服务器具体用于：对所述n份第二预测结果进行求和、加权求和或者求加权平均。
  16. 一种实现隐私保护的多方协同更新模型的装置，所述多方包括服务器和n个参与方；所述装置设置于所述n个参与方中任意的参与方i，用于执行多轮迭代，所述装置通过其包括的以下单元执行其中任意的第t轮迭代：
    接收单元，用于接收所述服务器下发的第t轮公共样本的聚合结果；其中，所述聚合结果，是所述服务器对所述n个参与方基于各自本地的模型针对所述第t轮公共样本输出的n份第一预测结果进行聚合得到；
    更新单元，用于根据所述第t轮公共样本和所述聚合结果，对其本地的第i模型进行第一更新；
    所述更新单元，还用于基于本地样本集中固定的第一私有样本及其样本标签，对第一更新后的第i模型进行第二更新；
    输入单元，用于将用于下一轮迭代的第t+1轮公共样本，输入第二更新后的第i模型，并将输出的第二预测结果发送给所述服务器，以供所述服务器聚合所述第二预测结果以及其它参与方发送的其它预测结果，以用于下一轮迭代；
    确定单元，用于在所述多轮迭代后，将其第二更新后的第i模型，作为其与其它参与方协同更新的模型。
  17. 根据权利要求16所述的装置，
    所述接收单元，还用于接收所述服务器下发的初始模型；
    所述更新单元，还用于基于本地样本集中固定的第一私有样本及其样本标签，更新所述初始模型，得到所述第i模型。
  18. 根据权利要求16所述的装置，所述更新单元具体用于：
    将所述第t轮公共样本输入其本地的第i模型，得到本地预测结果；
    将所述聚合结果作为所述第t轮公共样本的样本标签，并基于其以及所述本地预测结果，确定预测损失；
    根据所述预测损失，对其本地的第i模型进行第一更新。
  19. 一种计算机可读存储介质，其上存储有计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行权利要求1至9中任一项所述的方法。
  20. 一种计算设备，包括存储器和处理器，其中，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现权利要求1至9中任一项所述的方法。
PCT/CN2022/081672 2021-03-25 2022-03-18 实现隐私保护的多方协同更新模型的方法、装置及系统 WO2022199480A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/283,343 US20240176906A1 (en) 2021-03-25 2022-03-18 Methods, apparatuses, and systems for collaboratively updating model by multiple parties for implementing privacy protection

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110320900.4A CN112948885B (zh) 2021-03-25 2021-03-25 实现隐私保护的多方协同更新模型的方法、装置及系统
CN202110320900.4 2021-03-25

Publications (1)

Publication Number Publication Date
WO2022199480A1 true WO2022199480A1 (zh) 2022-09-29

Family

ID=76227771

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/081672 WO2022199480A1 (zh) 2021-03-25 2022-03-18 实现隐私保护的多方协同更新模型的方法、装置及系统

Country Status (3)

Country Link
US (1) US20240176906A1 (zh)
CN (1) CN112948885B (zh)
WO (1) WO2022199480A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112948885B (zh) * 2021-03-25 2023-10-27 支付宝(杭州)信息技术有限公司 实现隐私保护的多方协同更新模型的方法、装置及系统
CN114239064A (zh) * 2021-12-20 2022-03-25 杭州趣链科技有限公司 隐私计算方法、装置、电子设备及计算机可读存储介质
CN116415676A (zh) * 2021-12-29 2023-07-11 新智我来网络科技有限公司 联合学习中的预测方法及装置
CN115081642B (zh) * 2022-07-19 2022-11-15 浙江大学 一种多方协同更新业务预测模型的方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110647765A (zh) * 2019-09-19 2020-01-03 济南大学 协同学习框架下基于知识迁移的隐私保护方法及系统
CN111062487A (zh) * 2019-11-28 2020-04-24 支付宝(杭州)信息技术有限公司 基于数据隐私保护的机器学习模型特征筛选方法及装置
CN112329940A (zh) * 2020-11-02 2021-02-05 北京邮电大学 一种结合联邦学习与用户画像的个性化模型训练方法及系统
CN112948885A (zh) * 2021-03-25 2021-06-11 支付宝(杭州)信息技术有限公司 实现隐私保护的多方协同更新模型的方法、装置及系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200272945A1 (en) * 2019-02-21 2020-08-27 Hewlett Packard Enterprise Development Lp System and method of decentralized model building for machine learning and data privacy preserving using blockchain

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110647765A (zh) * 2019-09-19 2020-01-03 济南大学 协同学习框架下基于知识迁移的隐私保护方法及系统
CN111062487A (zh) * 2019-11-28 2020-04-24 支付宝(杭州)信息技术有限公司 基于数据隐私保护的机器学习模型特征筛选方法及装置
CN112329940A (zh) * 2020-11-02 2021-02-05 北京邮电大学 一种结合联邦学习与用户画像的个性化模型训练方法及系统
CN112948885A (zh) * 2021-03-25 2021-06-11 支付宝(杭州)信息技术有限公司 实现隐私保护的多方协同更新模型的方法、装置及系统

Also Published As

Publication number Publication date
US20240176906A1 (en) 2024-05-30
CN112948885A (zh) 2021-06-11
CN112948885B (zh) 2023-10-27

Similar Documents

Publication Publication Date Title
WO2022199480A1 (zh) 实现隐私保护的多方协同更新模型的方法、装置及系统
CN113191484B (zh) 基于深度强化学习的联邦学习客户端智能选取方法及系统
CN113408746B (zh) 一种基于区块链的分布式联邦学习方法、装置及终端设备
WO2022257730A1 (zh) 实现隐私保护的多方协同更新模型的方法、装置及系统
US11520899B2 (en) System and method for machine learning architecture with adversarial attack defense
Gao et al. FGFL: A blockchain-based fair incentive governor for Federated Learning
WO2019228232A1 (zh) 对话系统间的知识分享方法、对话方法及装置
CN113222175B (zh) 信息处理方法及系统
WO2023124296A1 (zh) 基于知识蒸馏的联合学习训练方法、装置、设备及介质
CN113469373A (zh) 基于联邦学习的模型训练方法、系统、设备及存储介质
CN111460528A (zh) 一种基于Adam优化算法的多方联合训练方法及系统
US11100406B2 (en) Knowledge network platform
US20220101189A1 (en) Federated inference
Adams AI-powered social bots
WO2022222795A1 (zh) 用于隐私保护的模型的训练方法、系统及装置
WO2023174018A1 (zh) 一种纵向联邦学习方法、装置、系统、设备及存储介质
US20230006980A1 (en) Systems and methods for securely training a decision tree
CN112292696A (zh) 确定执行设备的动作选择方针
CN116627970A (zh) 一种基于区块链和联邦学习的数据共享方法及装置
CN112541570A (zh) 一种多模型训练方法、装置、电子设备及存储介质
Wang et al. Social-aware clustered federated learning with customized privacy preservation
Moudoud et al. Multi-tasking federated learning meets blockchain to foster trust and security in the Metaverse
US20230394583A1 (en) Customer partner program methods and systems
Chen et al. Advances in Robust Federated Learning: Heterogeneity Considerations
Wu et al. Bayesian stochastic configuration networks for robust data modeling

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22774141

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18283343

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 11202307137U

Country of ref document: SG

122 Ep: pct application non-entry in european phase

Ref document number: 22774141

Country of ref document: EP

Kind code of ref document: A1