WO2022048195A1 - 纵向联邦建模方法、装置、设备及计算机可读存储介质 - Google Patents

纵向联邦建模方法、装置、设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2022048195A1
WO2022048195A1 PCT/CN2021/095251 CN2021095251W WO2022048195A1 WO 2022048195 A1 WO2022048195 A1 WO 2022048195A1 CN 2021095251 W CN2021095251 W CN 2021095251W WO 2022048195 A1 WO2022048195 A1 WO 2022048195A1
Authority
WO
WIPO (PCT)
Prior art keywords
party
participant
federation
calculation
federated
Prior art date
Application number
PCT/CN2021/095251
Other languages
English (en)
French (fr)
Inventor
康焱
Original Assignee
深圳前海微众银行股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海微众银行股份有限公司 filed Critical 深圳前海微众银行股份有限公司
Publication of WO2022048195A1 publication Critical patent/WO2022048195A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present application relates to the technical field of federated learning, and in particular, to a vertical federated modeling method, apparatus, device, and computer-readable storage medium.
  • Federated learning is a machine learning model established by using the data of multiple participants while protecting data privacy.
  • Vertical federated learning uses different features of overlapping samples owned by multiple participants to establish a machine learning model.
  • Figure 1 is a sample and feature view of longitudinal federated learning with 3 participants.
  • the datasets from participant A, participant B and participant C are aggregated into a large virtual data through sample alignment set, then Participant A, Participant B, and Participant C each own a part of the vertical division of the virtual data set, where the aligned samples are overlapping samples of Participant A, Participant B, and Participant C, that is, Participant A, Participant B, and Participant C each have different characteristics of overlapping samples.
  • the aligned samples are overlapping samples of Participant A, Participant B, and Participant C, that is, Participant A, Participant B, and Participant C each have different characteristics of overlapping samples.
  • the overlapping samples are insufficient, it is difficult for vertical federated learning to establish a machine learning model with good performance, which limits
  • Federated transfer learning uses sufficient data resources in the source domain to build a good predictive model for the target domain, but the model established by federated transfer learning can only predict the data in the target domain, but not the data of all participants.
  • Make label predictions that is, models built through federated transfer learning are not applicable to all parties involved.
  • Semi-supervised learning can improve model performance by complementing missing features (or feature representations) and labels.
  • semi-supervised learning under federated learning usually requires complex interactions between participants, which leads to the protection of data privacy. There are hidden dangers that cannot effectively protect data privacy.
  • the main purpose of this application is to provide a vertical federated modeling method, apparatus, device, and computer-readable storage medium, aiming to solve the technical problem that it is difficult to achieve a balance between data privacy protection and model generality in the existing federated learning.
  • the present application provides a vertical federated modeling method, and the vertical federated modeling method includes the following steps:
  • the first participant determines the multi-party overlapping samples between the first participant and each second participant, and obtains the multi-party federation calculation intermediate parameters corresponding to each second participant based on the multi-party overlapping samples;
  • the first participant determines the double-overlap samples between each second party and the first party, and based on the double-overlap samples, respectively obtains the intermediate parameters of the two-party federation calculation corresponding to each second party;
  • the first participant determines the unilateral gradient, the multi-party federated calculation intermediate gradient, and each of the two-party federated calculation intermediate gradients based on the local sample of the first participant, the multi-party federation calculation intermediate parameters, and each of the two-party federation calculation intermediate parameters, based on the unilateral gradient , the multi-party federation calculates the intermediate gradient, and each two-party federation calculates the intermediate gradient and performs the model update operation.
  • the present application also provides a vertical federated modeling method, the vertical federated modeling method includes the following steps:
  • the intermediate parameters of the multi-party federation calculation are obtained by joint calculation, wherein the second participant includes a plurality of participants;
  • the intermediate parameters of the federation calculation between the two parties are calculated respectively;
  • the second participant performs a model update operation based on the received target gradients fed back by the first participant based on the multi-party federated calculation intermediate parameters and each of the two-party federated calculation intermediate parameters.
  • the present application also provides a vertical federation modeling device, the vertical federation modeling device includes:
  • the first determination module is used to determine the multi-party overlapping samples between the first participant and each second participant, and obtain the multi-party federation calculation intermediate parameters corresponding to each second participant based on the multi-party overlapping samples;
  • the second determination module is configured to determine the double-overlap samples between each second participant and the first party, and obtain the two-party federation calculation intermediate parameters corresponding to each second party based on the double-overlap samples;
  • the first model update module is used for determining the unilateral gradient, the intermediate gradient of the multi-party federation, and the intermediate gradient of each federation of The unilateral gradient, the multi-party federation calculates the intermediate gradient, and each two-party federation calculates the intermediate gradient and performs a model update operation.
  • the present application also provides a vertical federation modeling device, the vertical federation modeling device includes:
  • a first calculation module configured to jointly calculate the multi-party federated computing intermediate parameters based on the multi-party overlapping samples with the first participant, wherein the second participant includes a plurality of participants;
  • the second calculation module is used to calculate the intermediate parameters of the federation calculation of the two parties respectively based on the overlapping samples of the two parties with the first participant;
  • the second model update module is configured to perform a model update operation based on the received target gradients fed back by the first participant based on the multi-party federated calculation intermediate parameters and each of the two-party federated calculation intermediate parameters.
  • the present application also provides a vertical federation modeling device
  • the vertical federation modeling device includes: a memory, a processor, and a vertical federation model stored on the memory and executable on the processor.
  • a federated modeling program that, when executed by the processor, implements the steps of the aforementioned vertical federated modeling method.
  • the present application also provides a computer-readable storage medium, where a vertical federated modeling program is stored on the computer-readable storage medium, and the vertical federated modeling program is executed by a processor to realize the foregoing Steps of a longitudinal federation modeling approach.
  • the first participant determines the multi-party overlapping samples between the first participant and each second participant, and obtains the multi-party federation calculation intermediate parameters corresponding to each second participant based on the multi-party overlapping samples; then the first participant determines The two-party overlapping samples between each second party and the first party, and the two-party federation calculation intermediate parameters corresponding to each second party are obtained based on the overlapping samples of the two parties; then the first party is based on the first party's local samples.
  • Multi-party federation calculation intermediate parameters and each of the two-party federation calculation intermediate parameters determine the unilateral gradient, the multi-party federation calculation intermediate gradient, and each two-party federation calculation intermediate gradient, based on the unilateral gradient, the multi-party federation calculation intermediate gradient and each two-party federation calculation intermediate gradients and perform model update operations. Make each participant including the first participant and the second participant complete the model update to obtain the target model. The first participant and each second participant can use their own target model to perform label prediction on their local samples.
  • One participant and each second participant can also use their own target models to perform label prediction on overlapping samples between each participant, so that all participants can complete the model training, so that the model training process can be applied to all participants; Increase the training process of overlapping samples of both parties, so that when the number of overlapping samples shared by each participant is small, a model with excellent performance can be trained.
  • Interaction compared with traditional semi-supervised learning, does not require completion of missing features and labels, which simplifies the interaction process during model training, completely prevents information leakage, and improves data security in federated learning.
  • Figure 1 is a sample and feature view of vertical federated learning with 3 participants
  • FIG. 2 is a schematic structural diagram of a vertical federation modeling device in the hardware operating environment involved in the solution of the embodiment of the present application;
  • FIG. 3 is a schematic flowchart of the first embodiment of the vertical federation modeling method of the present application
  • FIG. 4 is a schematic flowchart of another embodiment of the vertical federation modeling method of the present application.
  • FIG. 5 is a schematic diagram of functional modules of an embodiment of the vertical federation modeling apparatus of the present application.
  • FIG. 2 is a schematic structural diagram of a vertical federation modeling device in a hardware operating environment involved in an embodiment of the present application.
  • the vertical federation modeling device in this embodiment of the present application may be a PC, or a smartphone, a tablet computer, an e-book reader, or an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3) Player, MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Player, portable computer and other portable terminal equipment with display function.
  • MP3 Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3
  • MP4 Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4
  • the vertical federation modeling device may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 .
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface and a wireless interface (eg, a WI-FI interface).
  • the memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .
  • the vertical federation modeling device may also include cameras, RF (Radio Frequency, radio frequency) circuits, sensors, audio circuits, WiFi modules, etc.
  • RF Radio Frequency, radio frequency
  • sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors can also be configured in the longitudinal federation modeling device, which will not be described here.
  • the vertical federated modeling device structure shown in FIG. 2 does not constitute a limitation to the vertical federated modeling device, and may include more or less components than those shown in the figure, or combine some components, Or a different component arrangement.
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a vertical federation modeling program.
  • the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client (client) and perform data communication with the client ; and the processor 1001 can be used to call the vertical federation modeling program stored in the memory 1005 .
  • the vertical federation modeling device includes: a memory 1005, a processor 1001, and a vertical federation modeling program stored on the memory 1005 and executable on the processor 1001, wherein the processor 1001 calls
  • the steps of the vertical federation modeling method in the following embodiments are executed.
  • FIG. 3 is a schematic flowchart of the first embodiment of the vertical federated modeling method of the present application.
  • This vertical federation modeling approach should include:
  • Step S100 the first participant determines a multi-party overlapping sample between the first participant and each second participant, and obtains the multi-party federation calculation intermediate parameter corresponding to each second participant based on the multi-party overlapping sample;
  • the second participant includes multiple other participants except the first participant in the federated learning, that is to say, there are multiple second participants, and each second participant stores samples.
  • the first participant stores samples and sample labels.
  • Participant A, Participant B, and Participant C have a certain number of overlapping samples.
  • the first participant aligns its local samples with the samples of each second participant to obtain multiple overlapping samples of the first participant, and each second participant aligns their local samples with the first participant's samples.
  • the local samples of each participant and the local samples of other participants in the second party are aligned to obtain the multi-party overlapping samples of each second party, that is, the multi-party overlapping samples are the local samples of each participant and other participants. It is obtained that the first participant is a participant with sample labels of multiple overlapping samples.
  • each participant performs sample alignment according to the user identification (ID) of the local sample.
  • ID user identification
  • the first participant determines the multi-party sample label corresponding to the multi-party overlapping sample according to the sample label corresponding to the local sample, and determines the multi-party intermediate parameter according to the multi-party sample label. Intermediate parameters. Then, the multi-party intermediate parameters are sent to the second party to avoid the transmission of sample labels during the federation process and the leakage of sample labels.
  • each of the second participants jointly calculates the intermediate parameters of the multi-party federation calculation based on the multi-party overlapping samples, and sends the multi-party federated calculation intermediate parameters to the first participant.
  • each second participant is based on the multi-party overlapping samples. , respectively input the local sample features in the respective multi-party overlapping samples into the first feature extraction model of the respective multi-party federation classification model to obtain the second feature representation; each second participant respectively inputs the second feature representation into the respective multi-party federation
  • the intermediate parameters are calculated.
  • the second feature representation and the multi-party intermediate parameters are input into the first classification model of the respective multi-party federation classification model, and the intermediate parameters are calculated, and then the multi-party federation is calculated according to the intermediate parameters. Calculate intermediate parameters, and feed back the multi-party federated calculation intermediate parameters to the first participant.
  • Step S200 the first participant determines the double-overlap samples between each second party and the first party, and obtains the two-party federation calculation intermediate parameters corresponding to each second party based on the double-overlap samples;
  • the first participant determines the overlapping samples between each second participant and the first participant. Specifically, each second participant aligns its samples with the local samples of the first participant respectively. , and obtain the double-overlap samples of each second participant. After determining the double-overlap samples, each second party feeds back the identification information corresponding to each double-overlap sample to the first party. The first party is based on the identification information and the local The sample label of the sample, to determine the two-sided sample label corresponding to each two-sided overlapping sample, and to determine the two-sided intermediate parameter according to the two-sided sample label, wherein the identification information of the sample corresponding to the two-sidedly overlapping sample in the local sample of the first participant is the same as the two-sided overlapping sample. The corresponding identification information is the same.
  • each second participant calculates the intermediate parameters of the two-party federation calculation based on the overlapping samples of the two parties, and sends the two-party federal calculation intermediate parameters to the first party.
  • the local sample features are input into the second feature extraction model of the respective federated classification models of the two parties to obtain the third feature representation; each second participant respectively inputs the third feature representation into the first classification model of the respective federated classification models of the two parties, and calculates Obtain the intermediate parameters of the two-party federation calculation, and feed back the two-party federation calculation intermediate parameters to the first participant.
  • each second participant respectively inputs the third feature representation and the two-party intermediate parameters into their respective two-party federations
  • the first classification model of the classification model obtains the intermediate parameters of the two-way federation calculation.
  • Step S300 the first participant determines the unilateral gradient, the multi-party federated calculation intermediate gradient, and each of the two-party federated calculation intermediate gradients based on the local sample of the first participant, the multi-party federation calculation intermediate parameters, and each of the two-party federation calculation intermediate parameters, based on all the two-party federation calculation intermediate gradients.
  • the single-party gradient, the multi-party federation calculates the intermediate gradient, and each two-party federation calculates the intermediate gradient and performs the model update operation.
  • the first participant determines the unilateral The gradient, the multi-party federated calculation intermediate gradient, and each of the two-party federated calculation intermediate gradients are based on the unilateral gradient, the multi-party federated calculation of the intermediate gradient, and each of the two-party federated calculation of the intermediate gradient and the model update operation. Specifically, the first participant is based on the multi-party calculation of the intermediate gradient.
  • the federated calculation intermediate parameter determines the multi-party calculation loss value, determines each two-party calculation loss value based on each of the two-party federated calculation intermediate parameters, and determines the multi-party federated calculation intermediate gradient and each two-party calculation loss value based on each of the two-party calculation loss values and the multi-party calculation loss value.
  • the federation calculates the intermediate gradient, calculates the intermediate gradient based on the multi-party federation, and calculates the intermediate gradient for each two-party federation and performs the model update operation, that is, the model update operation is performed according to the multi-party federation calculation of the intermediate gradient and the intermediate gradient to obtain the target model.
  • the first participant Each party feeds back the intermediate gradients calculated by the multi-party federation, the intermediate gradients, and the intermediate gradients calculated by the two-party federation to each second participant.
  • the first participant and each second participant can use their own target models to perform label prediction on their local samples.
  • the participants and each second participant can also use their own target models to perform label prediction on the overlapping samples among the participants.
  • the vertical federation modeling method proposed in this embodiment uses the first participant to determine the multi-party overlapping samples between the first participant and each second participant, and obtains the multi-party federation calculation corresponding to each second participant based on the multi-party overlapping samples Intermediate parameters; then the first participant determines the double-overlap samples between each second party and the first party, and obtains the two-party federation calculation intermediate parameters corresponding to each second party based on the overlapping samples of the two parties; then the first party Based on the local samples of the first participant, the intermediate parameters of the multi-party federation calculation, and the intermediate parameters of each of the two-party federation calculations, the party determines the unilateral gradient, the multi-party federation calculation intermediate gradient, and the intermediate gradients of each two-party federation calculation.
  • Calculate the intermediate gradient and each two-party federation calculates the intermediate gradient and performs the model update operation. Make each participant including the first participant and the second participant complete the model update to obtain the target model.
  • the first participant and each second participant can use their own target model to perform label prediction on their local samples.
  • One participant and each second participant can also use their own target models to perform label prediction on overlapping samples between each participant, so that all participants can complete the model training, so that the model training process can be applied to all participants; Increase the training process of overlapping samples of both parties, so that when the number of overlapping samples shared by each participant is small, a model with excellent performance can be trained.
  • Interaction compared with traditional semi-supervised learning, does not require completion of missing features and labels, which simplifies the interaction process during model training, completely prevents information leakage, and improves data security in federated learning.
  • step S300 includes:
  • Step S310 the first participant determines the unilateral loss value based on the local sample of the first participant, determines the multi-party calculation loss value based on the local sample of the first participant and the multi-party federated computing intermediate parameter, and determines the loss value based on the local sample of the first participant.
  • the sample and each of the two-party federated calculation intermediate parameters determine the loss value of each of the two-party calculations;
  • Step S320 the first participant determines the unilateral gradient, the multi-party federated calculation intermediate gradient, and each two-party federated calculation intermediate gradient based on the unilateral loss value, the two-party calculated loss value, and the multi-party calculated loss value, respectively, and based on the The single-party gradient, the multi-party federation calculates the intermediate gradient, and each two-party federation calculates the intermediate gradient and performs the model update operation.
  • the first participant determines the unilateral loss value based on the local sample, and determines the multi-party loss value based on the local sample of the first participant and the multi-party federation calculation intermediate parameter. Calculate the loss value; and determine the loss value of each two-party calculation based on the local sample of the first participant and each of the two-party federal calculation intermediate parameters, so as to accurately obtain the multi-party calculation loss value and the two-party calculation intermediate parameter according to the two-party federal calculation intermediate parameter and the multi-party federal calculation intermediate parameter. Calculate the loss value.
  • the unilateral gradient calculates the intermediate gradient and each two-party federation calculates the intermediate gradient.
  • the unilateral gradient is determined based on the unilateral loss value
  • the multi-party federation calculates the intermediate gradient based on the multi-party calculation loss value
  • each two-party federation calculates the intermediate gradient based on the two-party calculation loss value
  • perform a model update operation that is, according to the unilateral gradient, the multi-party federated calculation of the intermediate gradient, and the intermediate gradient. Perform the model update operation to obtain target model.
  • the first participant determines the unilateral loss value based on the first participant's local sample
  • the multi-party calculation loss value is determined based on the first participant's local sample and the multi-party federation calculation intermediate parameter.
  • the first party is based on the unilateral loss value, the calculation loss value of each party and the multi-party calculation loss value respectively.
  • step S310 includes:
  • Step S311 the first participant inputs the multi-party overlapping samples in the local samples of the first participant into the feature extraction model of the first participant to obtain the feature representation of the multi-party overlapping samples, and the feature representation of the multi-party overlapping samples, the sample
  • the label and the intermediate parameters of the multi-party federation calculation are input into the first multi-party classification model of the multi-party federation model to calculate the multi-party calculation loss value.
  • the first participant after obtaining the intermediate parameters of the two-party federation calculation and the intermediate parameters of the multi-party federation calculation, the first participant inputs the multi-party overlapping samples in the local samples of the first participant into the feature extraction model of the first participant, so as to obtain the multi-party overlapping samples.
  • the feature representation of the overlapping samples specifically, the first participant obtains the multi-party overlapping sample features of the multi-party overlapping samples in its local samples, and inputs the multi-party overlapping sample features into the feature extraction model of the first participant to obtain the multi-party overlapping samples characteristic representation.
  • the first participant inputs the feature representation of the multi-party overlapping samples, the sample labels and the intermediate parameters of the multi-party federation calculation into the first multi-party classification model of the multi-party federation model. Specifically, the first participant obtains the multi-party overlap in the sample labels. For the sample label corresponding to the sample, the feature representation of the multi-party overlapping sample, the sample label corresponding to the multi-party overlapping sample, and the multi-party federation calculation intermediate parameter are input into the first multi-party classification model of the multi-party federation model, and the multi-party calculation loss value is obtained.
  • step S300 further includes:
  • Step S322 the first participant inputs each of the two-sided overlapping samples in the first participant's local samples into the feature extraction model of the first participant, so as to obtain the feature representation of each two-sided overlapping sample, and the feature of each two-sided overlapping sample is obtained.
  • the first two-party classification model of each two-party federation model corresponding to each input of the representation, the sample label, and each of the two-party federation calculation intermediate parameters is input to obtain the respective two-party calculation loss values.
  • the first participant after obtaining the intermediate parameters of the two-party federation calculation and the intermediate parameters of the multi-party federation calculation, the first participant inputs each of the two-party overlapping samples in the local samples of the first participant into the feature extraction model of the first participant to obtain The feature representation of each double-overlapped sample, the first participant obtains the double-overlapped sample features of each double-overlapped sample in its local sample, and inputs the features of each double-overlapped sample into the feature extraction model of the first party to obtain the respective double-overlapped sample features.
  • Feature representation of overlapping samples are examples of the first participant's overlapped sample features of each double-overlapped sample.
  • the first participant inputs the feature representation, sample label and each of the two-party federation calculation intermediate parameters of each of the two-party overlapping samples into the first two-party classification model of each corresponding two-party federation model.
  • the first participant is in the The sample label corresponding to each double-overlapping sample is obtained from the sample label, and the feature representation of each double-overlapping sample, the sample label corresponding to each double-overlapping sample, and the multi-party federation calculation intermediate parameter are input into the first two-party classification model of the two-party federation model, respectively, Get both sides to calculate the loss value.
  • step S300 further includes:
  • Step S333 the first participant inputs the local sample of the first participant into the feature extraction model of the first participant to obtain the first feature representation, and inputs the first feature representation and the sample label into the unilateral classification model to calculate the result.
  • the one-sided loss value is stated.
  • the first participant after obtaining the intermediate parameters of the two-party federation calculation and the intermediate parameters of the multi-party federation calculation, the first participant inputs the local samples of the first party into the feature extraction model of the first party for model training to obtain the first feature Then, the first feature representation and the sample label are input into the unilateral classification model for model training, and the unilateral loss value is obtained.
  • the first participant inputs the multi-party overlapping samples in the local samples of the first participant into the feature extraction model of the first participant, so as to obtain the feature representation of the multi-party overlapping samples.
  • the feature representation of the multi-party overlapping samples, the sample label and the multi-party federation calculation intermediate parameters are input into the first multi-party classification model of the multi-party federation model to calculate the multi-party calculation loss value, and the multi-party calculation loss can be accurately obtained according to the multi-party federation calculation intermediate parameters. value, which improves the accuracy of multi-party calculation of loss value, thereby improving the efficiency and accuracy of federated learning.
  • step S320 includes:
  • Step S410 the first participant calculates the unilateral gradient based on the unilateral loss value, and updates the unilateral classification model of the first participant based on the unilateral gradient;
  • Step S420 the first participant calculates the multi-party federation calculation intermediate gradient based on the multi-party calculation loss value; updates the first multi-party classification model of the multi-party federation classification model based on the multi-party federation calculation intermediate gradient;
  • Step S430 the first participant calculates the intermediate gradient of each two-party federation calculation based on the respective two-party calculation loss values; calculates the intermediate gradient based on each two-party federation, and correspondingly updates the first two-party classification model of each two-party federation classification model;
  • Step S430 the first participant updates the feature extraction model of the first participant based on the unilateral gradient, the multi-party federated calculation of the intermediate gradient, and each of the two-way federated calculation of the intermediate gradient.
  • the first participant calculates the unilateral gradient based on the unilateral loss value.
  • Gradient update the unilateral classification model, that is, update the model parameters of the unilateral classification model according to the unilateral gradient.
  • the first participant calculates the intermediate gradient of the multi-party federation calculation based on the multi-party calculation loss value, and updates the first multi-party classification model of the multi-party federation classification model based on the multi-party federation calculation intermediate gradient, that is, updates the intermediate gradient according to the multi-party federation calculation
  • the intermediate gradient of the multi-party federation calculation is fed back to each second participant.
  • each second participant After receiving the intermediate gradient of the multi-party federation calculation, each second participant updates the second multi-party classification model of the respective multi-party federation classification model, that is, each second The participating parties update the model parameters of the second multi-party classification model of their own multi-party federation classification model according to the multi-party federation calculation intermediate gradient.
  • the first participant calculates the intermediate gradients of the federations of each of the two parties based on the calculated loss values of the two parties. For example, a weighted average algorithm is used to calculate the intermediate gradients of the federations of the two parties according to the calculated loss values of the two parties. Gradient, corresponding to update the first two-party classification model of each two-party federated classification model; at the same time, feed back the intermediate gradient of each two-way federation calculation to each corresponding second participant, so that each second participant can update the first two-party federal classification model. Two-sided classification model.
  • the first participant updates the feature extraction model of the first participant based on the unilateral gradient, the multi-party federated calculation intermediate gradient, and each of the two-party federated calculation intermediate gradients.
  • the gradients are fused to obtain the first target gradient, and the model parameters of the feature extraction model of the first participant are updated according to the first target gradient.
  • Each second participant updates its feature extraction model based on the multi-party federation calculation intermediate gradient and the two-party federation calculation intermediate gradient respectively.
  • Two target gradients each second participant updates the model parameters of its own feature extraction model according to the second target gradient.
  • the unilateral gradient is calculated by the first participant based on the unilateral loss value, and the unilateral classification model of the first participant is updated based on the unilateral gradient;
  • the multi-party calculation loss value is calculated to obtain the multi-party federation calculation intermediate gradient; based on the multi-party federation calculation intermediate gradient, the first multi-party classification model of the multi-party federation classification model is updated; and then the first participant calculates each two-party federation based on the respective two-party calculation loss values.
  • Calculate the intermediate gradient calculate the intermediate gradient based on each two-party federation, correspondingly update the first two-party classification model of each two-party federation classification model; then calculate the intermediate gradient based on the unilateral gradient, the multi-party federation calculation and the intermediate gradient of each two-party federation, update the first participant's
  • the feature extraction model can update the two-party federated classification model and the multi-party federated classification model of each participant according to the single-party loss value, the multi-party calculation loss value, and the two-party calculation loss value, so as to accurately update the model and obtain a trained model.
  • the efficiency and accuracy of federated learning can update the two-party federated classification model and the multi-party federated classification model of each participant according to the single-party loss value, the multi-party calculation loss value, and the two-party calculation loss value, so as to accurately update the model and obtain a trained model.
  • step S320 a fifth embodiment of the vertical federation modeling method of the present application is proposed.
  • the method further includes:
  • Step S510 the first participant calculates the total loss value based on the single-party loss value, the respective two-party loss value, and the multi-party calculation loss value;
  • Step S520 if the total loss value is less than the preset threshold, the first participant uses the model obtained after the model update operation as the target model;
  • Step S530 if the total loss value is greater than or equal to the preset threshold, the first participant continues to determine the multi-party overlap between the first participant and each second participant based on the model obtained after the model update operation. Sample steps.
  • the first participant calculates the total loss value based on the one-party loss value, the two-party loss value, and the multi-party calculation loss value.
  • the one-party loss value can be obtained.
  • the first participant determines whether the total loss value is less than the preset threshold value, and if it is less than the predetermined threshold, the first participant uses the model obtained after the model update operation as the target model. Otherwise, based on the model obtained after the model update operation, the first participant continues to perform the step of determining the multi-party overlapping samples between the first participant and each second participant by the first participant.
  • the updated third The feature extraction model is used as the third feature extraction model of the first participant
  • the updated third classification model is used as the third classification model of the first participant
  • the updated multi-party federated classification model of the first participant is used as the first participant
  • the updated multi-party federal classification model of the first party is used as the dual-federal classification model of the first party
  • the updated multi-party federal classification model of each second party is used as the multi-party federal classification model of each second party.
  • Model making, after the update, the two-party federated classification model of each second participant is used as the two-way federated classification model of each second participant, and step S100 is continued until the new total loss value is less than the preset threshold.
  • the first participant calculates the total loss value based on the unilateral loss value, the loss value of each two parties, and the multi-party calculation loss value; then, if the total loss value is less than a preset value threshold value, the first participant will use the model obtained after the model update operation as the target model; then if the total loss value is greater than or equal to the preset threshold, the first participant will continue to perform the determination of the first party based on the model obtained after the model update operation.
  • the step of multiple overlapping samples between the participant and each second participant further improves the efficiency of federated learning by ensuring that the total loss value converges to obtain a model that meets the requirements.
  • FIG. 4 is a schematic flowchart of another embodiment of the vertical federated modeling method of the present application.
  • This vertical federation modeling approach should include:
  • Step 610 the second participant performs joint calculation based on the multi-party overlapping samples with the first participant to obtain the intermediate parameters of the multi-party federation calculation, wherein the second participant includes a plurality of participants;
  • the second participant aligns their respective local samples with the local samples of the first participant and the local samples of other participants in the second participant to obtain multiple overlapping samples of each second participant, That is, the multi-party overlapping samples are obtained by aligning the local samples of each participant with the samples of other participants.
  • Each multi-party overlapping sample has different sample characteristics, and each multi-party overlapping sample has the same ID.
  • the second party jointly calculates the intermediate parameters of the multi-party federation calculation. Specifically, each second participant calculates the local samples in the respective multi-party overlapping samples based on the multi-party overlapping samples.
  • each second participant respectively inputs the second feature representation into the first classification model of the respective multi-party federated classification model, and calculates to obtain Intermediate parameters, for example, input the second feature representation and the multi-party intermediate parameters into the first classification model of the respective multi-party federation classification model, calculate the intermediate parameters, and then calculate the multi-party federation according to the intermediate parameters to calculate the intermediate parameters, and feed back the multi-party federation Calculate intermediate parameters to the first party.
  • Step 620 the second participant calculates and obtains the intermediate parameters of the federation calculation between the two parties respectively based on the overlapping samples between the second party and the first participant;
  • each second participant aligns its samples with the local samples of the first participant, respectively, to obtain two-sided overlapping samples of each second participant. After each second participant determines the two-sided overlapping samples, respectively The identification information corresponding to each overlapping sample of both sides is fed back to the first participant.
  • the second participant calculates the intermediate parameters of the two-way federation calculation based on the overlapping samples of the two parties with the first participant. Specifically, each second participant inputs the local sample features in the overlapping samples of the two parties into the respective federal classifications of the two parties.
  • each second participant respectively inputs the third feature representation into the first classification model of the respective two-way federation classification model, and calculates the intermediate parameters of the two-way federation calculation, and Feeding back the intermediate parameters of the two-party federation calculation to the first participant, specifically, each second participant respectively inputs the third feature representation and the two-party intermediate parameters into the first classification model of the respective two-party federation classification models, Obtain the intermediate parameters of the federation calculation of both parties.
  • Step 630 the second participant performs a model update operation based on the received target gradients fed back by the first participant based on the multi-party federated calculation intermediate parameters and each of the two federated federated calculation intermediate parameters.
  • the target gradient includes the target gradient including the intermediate gradient of the multi-party federation calculation and the intermediate gradient of the two-party federation calculation.
  • the second participant updates its own multi-party federation classification model based on the multi-party federation calculation of the intermediate gradient and the two-party federation calculation of the intermediate gradient.
  • the model, the two-way federation classification model, and the features Extract the model.
  • step S610 includes:
  • Step 611 the second participant inputs the local sample features in the respective multi-party overlapping samples into their respective feature extraction models to obtain the second feature representation
  • Step 612 the second participant inputs the second feature representation into the second multi-party classification model of the respective multi-party federation classification model, calculates the intermediate parameters, and jointly calculates the multi-party federation calculation based on the intermediate parameters of each second participant.
  • Intermediate parameters the second participant inputs the second feature representation into the second multi-party classification model of the respective multi-party federation classification model, calculates the intermediate parameters, and jointly calculates the multi-party federation calculation based on the intermediate parameters of each second participant.
  • each second participant inputs the local sample features in the respective multi-party overlapping samples into their respective feature extraction models to obtain the second feature representation; then, each second participant respectively inputs the second feature representation into their respective feature extraction models
  • the second multi-party classification model of the multi-party federated classification model is calculated to obtain intermediate parameters.
  • the second feature representation and the multi-party intermediate parameters are input into the second multi-party classification model of the respective multi-party federated classification model, and the intermediate parameters are calculated; then based on The intermediate parameters of each second participant are jointly calculated to obtain the intermediate parameters of the multi-party federation calculation, and the intermediate parameters of the multi-party federation calculation are fed back to the first participant.
  • each second participant sends the intermediate parameters to the coordinator for coordination.
  • the party obtains the intermediate parameters of the multi-party federation calculation by joint calculation according to the respective intermediate parameters, and feeds back the intermediate parameters of the multi-party federation calculation to the first participant.
  • step S620 includes:
  • Step 621 the second participant inputs the local sample features in the overlapping samples of both parties into their respective feature extraction models to obtain a third feature representation
  • Step 622 the second participant inputs the third feature representation into the second bilateral classification model of the respective bilateral federation classification models, and calculates the intermediate parameters of the bilateral federation calculation.
  • each second participant calculates the intermediate parameters of the two-party federation calculation based on the overlapping samples of the two parties, and sends the two-party federated calculation intermediate parameters to the first party.
  • the parameters are sent to the corresponding second participants, and each second participant inputs the local sample features in the overlapping samples of both parties into their respective feature extraction models to obtain third feature representations; each second participant respectively uses the third feature Characterize the second two-party classification model inputting the respective two-party federation classification models, calculate and obtain the intermediate parameters of the two-party federation calculation, and feed back the two-party federation calculation intermediate parameters to the first participant, specifically, each second participant respectively
  • the third feature representation and the intermediate parameters of the two parties are input into the second two-party classification model of the respective two-way federation classification models, and the intermediate parameters of the two-way federation calculation are obtained.
  • the target gradient includes a multi-party federated calculation intermediate gradient and a two-party federated calculation intermediate gradient
  • step S630 includes:
  • Step 631 the second participant updates the second multi-party classification model of the multi-party federation classification model based on the multi-party federation calculation intermediate gradient;
  • Step 632 the second participant updates the second two-party classification model of the two-way federation classification model based on the intermediate gradient of the two-way federation calculation;
  • Step 633 the second participant updates the feature extraction model based on the multi-party federated calculation of the intermediate gradient and the two-way federated calculation of the intermediate gradient.
  • each second participant after receiving the intermediate gradient of the multi-party federation calculation and the intermediate gradient of the two-party federation calculation, each second participant updates the second multi-party classification model of the respective multi-party federation classification model, that is, each second participant is based on the multi-party classification model.
  • the federated computing intermediate gradient updates the model parameters of the second multi-party classification model of the own multi-party federated classification model, and updates the second multi-party classification model of the multi-party federated classification model based on the multi-party federated computing intermediate gradient.
  • each second participant updates its feature extraction model based on the multi-party federation calculation intermediate gradient and the two-party federation calculation intermediate gradient respectively.
  • the second target gradient each second participant updates the model parameters of its own feature extraction model according to the second target gradient.
  • the multi-party federation calculation intermediate parameter is obtained by joint calculation based on the multi-party overlapping samples between the second participant and the first participant, wherein the second participant includes multiple participants; then the second participant is based on The two-party overlapping samples between the first participant and the first participant are calculated respectively to obtain the intermediate parameters of the two-party federation calculation; then the second participant is based on the received first party.
  • the target gradient of and perform the model update operation, so that the second participant completes the model update to obtain the target model, and each second participant can use its own target model to perform label prediction on its local samples.
  • the first participant and each second participant The party can also use its own target model to perform label prediction on the overlapping samples between each participant, so that all participants can complete the model training, so that the model training process can be applied to all participants; by adding the training process of overlapping samples of both parties,
  • each participant conducts data exchange through multi-party federated computing intermediate parameters and two-party federated computing intermediate parameters, and traditional semi-supervised learning.
  • there is no need to complete missing features and labels which simplifies the interaction process during model training, completely prevents information leakage, and improves data security in federated learning.
  • the embodiment of the present application further provides a vertical federation modeling device, referring to FIG. 5 , the vertical federation modeling device includes:
  • the first determination module 100 is configured to determine the multi-party overlapping samples between the first participant and each second participant, and obtain the multi-party federation calculation intermediate parameters corresponding to each second participant based on the multi-party overlapping samples;
  • the second determination module 200 is configured to determine the double-overlap samples between each second participant and the first party, and based on the double-overlap samples, respectively obtain the intermediate parameters of the two-way federation calculation corresponding to each second party;
  • the model update module 300 is configured to determine the unilateral gradient, the intermediate gradient of the multi-party federation, and the intermediate gradient of each federation of The single-party gradient, the multi-party federation calculates the intermediate gradient, and each two-party federation calculates the intermediate gradient and performs the model update operation.
  • model update module 300 is also used for:
  • the unilateral loss value is determined based on the local sample of the first participant, the multi-party calculation loss value is determined based on the local sample of the first participant and the multi-party federation calculation intermediate parameter, and the multi-party calculation loss value is determined based on the local sample of the first participant and each of the two-party federations. Calculate the intermediate parameters to determine the calculated loss value of each party;
  • the two-party calculated loss value, and the multi-party calculated loss value determine the unilateral gradient, the multi-party federated calculation intermediate gradient, and each of the two-party federated calculation intermediate gradients, and based on the unilateral gradient and the multi-party federated calculation
  • the intermediate gradient and each two-party federation computes the intermediate gradient and performs model update operations.
  • model update module 300 is also used for:
  • the multi-party overlapping samples in the local samples of the first participant are input into the feature extraction model of the first participant to obtain the feature representation of the multi-party overlapping samples, the feature representation of the multi-party overlapping samples, the sample labels and the multi-party federation calculation
  • the intermediate parameter is input into the first multi-party classification model of the multi-party federated model to obtain the multi-party calculation loss value.
  • model update module 300 is also used for:
  • model update module 300 is also used for:
  • model update module 300 is also used for:
  • the multi-party federation calculation intermediate gradient is calculated based on the multi-party calculation loss value; based on the multi-party federation calculation intermediate gradient, the first multi-party classification model of the multi-party federation classification model is updated;
  • the feature extraction model of the first participant is updated based on the unilateral gradient, the multi-party federated calculation intermediate gradient, and each two-party federated calculation intermediate gradient.
  • the embodiment of the present application also provides a vertical federation modeling device, and the vertical federation modeling device includes:
  • a vertical federation modeling device wherein the vertical federated modeling device includes:
  • a first calculation module configured to jointly calculate the multi-party federated computing intermediate parameters based on the multi-party overlapping samples with the first participant, wherein the second participant includes a plurality of participants;
  • the second calculation module is used to calculate the intermediate parameters of the federation calculation of the two parties respectively based on the overlapping samples of the two parties with the first participant;
  • the second model update module is configured to perform a model update operation based on the received target gradients fed back by the first participant based on the multi-party federated calculation intermediate parameters and each of the two-party federated calculation intermediate parameters.
  • the first computing module is also used for:
  • the second feature representation is input into the second multi-party classification model of the respective multi-party federation classification model, the intermediate parameters are calculated, and the multi-party federation calculation intermediate parameters are jointly calculated based on the intermediate parameters of each second participant.
  • the second computing module is also used for:
  • the third feature representation is input into the second bilateral classification model of the respective bilateral federation classification models, and the intermediate parameters of the bilateral federation calculation are obtained by calculation.
  • the second model update module is also used for:
  • a second multi-party classification model based on the multi-party federation calculation of the intermediate gradient to update the multi-party federated classification model
  • the feature extraction model is updated based on the multi-party federation computing the intermediate gradient and the two-party federation computing the intermediate gradient.
  • an embodiment of the present application further proposes a computer-readable storage medium, where a vertical federation modeling program is stored on the computer-readable storage medium, and when the vertical federated modeling program is executed by a processor, the vertical federation modeling program as described above is implemented. Steps of a federated modeling approach.
  • the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation.
  • the technical solutions of the present application can be embodied in the form of software products in essence or the parts that make contributions to the prior art.
  • the computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请公开了一种纵向联邦建模方法，包括以下步骤：第一参与方确定多方重叠样本，并获取多方联邦计算中间参数；第一参与方确定双方重叠样本，并获取双方联邦计算中间参数；第一参与方基于本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。

Description

纵向联邦建模方法、装置、设备及计算机可读存储介质
本申请要求2020年9月7日申请的，申请号为202010932992.7，名称为“纵向联邦建模方法、装置、设备及计算机可读存储介质”的中国专利申请的优先权，在此将其全文引入作为参考。
技术领域
本申请涉及联邦学习技术领域，尤其涉及一种纵向联邦建模方法、装置、设备及计算机可读存储介质。
 背景技术
联邦学习是为了在保护数据隐私的情况下利用多个参与方的数据建立的机器学习模型，其中，纵向联邦学习利用多个参与方所拥有的重叠样本的不同特征来建立机器学习模型。参照图1，图1为具有3个参与方的纵向联邦学习的样本和特征视图，将来自参与方A、参与方B和参与方C的数据集通过样本对齐的方式聚合成一个大的虚拟数据集，则参与方A、参与方B和参与方C各自拥有该虚拟数据集纵向划分的一部分，其中，对齐的样本为参与方A、参与方B和参与方C的重叠样本，也就是说，参与方A、参与方B和参与方C分别拥有重叠样本的不同特征。然而，当重叠样本不够充分时纵向联邦学习很难建立起性能良好的机器学习模型，这在一定程度上限制了纵向联邦学习应用于更广泛的场景。
目前，往往通过联邦迁移学习或者半监督学习解决当重叠样本不够充分的问题。联邦迁移学习利用源领域中所拥有的充足数据资源为目标领域建立一个表现良好的预测模型，但通过联邦迁移学习建立的模型只能对目标领域的数据进行预测，而不能对所有参与方的数据进行标签预测，也就是说，通过联邦迁移学习建立的模型并不能适用于所有参与方。半监督学习可以通过对缺失特征（或特征表征）和标签的补全来提高模型性能，然而，在联邦学习下的半监督学习通常需要参与方之间复杂的交互，从而导致数据隐私的保护上存在隐患，无法有效的保护数据隐私。
上述内容仅用于辅助理解本申请的技术方案，并不代表承认上述内容是现有技术。
 技术问题
本申请的主要目的在于提供一种纵向联邦建模方法、装置、设备及计算机可读存储介质，旨在解决现有联邦学习中难以实现数据隐私保护与模型通用性之间的均衡的技术问题。
 技术解决方案
为实现上述目的，本申请提供一种纵向联邦建模方法，所述纵向联邦建模方法包括以下步骤：
第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
此外，为实现上述目的，本申请还提供一种纵向联邦建模方法，所述纵向联邦建模方法包括以下步骤：
第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
第二参与方基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
此外，为实现上述目的，本申请还提供一种纵向联邦建模装置，所述纵向联邦建模装置包括：
第一确定模块，用于确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
第二确定模块，用于确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
第一模型更新模块，用于基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
此外，为实现上述目的，本申请还提供一种纵向联邦建模装置，所述纵向联邦建模装置包括：
第一计算模块，用于基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
第二计算模块，用于基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
第二模型更新模块，用于基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
此外，为实现上述目的，本申请还提供一种纵向联邦建模设备，所述纵向联邦建模设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的纵向联邦建模程序，所述纵向联邦建模程序被所述处理器执行时实现前述的纵向联邦建模方法的步骤。
此外，为实现上述目的，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现前述的纵向联邦建模方法的步骤。
 有益效果
本申请通过第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；接着第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；而后第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。使得包括第一参与方以及第二参与方在内的各个参与方完成模型更新得到目标模型，第一参与方以及各个第二参与方均可采用自身的目标模型对其本地样本进行标签预测，第一参与方以及各个第二参与方也可采用自身的目标模型对各个参与方之间的重叠样本进行标签预测，以使所有参与方完成模型训练，使得模型训练过程能够适用于所有参与方；通过增加双方重叠样本的训练过程，以在各个参与方所共同拥有的重叠样本数量较少时，训练出性能优良的模型，各个参与方之间通过多方联邦计算中间参数以及双方联邦计算中间参数进行数据交互，与传统的半监督学习相比，无需对缺失特征和标签进行补全，简化了模型训练过程中的交互流程，能够完全阻止信息泄漏，提高联邦学习中数据的安全性。
 附图说明
图1为具有3个参与方的纵向联邦学习的样本和特征视图；
图2是本申请实施例方案涉及的硬件运行环境中的纵向联邦建模设备的结构示意图；
图3为本申请纵向联邦建模方法第一实施例的流程示意图；
图4为本申请纵向联邦建模方法另一实施例的流程示意图；
图5为本申请纵向联邦建模装置一实施例的功能模块示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
 本发明的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
如图2所示，图2是本申请实施例方案涉及的硬件运行环境中的纵向联邦建模设备的结构示意图。
本申请实施例纵向联邦建模设备可以是PC，也可以是智能手机、平板电脑、电子书阅读器、MP3（Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3）播放器、MP4（Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4）播放器、便携计算机等具有显示功能的可移动式终端设备。
如图2所示，该纵向联邦建模设备可以包括：处理器1001，例如CPU，网络接口1004，用户接口1003，存储器1005，通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏（Display）、输入单元比如键盘（Keyboard），可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口（如WI-FI接口）。存储器1005可以是高速RAM存储器，也可以是稳定的存储器（non-volatile memory），例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。
可选地，纵向联邦建模设备还可以包括摄像头、RF（Radio Frequency，射频）电路，传感器、音频电路、WiFi模块等等。当然，纵向联邦建模设备还可配置陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
本领域技术人员可以理解，图2中示出的纵向联邦建模设备结构并不构成对纵向联邦建模设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图2所示，作为一种计算机存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及纵向联邦建模程序。
在图2所示的纵向联邦建模设备中，网络接口1004主要用于连接后台服务器，与后台服务器进行数据通信；用户接口1003主要用于连接客户端（用户端），与客户端进行数据通信；而处理器1001可以用于调用存储器1005中存储的纵向联邦建模程序。
在本实施例中，纵向联邦建模设备包括：存储器1005、处理器1001及存储在所述存储器1005上并可在所述处理器1001上运行的纵向联邦建模程序，其中，处理器1001调用存储器1005中存储的纵向联邦建模程序时，并执行以下各个实施例中纵向联邦建模方法的步骤。
本申请还提供一种纵向联邦建模方法，参照图3，图3为本申请纵向联邦建模方法第一实施例的流程示意图。
该纵向联邦建模方法应包括：
步骤S100，第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
需要说明的是，第二参与方包括联邦学习中除第一参与方之外的多个其他参与方，也就是说，第二参与方为多个，各个第二参与方中均存储有样本，第一参与方存储有样本以及样本标签。以包括参与方A、参与方B和参与方C三个参与方联邦学习系统为例，参与方A、参与方B和参与方C拥有一定数量的重叠样本。
本实施例中，第一参与方将其本地样本与各个第二参与方的样本进行样本对齐，得到第一参与方的多方重叠样本，各个第二参与方分别将各自本地样本，与第一参与方的本地样本以及第二参与方中其他参与方的本地样本进行样本对齐，得到各个第二参与方的多方重叠样本，即该多方重叠样本为各个参与方的本地样本与其他参与方进行样本对齐得到的，第一参与方为拥有多方重叠样本的样本标签的参与方，在进行样本对齐时，各个参与方根据本地样本的用户标识（ID）进行样本对齐。各个多方重叠样本具有不同的样本特征，且各个多方重叠样本具有相同的ID。
第一参与方根据本地样本所对应的样本标签，确定多方重叠样本对应的多方样本标签，并根据该多方样本标签确定多方中间参数，具体地，可将多方样本标签输入待训练模型，以获得多方中间参数。而后，将多方中间参数发送至第二参与方，以避免在联邦学过程中传输样本标签，避免样本标签的泄露。
而后，各个第二参与方分别基于多方重叠样本，联合计算得到多方联邦计算中间参数，并将所述多方联邦计算中间参数发送至第一参与方，具体地，各个第二参与方基于多方重叠样本，分别将各自多方重叠样本中本地样本特征，输入各自多方联邦分类模型的第一特征提取模型，以获得第二特征表征；各个第二参与方分别将所述第二特征表征输入各自的多方联邦分类模型的第一分类模型，计算得到中间参数，例如，将第二特征表征以及多方中间参数输入各自的多方联邦分类模型的第一分类模型，计算得到中间参数，而后根据中间参数计算得到多方联邦计算中间参数，并反馈所述多方联邦计算中间参数至所述第一参与方。
步骤S200，第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
本实施例中，第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，具体地，各个第二参与方分别将其样本与第一参与方的本地样本进行样本对齐，得到各个第二参与方的双方重叠样本，各个第二参与方在确定双方重叠样本之后，分别反馈各个双方重叠样本对应的标识信息至第一参与方，第一参与方基于各个标识信息以及本地样本的样本标签，确定各个双方重叠样本对应的双方样本标签，并根据双方样本标签确定双方中间参数，其中，第一参与方的本地样本中双方重叠样本对应的样本的标识信息，与双方重叠样本对应的标识信息相同。
而后，各个第二参与方基于双方重叠样本，计算得到双方联邦计算中间参数，并将所述双方联邦计算中间参数发送至第一参与方，具体地，各个第二参与方分别将双方重叠样本中本地样本特征输入各自双方联邦分类模型的第二特征提取模型，以获得第三特征表征；各个第二参与方分别将所述第三特征表征输入各自的双方联邦分类模型的第一分类模型，计算得到双方联邦计算中间参数，并反馈所述双方联邦计算中间参数至所述第一参与方，具体地，各个第二参与方分别将所述第三特征表征以及双方中间参数，输入各自的双方联邦分类模型的第一分类模型，得到双方联邦计算中间参数。
步骤S300，第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
本实施例中，在获得双方联邦计算中间参数以及多方联邦计算中间参数之后，第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作，具体地，第一参与方基于所述多方联邦计算中间参数确定多方计算损失值，基于各个所述双方联邦计算中间参数确定各个双方计算损失值，分别基于各个双方计算损失值以及所述多方计算损失值，确定多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作，即根据多方联邦计算中间梯度以及中间梯度执行模型更新操作，以获得目标模型，同时，第一参与方将多方联邦计算中间梯度、中间梯度以及双方联邦计算中间梯度反馈至各个第二参与方，第二参与方根据多方联邦计算中间梯度、中间梯度以及双方联邦计算中间梯度更新各自的模型。
需要说明的是，在第一参与方以及第二参与方均得到训练好的模型后，第一参与方以及各个第二参与方均可采用自身的目标模型对其本地样本进行标签预测，第一参与方以及各个第二参与方也可采用自身的目标模型对各个参与方之间的重叠样本进行标签预测。
本实施例中，通过增加双方重叠样本的训练过程，以在各个参与方所共同拥有的重叠样本数量较少时，训练出性能优良的模型，与联邦迁移学习相比，减少样本迁移学习的流程，提升各个参与方对应的终端的处理效率，同时，各个参与方之间通过多方联邦计算中间参数以及双方联邦计算中间参数进行数据交互，能够完全阻止信息泄漏，提高联邦学习中数据的安全性。
本实施例提出的纵向联邦建模方法，通过第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；接着第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；而后第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。使得包括第一参与方以及第二参与方在内的各个参与方完成模型更新得到目标模型，第一参与方以及各个第二参与方均可采用自身的目标模型对其本地样本进行标签预测，第一参与方以及各个第二参与方也可采用自身的目标模型对各个参与方之间的重叠样本进行标签预测，以使所有参与方完成模型训练，使得模型训练过程能够适用于所有参与方；通过增加双方重叠样本的训练过程，以在各个参与方所共同拥有的重叠样本数量较少时，训练出性能优良的模型，各个参与方之间通过多方联邦计算中间参数以及双方联邦计算中间参数进行数据交互，与传统的半监督学习相比，无需对缺失特征和标签进行补全，简化了模型训练过程中的交互流程，能够完全阻止信息泄漏，提高联邦学习中数据的安全性。
基于第一实施例，提出本申请纵向联邦建模方法的第二实施例，在本实施例中，步骤S300包括：
步骤S310，第一参与方基于第一参与方的本地样本确定单方损失值，基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值，并基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值；
步骤S320，第一参与方分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
本实施例中，在获得双方联邦计算中间参数以及多方联邦计算中间参数之后，第一参与方基于本地样本确定单方损失值，基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值；并基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值，以根据双方联邦计算中间参数以及多方联邦计算中间参数准确得到多方计算损失值以及双方计算损失值。
本实施例中，在获取到单方损失值、多方计算损失值以及双方计算损失值之后，分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，具体地，基于单方损失值确定单方梯度，基于多方计算损失值确定多方联邦计算中间梯度，基于双方计算损失值确定各个双方联邦计算中间梯度，而后，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并执行模型更新操作，即根据单方梯度所述、多方联邦计算中间梯度以及所述中间梯度执行模型更新操作，以获得目标模型。
本实施例提出的纵向联邦建模方法，通过第一参与方基于第一参与方的本地样本确定单方损失值，基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值，并基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值，接着第一参与方分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作，能够根据多方联邦计算中间参数以及双方联邦计算中间参数实现模型更新，能够完全阻止信息泄漏，进一步提高联邦学习中数据的安全性。
基于第二实施例，提出本申请纵向联邦建模方法的第三实施例，在本实施例中，步骤S310包括：
步骤S311，第一参与方将第一参与方的本地样本中的多方重叠样本输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征，将所述多方重叠样本的特征表征、样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型计算得到所述多方计算损失值。
本实施例中，在获得双方联邦计算中间参数以及多方联邦计算中间参数之后，第一参与方将第一参与方的本地样本中的多方重叠样本输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征，具体地，第一参与方获取其本地样本中的多方重叠样本的多方重叠样本特征，并将该多方重叠样本特征输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征。
而后，第一参与方将多方重叠样本的特征表征、样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型，具体地，第一参与方在样本标签中获取多方重叠样本对应的样本标签，将多方重叠样本的特征表征、多方重叠样本对应的样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型，得到所述多方计算损失值。
进一步地，在一实施例中，步骤S300还包括：
步骤S322，第一参与方将第一参与方的本地样本中的各个双方重叠样本输入第一参与方的特征提取模型，以获得各个双方重叠样本的特征表征，将所述各个双方重叠样本的特征表征、样本标签以及各个所述双方联邦计算中间参数输入对应的各个双方联邦模型的第一双方分类模型计算得到所述各个双方计算损失值。
本实施例中，在获得双方联邦计算中间参数以及多方联邦计算中间参数之后，第一参与方将第一参与方的本地样本中的各个双方重叠样本输入第一参与方的特征提取模型，以获得各个双方重叠样本的特征表征，第一参与方获取其本地样本中的各个双方重叠样本的双方重叠样本特征，并分别将各个双方重叠样本特征输入第一参与方的特征提取模型，以获得各个双方重叠样本的特征表征。
而后，第一参与方将所述各个双方重叠样本的特征表征、样本标签以及各个所述双方联邦计算中间参数输入对应的各个双方联邦模型的第一双方分类模型，具体地，第一参与方在样本标签中获取各个双方重叠样本对应的样本标签，分别将各个双方重叠样本的特征表征、各个双方重叠样本对应的样本标签以及所述多方联邦计算中间参数输入双方联邦模型的第一双方分类模型，得到双方计算损失值。
进一步地，又一实施例中，步骤S300还包括：
步骤S333，第一参与方将第一参与方的本地样本输入第一参与方的特征提取模型，以获得第一特征表征，将所述第一特征表征以及样本标签，输入单方分类模型计算得到所述单方损失值。
本实施例中，在获得双方联邦计算中间参数以及多方联邦计算中间参数之后，第一参与方将第一参与方的本地样本输入第一参与方的特征提取模型进行模型训练，以获得第一特征表征，而后将第一特征表征以及样本标签，输入单方分类模型进行模型训练，得到所述单方损失值。
本实施例提出的纵向联邦建模方法，通过第一参与方将第一参与方的本地样本中的多方重叠样本输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征，将所述多方重叠样本的特征表征、样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型计算得到所述多方计算损失值，能够根据多方联邦计算中间参数准确得到多方计算损失值，提升了多方计算损失值的准确性，进而提高了联邦学习的效率以及准确性。
基于第二实施例，提出本申请纵向联邦建模方法的第四实施例，在本实施例中，步骤S320包括：
步骤S410，第一参与方基于所述单方损失值计算得到单方梯度，并基于单方梯度，更新第一参与方的单方分类模型；
步骤S420，第一参与方基于所述多方计算损失值计算得到多方联邦计算中间梯度；基于多方联邦计算中间梯度，更新多方联邦分类模型的第一多方分类模型；
步骤S430，第一参与方基于所述各个双方计算损失值计算得到各个双方联邦计算中间梯度；基于各个双方联邦计算中间梯度，对应更新各个双方联邦分类模型的第一双方分类模型；
步骤S430，第一参与方基于单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，更新第一参与方的特征提取模型。
本实施例中，在获取到单方损失值、多方计算损失值以及双方计算损失值之后，第一参与方基于所述单方损失值计算得到单方梯度，例如采用梯度下降法计算单方梯度，并基于单方梯度，更新单方分类模型，即根据单方梯度分别更新单方分类模型的模型参数。
接着，第一参与方基于所述多方计算损失值计算得到多方联邦计算中间梯度，并基于多方联邦计算中间梯度，更新多方联邦分类模型的第一多方分类模型，即根据多方联邦计算中间梯度更新第一参与方中多方联邦分类模型的第一多方分类模型。同时，将多方联邦计算中间梯度反馈给各个第二参与方，在接收到多方联邦计算中间梯度后，各个第二参与方更新各自多方联邦分类模型的第二多方分类模型，即每个第二参与方分别根据多方联邦计算中间梯度更新自身多方联邦分类模型的第二多方分类模型的模型参数。
同时，第一参与方基于所述各个双方计算损失值计算得到各个双方联邦计算中间梯度，例如采用加权平均算法根据各个双方计算损失值计算得到各个双方联邦计算中间梯度，并基于各个双方联邦计算中间梯度，对应更新各个双方联邦分类模型的第一双方分类模型；同时，将各个双方联邦计算中间梯度反馈给各个对应的第二参与方，以供各个第二参与方更新各自双方联邦分类模型的第二双方分类模型。
并且，第一参与方基于单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，更新第一参与方的特征提取模型，例如，根据单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度进行融合得到第一目标梯度，根据第一目标梯度更新第一参与方的特征提取模型的模型参数。而各个第二参与方则分别基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新各自的特征提取模型，具体地，根据多方联邦计算中间梯度以及对应的双方联邦计算中间梯度进行融合得到各自的第二目标梯度，各个第二参与方根据第二目标梯度更新自身的特征提取模型的模型参数。
本实施例提出的纵向联邦建模方法，通过第一参与方基于所述单方损失值计算得到单方梯度，并基于单方梯度，更新第一参与方的单方分类模型；接着第一参与方基于所述多方计算损失值计算得到多方联邦计算中间梯度；基于多方联邦计算中间梯度，更新多方联邦分类模型的第一多方分类模型；而后第一参与方基于所述各个双方计算损失值计算得到各个双方联邦计算中间梯度；基于各个双方联邦计算中间梯度，对应更新各个双方联邦分类模型的第一双方分类模型；然后基于单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，更新第一参与方的特征提取模型，能够根据单方损失值、多方计算损失值以及双方计算损失值更新各个参与方的双方联邦分类模型以及多方联邦分类模型，进而准确实现模型的更新，得到训练好的模型，进一步提高了联邦学习的效率以及准确性。
基于第二实施例，提出本申请纵向联邦建模方法的第五实施例，在本实施例中，步骤S320之后，还包括：
步骤S510，第一参与方基于所述单方损失值、所述各个双方损失值以及所述多方计算损失值，计算总损失值；
步骤S520，若总损失值小于预设阈值，则第一参与方将模型更新操作后获得的模型作为目标模型；
步骤S530，若总损失值大于或等于预设阈值，则第一参与方基于模型更新操作后获得的模型，继续执行第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本的步骤。
本实施例中，在更新各个参与方的模型之后，第一参与方基于所述单方损失值、所述各个双方损失值以及所述多方计算损失值，计算总损失值，具体地，可获取单方损失值、各个双方损失值以及多方计算损失值对应的权重，并基于单方损失值、所述各个双方损失值以及所述多方计算损失值以及各自对应的权重，计算总损失值，例如，总损失值=多方计算损失值+单方损失值*单方损失值对应的权重+双方损失值1*双方损失值1对应的权重……+双方损失值n*双方损失值n对应的权重。
而后，第一参与方判断总损失值是否小于预设阈值，若小于，则第一参与方将模型更新操作后获得的模型作为目标模型。否则，第一参与方基于模型更新操作后获得的模型，继续执行第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本的步骤，具体地，将更新后的第三特征提取模型作为第一参与方的第三特征提取模型，更新后的第三分类模型作为第一参与方的第三分类模型，更新后的第一参与方的多方联邦分类模型作为第一参与方的多方联邦分类模型，更新后第一参与方的双方联邦分类模型作为第一参与方的双方联邦分类模型，更新后各个第二参与方的多方联邦分类模型作为各个第二参与方的多方联邦分类模型作，更新后各个第二参与方的双方联邦分类模型作为各个第二参与方的双方联邦分类模型作，并继续执行步骤S100，直至新的总损失值小于预设阈值。
本实施例提出的纵向联邦建模方法，通过第一参与方基于所述单方损失值、所述各个双方损失值以及所述多方计算损失值，计算总损失值；接着若总损失值小于预设阈值，则第一参与方将模型更新操作后获得的模型作为目标模型；而后若总损失值大于或等于预设阈值，则第一参与方基于模型更新操作后获得的模型，继续执行确定第一参与方与各个第二参与方之间的多方重叠样本的步骤，通过确保总损失值收敛，以得到符合要求的模型，进一步提升联邦学习的效率。
本申请还提供一种纵向联邦建模方法，参照图4，图4为本申请纵向联邦建模方法另一实施例的流程示意图。
该纵向联邦建模方法应包括：
步骤610，第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
本实施例中，第二参与方分别将各自本地样本，与第一参与方的本地样本以及第二参与方中其他参与方的本地样本进行样本对齐，得到各个第二参与方的多方重叠样本，即该多方重叠样本为各个参与方的本地样本与其他参与方进行样本对齐得到的。各个多方重叠样本具有不同的样本特征，且各个多方重叠样本具有相同的ID。
而后，第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，具体的，各个第二参与方基于多方重叠样本，分别将各自多方重叠样本中本地样本特征，输入各自多方联邦分类模型的第一特征提取模型，以获得第二特征表征；各个第二参与方分别将所述第二特征表征输入各自的多方联邦分类模型的第一分类模型，计算得到中间参数，例如，将第二特征表征以及多方中间参数输入各自的多方联邦分类模型的第一分类模型，计算得到中间参数，而后根据中间参数计算得到多方联邦计算中间参数，并反馈所述多方联邦计算中间参数至所述第一参与方。
步骤620，第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
本实施例中，各个第二参与方分别将其样本与第一参与方的本地样本进行样本对齐，得到各个第二参与方的双方重叠样本，各个第二参与方在确定双方重叠样本之后，分别反馈各个双方重叠样本对应的标识信息至第一参与方。
而后，第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数，具体地，各个第二参与方分别将双方重叠样本中本地样本特征输入各自双方联邦分类模型的第二特征提取模型，以获得第三特征表征；各个第二参与方分别将所述第三特征表征输入各自的双方联邦分类模型的第一分类模型，计算得到双方联邦计算中间参数，并反馈所述双方联邦计算中间参数至所述第一参与方，具体地，各个第二参与方分别将所述第三特征表征以及双方中间参数，输入各自的双方联邦分类模型的第一分类模型，得到双方联邦计算中间参数。
步骤630，第二参与方基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
其中，目标梯度包括目标梯度包括多方联邦计算中间梯度以及双方联邦计算中间梯度，第二参与方基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新自身的多方联邦分类模型、双方联邦分类模型以及特征提取模型。
进一步地，在一实施例中，步骤S610包括：
步骤611，第二参与方将各自多方重叠样本中本地样本特征输入各自的特征提取模型，以获得第二特征表征；
步骤612，第二参与方将所述第二特征表征输入各自的多方联邦分类模型的第二多方分类模型，计算得到中间参数，并基于各个第二参与方的中间参数联合计算得到多方联邦计算中间参数。
本实施例中，各个第二参与方分别将各自多方重叠样本中本地样本特征输入各自的特征提取模型，以获得第二特征表征；而后，各个第二参与方分别将第二特征表征输入各自的多方联邦分类模型的第二多方分类模型，计算得到中间参数，例如，将第二特征表征以及多方中间参数输入各自的多方联邦分类模型的第二多方分类模型，计算得到中间参数；接着基于各个第二参与方的中间参数联合计算得到多方联邦计算中间参数，并反馈所述多方联邦计算中间参数至所述第一参与方，例如，各个第二参与方将中间参数发送至协调方，协调方根据各个中间参数联合计算得到多方联邦计算中间参数，并反馈所述多方联邦计算中间参数至所述第一参与方。
进一步地，另一实施例中，步骤S620包括：
步骤621，第二参与方将双方重叠样本中本地样本特征输入各自的特征提取模型，以获得第三特征表征；
步骤622，第二参与方将所述第三特征表征输入各自的双方联邦分类模型的第二双方分类模型，计算得到双方联邦计算中间参数。
本实施例中，各个第二参与方分别基于双方重叠样本计算得到双方联邦计算中间参数，并将所述双方联邦计算中间参数发送至第一参与方，具体地，第一参与方分别将双方中间参数发送至对应的第二参与方，各个第二参与方分别将双方重叠样本中本地样本特征输入各自的特征提取模型，以获得第三特征表征；各个第二参与方分别将所述第三特征表征输入各自的双方联邦分类模型的第二双方分类模型，计算得到双方联邦计算中间参数，并反馈所述双方联邦计算中间参数至所述第一参与方，具体地，各个第二参与方分别将所述第三特征表征以及双方中间参数，输入各自的双方联邦分类模型的第二双方分类模型，得到双方联邦计算中间参数。
进一步地，又一实施例中，所述目标梯度包括多方联邦计算中间梯度以及双方联邦计算中间梯度，步骤S630包括：
步骤631，第二参与方基于多方联邦计算中间梯度更新多方联邦分类模型的第二多方分类模型；
步骤632，第二参与方基于双方联邦计算中间梯度更新双方联邦分类模型的第二双方分类模型；
步骤633，第二参与方基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新特征提取模型。
本实施例中，在接收到多方联邦计算中间梯度以及双方联邦计算中间梯度后，各个第二参与方更新各自多方联邦分类模型的第二多方分类模型，即每个第二参与方分别根据多方联邦计算中间梯度更新自身多方联邦分类模型的第二多方分类模型的模型参数，并基于多方联邦计算中间梯度更新多方联邦分类模型的第二多方分类模型。
同时，各个第二参与方则分别基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新各自的特征提取模型，具体地，根据多方联邦计算中间梯度以及对应的双方联邦计算中间梯度进行融合得到各自的第二目标梯度，各个第二参与方根据第二目标梯度更新自身的特征提取模型的模型参数。
本实施例中，通过第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；接着第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；而后第二参与方基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作，使得第二参与方完成模型更新得到目标模型，各个第二参与方均可采用自身的目标模型对其本地样本进行标签预测，第一参与方以及各个第二参与方也可采用自身的目标模型对各个参与方之间的重叠样本进行标签预测，以使所有参与方完成模型训练，使得模型训练过程能够适用于所有参与方；通过增加双方重叠样本的训练过程，以在各个参与方所共同拥有的重叠样本数量较少时，训练出性能优良的模型，各个参与方之间通过多方联邦计算中间参数以及双方联邦计算中间参数进行数据交互，与传统的半监督学习相比，无需对缺失特征和标签进行补全，简化了模型训练过程中的交互流程，能够完全阻止信息泄漏，提高联邦学习中数据的安全性。
本申请实施例还提供一种纵向联邦建模装置，参照图5，所述纵向联邦建模装置包括：
第一确定模块100，用于确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
第二确定模块200，用于确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
模型更新模块300，用于基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
可选地，模型更新模块300还用于：
基于第一参与方的本地样本确定单方损失值，基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值，并基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值；
分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
可选地，模型更新模块300还用于：
将第一参与方的本地样本中的多方重叠样本输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征，将所述多方重叠样本的特征表征、样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型计算得到所述多方计算损失值。
可选地，模型更新模块300还用于：
将第一参与方的本地样本中的各个双方重叠样本输入第一参与方的特征提取模型，以获得各个双方重叠样本的特征表征，将所述各个双方重叠样本的特征表征、样本标签以及各个所述双方联邦计算中间参数输入对应的各个双方联邦模型的第一双方分类模型计算得到所述各个双方计算损失值。
可选地，模型更新模块300还用于：
将第一参与方的本地样本输入第一参与方的特征提取模型，以获得第一特征表征，将所述第一特征表征以及样本标签，输入单方分类模型计算得到所述单方损失值。
可选地，模型更新模块300还用于：
基于所述单方损失值计算得到单方梯度，并基于单方梯度，更新第一参与方的单方分类模型；
基于所述多方计算损失值计算得到多方联邦计算中间梯度；基于多方联邦计算中间梯度，更新多方联邦分类模型的第一多方分类模型；
基于所述各个双方计算损失值计算得到各个双方联邦计算中间梯度；基于各个双方联邦计算中间梯度，对应更新各个双方联邦分类模型的第一双方分类模型；
基于单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，更新第一参与方的特征提取模型。
本申请实施例还提供一种纵向联邦建模装置，所述纵向联邦建模装置包括：
纵向联邦建模装置，其中，所述纵向联邦建模装置包括：
第一计算模块，用于基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
第二计算模块，用于基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
第二模型更新模块，用于基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
可选地，第一计算模块还用于：
将各自多方重叠样本中本地样本特征输入各自的特征提取模型，以获得第二特征表征；
将所述第二特征表征输入各自的多方联邦分类模型的第二多方分类模型，计算得到中间参数，并基于各个第二参与方的中间参数联合计算得到多方联邦计算中间参数。
可选地，第二计算模块还用于：
将双方重叠样本中本地样本特征输入各自的特征提取模型，以获得第三特征表征；
将所述第三特征表征输入各自的双方联邦分类模型的第二双方分类模型，计算得到双方联邦计算中间参数。
可选地，第二模型更新模块还用于：
基于多方联邦计算中间梯度更新多方联邦分类模型的第二多方分类模型；
基于双方联邦计算中间梯度更新双方联邦分类模型的第二双方分类模型；
基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新特征提取模型。
此外，本申请实施例还提出一种计算机可读存储介质，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现如上所述的纵向联邦建模方法的步骤。
其中，在所述处理器上运行的纵向联邦建模程序被执行时所实现的方法可参照本申请纵向联邦建模方法各个实施例，此处不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种纵向联邦建模方法，其中，所述纵向联邦建模方法包括以下步骤：
    第一参与方确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
    第一参与方确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
    第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
  2. 如权利要求1所述的纵向联邦建模方法，其中，所述第一参与方基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作的步骤包括：
    第一参与方基于第一参与方的本地样本确定单方损失值，基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值，并基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值；
    第一参与方分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
  3. 如权利要求2所述的纵向联邦建模方法，其中，所述基于第一参与方的本地样本和所述多方联邦计算中间参数确定多方计算损失值的步骤包括：
    第一参与方将第一参与方的本地样本中的多方重叠样本输入第一参与方的特征提取模型，以获得多方重叠样本的特征表征，将所述多方重叠样本的特征表征、样本标签以及所述多方联邦计算中间参数输入多方联邦模型的第一多方分类模型计算得到所述多方计算损失值。
  4. 如权利要求2所述的纵向联邦建模方法，其中，所述基于第一参与方的本地样本和各个所述双方联邦计算中间参数确定各个双方计算损失值的步骤包括：
    第一参与方将第一参与方的本地样本中的各个双方重叠样本输入第一参与方的特征提取模型，以获得各个双方重叠样本的特征表征，将所述各个双方重叠样本的特征表征、样本标签以及各个所述双方联邦计算中间参数输入对应的各个双方联邦模型的第一双方分类模型计算得到所述各个双方计算损失值。
  5. 如权利要求2所述的纵向联邦建模方法，其中，所述第一参与方基于第一参与方的本地样本确定单方损失值的步骤包括：
    第一参与方将第一参与方的本地样本输入第一参与方的特征提取模型，以获得第一特征表征，将所述第一特征表征以及样本标签，输入单方分类模型计算得到所述单方损失值。
  6. 如权利要求2至5任一项所述的纵向联邦建模方法，其中，所述第一参与方分别基于所述单方损失值、所述各个双方计算损失值以及所述多方计算损失值，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，并基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作的步骤包括：
    第一参与方基于所述单方损失值计算得到单方梯度，并基于单方梯度，更新第一参与方的单方分类模型；
    第一参与方基于所述多方计算损失值计算得到多方联邦计算中间梯度；基于多方联邦计算中间梯度，更新多方联邦分类模型的第一多方分类模型；
    第一参与方基于所述各个双方计算损失值计算得到各个双方联邦计算中间梯度；基于各个双方联邦计算中间梯度，对应更新各个双方联邦分类模型的第一双方分类模型；
    第一参与方基于单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，更新第一参与方的特征提取模型。
  7. 一种纵向联邦建模方法，其中，所述纵向联邦建模方法包括以下步骤：
    第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
    第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
    第二参与方基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
  8. 如权利要求7所述的纵向联邦建模方法，其中，所述第二参与方基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数的步骤包括：
    第二参与方将各自多方重叠样本中本地样本特征输入各自的特征提取模型，以获得第二特征表征；
    第二参与方将所述第二特征表征输入各自的多方联邦分类模型的第二多方分类模型，计算得到中间参数，并基于各个第二参与方的中间参数联合计算得到多方联邦计算中间参数。
  9. 如权利要求7所述的纵向联邦建模方法，其中，所述第二参与方基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数的步骤包括：
    第二参与方将双方重叠样本中本地样本特征输入各自的特征提取模型，以获得第三特征表征；
    第二参与方将所述第三特征表征输入各自的双方联邦分类模型的第二双方分类模型，计算得到双方联邦计算中间参数。
  10. 如权利要求7至9任一项所述的纵向联邦建模方法，其中，所述目标梯度包括多方联邦计算中间梯度以及双方联邦计算中间梯度，所述基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作的步骤包括：
    第二参与方基于多方联邦计算中间梯度更新多方联邦分类模型的第二多方分类模型；
    第二参与方基于双方联邦计算中间梯度更新双方联邦分类模型的第二双方分类模型；
    第二参与方基于多方联邦计算中间梯度以及双方联邦计算中间梯度更新特征提取模型。
  11. 一种纵向联邦建模装置，其中，所述纵向联邦建模装置包括：
    第一确定模块，用于确定第一参与方与各个第二参与方之间的多方重叠样本，并基于多方重叠样本获取各个第二参与方对应的多方联邦计算中间参数；
    第二确定模块，用于确定各个第二参与方与第一参与方之间的双方重叠样本，并基于双方重叠样本分别获取各个第二参与方对应的双方联邦计算中间参数；
    第一模型更新模块，用于基于第一参与方的本地样本、多方联邦计算中间参数以及各个所述双方联邦计算中间参数，确定单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度，基于所述单方梯度、多方联邦计算中间梯度以及各个双方联邦计算中间梯度并执行模型更新操作。
  12. 一种纵向联邦建模装置，其中，所述纵向联邦建模装置包括：
    第一计算模块，用于基于与第一参与方之间的多方重叠样本，联合计算得到多方联邦计算中间参数，其中，第二参与方包括多个参与方；
    第二计算模块，用于基于与第一参与方之间的双方重叠样本，分别计算得到双方联邦计算中间参数；
    第二模型更新模块，用于基于接收到的第一参与方基于所述多方联邦计算中间参数以及各个双方联邦计算中间参数反馈的目标梯度，执行模型更新操作。
  13. 一种纵向联邦建模设备，其中，所述纵向联邦建模设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的纵向联邦建模程序，所述纵向联邦建模程序被所述处理器执行时实现如权利要求1所述的纵向联邦建模方法的步骤。
  14. 一种纵向联邦建模设备，其中，所述纵向联邦建模设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的纵向联邦建模程序，所述纵向联邦建模程序被所述处理器执行时实现如权利要求2所述的纵向联邦建模方法的步骤。
  15. 一种纵向联邦建模设备，其中，所述纵向联邦建模设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的纵向联邦建模程序，所述纵向联邦建模程序被所述处理器执行时实现如权利要求7所述的纵向联邦建模方法的步骤。
  16. 一种纵向联邦建模设备，其中，所述纵向联邦建模设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的纵向联邦建模程序，所述纵向联邦建模程序被所述处理器执行时实现如权利要求8所述的纵向联邦建模方法的步骤。
  17. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现如权利要求1所述的纵向联邦建模方法的步骤。
  18. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现如权利要求2所述的纵向联邦建模方法的步骤。
  19. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现如权利要求7所述的纵向联邦建模方法的步骤。
  20. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有纵向联邦建模程序，所述纵向联邦建模程序被处理器执行时实现如权利要求8所述的纵向联邦建模方法的步骤。
PCT/CN2021/095251 2020-09-07 2021-05-21 纵向联邦建模方法、装置、设备及计算机可读存储介质 WO2022048195A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010932992.7A CN112052960A (zh) 2020-09-07 2020-09-07 纵向联邦建模方法、装置、设备及计算机可读存储介质
CN202010932992.7 2020-09-07

Publications (1)

Publication Number Publication Date
WO2022048195A1 true WO2022048195A1 (zh) 2022-03-10

Family

ID=73610208

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/095251 WO2022048195A1 (zh) 2020-09-07 2021-05-21 纵向联邦建模方法、装置、设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN112052960A (zh)
WO (1) WO2022048195A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023197467A1 (zh) * 2022-04-13 2023-10-19 上海阵方科技有限公司 一种使安全多方计算与联邦学习数据互联的系统

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112052960A (zh) * 2020-09-07 2020-12-08 深圳前海微众银行股份有限公司 纵向联邦建模方法、装置、设备及计算机可读存储介质
CN112836767A (zh) * 2021-03-03 2021-05-25 深圳前海微众银行股份有限公司 联邦建模方法、装置、设备、存储介质及程序产品
CN113298191B (zh) * 2021-04-01 2022-09-06 山东大学 基于个性化半监督在线联邦学习的用户行为识别方法
CN113536667B (zh) * 2021-06-22 2024-03-01 同盾科技有限公司 联邦模型训练方法、装置、可读存储介质及设备
CN113537361B (zh) * 2021-07-20 2024-04-02 同盾科技有限公司 联邦学习系统中的跨样本特征选择方法及联邦学习系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080082475A1 (en) * 2006-09-12 2008-04-03 International Business Machines Corporation System and method for resource adaptive classification of data streams
CN106445936A (zh) * 2015-08-05 2017-02-22 中国移动通信集团福建有限公司 一种数据处理方法及设备
CN109492420A (zh) * 2018-12-28 2019-03-19 深圳前海微众银行股份有限公司 基于联邦学习的模型参数训练方法、终端、系统及介质
CN111582505A (zh) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 联邦建模方法、装置、设备及计算机可读存储介质
CN112052960A (zh) * 2020-09-07 2020-12-08 深圳前海微众银行股份有限公司 纵向联邦建模方法、装置、设备及计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080082475A1 (en) * 2006-09-12 2008-04-03 International Business Machines Corporation System and method for resource adaptive classification of data streams
CN106445936A (zh) * 2015-08-05 2017-02-22 中国移动通信集团福建有限公司 一种数据处理方法及设备
CN109492420A (zh) * 2018-12-28 2019-03-19 深圳前海微众银行股份有限公司 基于联邦学习的模型参数训练方法、终端、系统及介质
CN111582505A (zh) * 2020-05-14 2020-08-25 深圳前海微众银行股份有限公司 联邦建模方法、装置、设备及计算机可读存储介质
CN112052960A (zh) * 2020-09-07 2020-12-08 深圳前海微众银行股份有限公司 纵向联邦建模方法、装置、设备及计算机可读存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023197467A1 (zh) * 2022-04-13 2023-10-19 上海阵方科技有限公司 一种使安全多方计算与联邦学习数据互联的系统

Also Published As

Publication number Publication date
CN112052960A (zh) 2020-12-08

Similar Documents

Publication Publication Date Title
WO2022048195A1 (zh) 纵向联邦建模方法、装置、设备及计算机可读存储介质
WO2021120676A1 (zh) 联邦学习网络下的模型训练方法及其相关设备
US20210201147A1 (en) Model training method, machine translation method, computer device, and storage medium
TWI694396B (zh) 匯款處理方法、系統及電腦可讀存儲介質
US10891161B2 (en) Method and device for virtual resource allocation, modeling, and data prediction
WO2022016964A1 (zh) 纵向联邦建模优化方法、设备及可读存储介质
WO2021083276A1 (zh) 横向联邦和纵向联邦联合方法、装置、设备及介质
WO2023005133A1 (zh) 联邦学习建模优化方法、设备、可读存储介质及程序产品
WO2022022024A1 (zh) 训练样本构建方法、装置、设备及计算机可读存储介质
WO2022007321A1 (zh) 纵向联邦建模优化方法、装置、设备及可读存储介质
WO2022193432A1 (zh) 模型参数更新方法、装置、设备、存储介质及程序产品
WO2021151336A1 (zh) 基于注意力机制的道路图像目标检测方法及相关设备
WO2021174877A1 (zh) 基于智能决策的目标检测模型的处理方法、及其相关设备
CN110837653B (zh) 标签预测方法、装置以及计算机可读存储介质
WO2022100452A1 (zh) Ocr系统的评估方法、装置、设备及可读存储介质
WO2021228110A1 (zh) 联邦建模方法、装置、设备及计算机可读存储介质
CN111815169B (zh) 业务审批参数配置方法及装置
WO2022105297A1 (zh) 表格结构的还原方法、系统、计算机设备及存储介质
CN113627085A (zh) 横向联邦学习建模优化方法、设备、介质及程序产品
CN112785002A (zh) 模型构建优化方法、设备、介质及计算机程序产品
CN106470200A (zh) 一种身份验证方法及相关装置
WO2023024350A1 (zh) 纵向联邦预测优化方法、设备、介质及计算机程序产品
CN112381236A (zh) 联邦迁移学习的数据处理方法、装置、设备及存储介质
CN111737921A (zh) 基于循环神经网络的数据处理方法、设备及介质
CN111737920A (zh) 基于循环神经网络的数据处理方法、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21863265

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21863265

Country of ref document: EP

Kind code of ref document: A1