WO2021092977A1 - 纵向联邦学习优化方法、装置、设备及存储介质 - Google Patents

纵向联邦学习优化方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021092977A1
WO2021092977A1 PCT/CN2019/119409 CN2019119409W WO2021092977A1 WO 2021092977 A1 WO2021092977 A1 WO 2021092977A1 CN 2019119409 W CN2019119409 W CN 2019119409W WO 2021092977 A1 WO2021092977 A1 WO 2021092977A1
Authority
WO
WIPO (PCT)
Prior art keywords
participant
target
main
encrypted
data
Prior art date
Application number
PCT/CN2019/119409
Other languages
English (en)
French (fr)
Inventor
范涛
杨恺
陈天健
杨强
Original Assignee
深圳前海微众银行股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海微众银行股份有限公司 filed Critical 深圳前海微众银行股份有限公司
Publication of WO2021092977A1 publication Critical patent/WO2021092977A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis

Definitions

  • This application relates to the technical field of financial technology (Fintech), in particular to vertical federated learning optimization methods, devices, equipment, and storage media.
  • the principle of this scheme is to first use the loss function of the logistic regression in the second-order Taylor expansion of the zero point, and then the approximate loss function value and the gradient value can be obtained through the data federation of the AB parties, and the homomorphic encryption will be used to calculate the middle
  • the result is encrypted, decrypted by the C party, and transmitted back to the AB parties for model parameter update.
  • the route adopted in this scheme is the random gradient descent method, that is, a small batch of data is randomly selected, and the two parties AB obtain the encrypted gradient value of their respective parameter components through calculation, encryption and data interaction, and send it to the C party for decryption and multiply the corresponding step size. Get the direction of descent.
  • Stochastic gradient descent is a first-order algorithm with slow convergence speed and requires a large number of rounds of data interaction, which makes the current vertical federated learning required high communication volume, consumes large network bandwidth, and high computational cost.
  • the main purpose of this application is to propose a vertical federated learning optimization method, device, equipment and storage medium, which aims to solve the current technical problem of high communication volume for vertical federated learning.
  • the longitudinal federated learning optimization method includes the following steps:
  • this application provides a longitudinal federated learning optimization method including the following steps:
  • the target value set includes an encrypted value set corresponding to the data and a new encrypted value set corresponding to the new data
  • the target main gradient value is sent to the main participant respectively, and the main participant is used to update the local model parameters in the main participant based on the target main gradient value, and continue to execute the acquisition of data.
  • the present application also provides a longitudinal federated learning optimization device, the longitudinal federated learning optimization device includes:
  • the obtaining module is used to obtain the encrypted value set of the data in the main participant, and obtain the new encrypted value set of the new data in the main participant based on a preset judgment condition;
  • a sending module configured to send the encrypted value set and the new encrypted value set as a target value set to a secondary participant, so as to obtain an intermediate result value fed back by the secondary participant based on the target value set;
  • the first calculation module is configured to calculate the main encrypted data in the main participant according to the intermediate result value, and send the main encrypted data to the coordinator, where the coordinator is used to respond to the vertical logistic regression
  • the model does not converge, update the second-order derivative matrix according to the primary encrypted data and the secondary encrypted data sent by the secondary participant, and calculate the target primary gradient value according to the updated second-order derivative matrix;
  • the first update module is configured to receive the target main gradient value fed back by the coordinator, update the local model parameters in the main participant based on the target main gradient value, and continue to execute the acquisition of data in the main participant Until the longitudinal logistic regression model corresponding to the coordinator converges.
  • the longitudinal federated learning optimization device includes:
  • the receiving module is used to receive the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, where the primary encrypted data is an intermediate result based on the secondary participant's sending of a target value set feedback based on the primary participant Value calculated, the target value set includes an encrypted value set corresponding to the data and a new encrypted value set corresponding to the new data;
  • the second calculation module is used to update the second derivative matrix according to the main encrypted data and the auxiliary encrypted data in response to the longitudinal logistic regression model not converging, and calculate the target main gradient value according to the updated second derivative matrix ;
  • the second update module is configured to send the target main gradient value to the main participant respectively, and the main participant is used to update the local model parameters in the main participant based on the target main gradient value, and Continue to perform the step of obtaining the encrypted value set of the data in the main participant until the longitudinal logistic regression model corresponding to the coordinator converges.
  • this application also provides a longitudinal federated learning optimization device
  • the longitudinal federated learning optimization device includes: a memory, a processor, and a computer stored on the memory and capable of running on the processor
  • a readable instruction when the computer readable instruction is executed by the processor, implements the steps of the vertical federated learning optimization method as described above.
  • the present application also provides a storage medium having computer-readable instructions stored on the storage medium, and when the computer-readable instructions are executed by a processor, the above-mentioned vertical federated learning optimization method is implemented. step.
  • FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in a solution of an embodiment of the present application
  • FIG. 2 is a schematic flowchart of the first embodiment of the vertical federated learning optimization method according to this application;
  • FIG. 3 is a schematic flowchart of another embodiment of the vertical federated learning optimization method of this application.
  • Figure 4 is a schematic diagram of the device modules of the vertical federated learning optimization device of the application.
  • Figure 5 is a schematic diagram of the calculation and interaction process of the vertical federated learning optimization method of this application.
  • FIG. 1 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.
  • the longitudinal federated learning optimization device in the embodiment of the present application may be a PC or a server device, on which a Java virtual machine runs.
  • the vertical federated learning optimization device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002.
  • the communication bus 1002 is used to implement connection and communication between these components.
  • the user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001. Those skilled in the art can understand that the structure of the device shown in FIG.
  • the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and computer readable instructions.
  • the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server;
  • the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client;
  • the processor 1001 can be used to call computer-readable instructions stored in the memory 1005, and perform operations in the following vertical federated learning optimization method.
  • FIG. 2 is a schematic flowchart of a first embodiment of a longitudinal federated learning optimization method according to this application, and the method includes:
  • Step S10 Obtain the encrypted value set of the data in the main participant, and obtain the new encrypted value set of the new data in the main participant based on a preset judgment condition;
  • Logistic regression is a binary classification method based on probability model.
  • Vertical federated learning means that different parties have different characteristic data, which is equivalent to dividing each complete data into multiple parts vertically. Each party hopes to implement the training of the logistic regression model while protecting the data privacy, so as to use the model The parameters predict and score the new data.
  • Party A holds data x A and maintains the corresponding model parameters w A
  • Party B holds x B , y B and owns and maintains the corresponding model parameters.
  • w B In order to realize the longitudinal federated logistic regression, this solution adopts the existing approximate solution to the logistic regression function, and carries out the second-order Taylor expansion at the zero point, namely: After approximation, its gradient is:
  • the loss function and gradient can be expressed as the operation of the homomorphic encrypted data of both parties, namely:
  • This scheme uses the second-order information to propose a fast-convergent technical scheme.
  • the second-order derivative matrix of the approximate loss function ie, the Hessian matrix
  • the quasi-Newton method is used to estimate an inverse Hessian matrix H using second-order information, and the gradient g is not used in the algorithm but H g is used as the descending direction to speed up the convergence speed of the algorithm. Since the dimension of the inverse Hessian matrix H is much larger than the gradient, the core point of the design is how to reduce the data communication volume of all parties.
  • This scheme proposes to maintain the inverse Hessian matrix H at the C end, and in addition to calculating the gradient for each L step AB, a small batch of data is randomly selected, and the average value of the previous L step model is calculated Average with the last L-step model Difference Then calculate a vector containing the second-order information of the batch of data Sent to the C end, the dimension is the same as the gradient.
  • the C terminal uses the information of the first M vectors v to update the inverse Hessian matrix once. Therefore, in this embodiment, the main participant is taken as party A, and the sub-participant is taken as party B.
  • the logistic regression score of data on party A can be calculated by the formula For calculation, it represents the logistic regression score of each sample on side A, where w represents the linear model parameter of the logistic regression, x represents the data, y represents the data label, [[ ⁇ ]] represents the homomorphic encryption operation, and randomly selects one
  • the value set u A corresponding to the data ID in S is calculated in the main participant (ie party A), And use homomorphic encryption to calculate all u A ,
  • the value is encrypted, and the encrypted value set [[u A ]] is obtained, And transmit it to party B, and then update And determine whether the current iteration number of the longitudinal logistic regression model corresponding to the main participant is greater than the preset number (that is, determine whether the longitudinal logistic regression model meets the preset judgment condition).
  • the new encrypted value set is obtained, and the new encrypted value set and the encrypted value set are sent to the secondary participant as the target value set.
  • the A side only updates And the main participant only sends the encrypted value set to the secondary participant.
  • the initialization operation is also required, that is, the model parameters w A , w B on both sides of A and B are initialized, and the all-zero vector of the same size And initialize the matrix H in the C terminal (ie the coordinator), which is used to estimate the memory length M of H, and update the iteration step interval L of H, and the all-zero vector whose dimension is the sum of the model parameters of A and B
  • Step S20 sending the encrypted value set and the new encrypted value set as a target value set to a secondary participant to obtain an intermediate result value fed back by the secondary participant based on the target value set;
  • the main participant sends the encrypted value set and the new encrypted value set as the target value set to the secondary participant, that is, after the secondary participant (that is, party B) receives the target value set sent by the main participant (party A), B Fang will also first calculate the value set u B corresponding to the data ID in S, And calculate the loss function value and encryption coefficient according to the target value set. That is, Party B uses the property of homomorphic encryption to calculate the encrypted loss (loss function) value, At the same time, the encrypted value of each corresponding data is calculated That is, the encryption coefficient and transmit it to the A terminal.
  • the B party ie, the deputy participant
  • the B party will return each encryption coefficient to the A party, and the two parties AB each use the property of homomorphic encryption to each [ [d]]
  • Step S30 Calculate the main encrypted data in the main participant according to the intermediate result value, and send the main encrypted data to the coordinator, where the coordinator is used to respond to the failure of the vertical logistic regression model to converge, Update the second-order derivative matrix according to the primary encrypted data and the secondary encrypted data sent by the secondary participant, and calculate the target primary gradient value according to the updated second-order derivative matrix;
  • Both parties A and B will transmit [[g A ]], [[g B ]] to party C (that is, the coordinator), and party B will pass the encrypted loss value to party C, that is, the coordinator has The loss function value and gradient value (main gradient value and sub gradient value) are received.
  • the longitudinal logistic regression model If the longitudinal logistic regression model has converged, it sends an iteration stop signal to A , B two parties (that is, the main participant and the sub-participant), and the main participant will stop obtaining the encrypted value set with the logistic regression score in the main participant according to the iteration stop signal.
  • Judgment that is, to determine whether the longitudinal logistic regression model converges, and if it converges, it sends an iterative stop signal to party A and B, and stops the training of the longitudinal logistic regression model. If it does not converge, execute again Until the longitudinal logistic regression model converges.
  • the two gradients are merged into a long vector g, the product of the step length, H and g is calculated, and split into the corresponding A and B parts (that is, the target main gradient corresponding to the A side Value and the target sub-gradient value corresponding to party B) are transmitted to A and B respectively, namely: Further, if k is not an integer multiple of L at the same time, then end step 4; if k is an integer multiple of L, then C has also received the encrypted data [[v A ]], [[v B ]], decrypt it and merge it Can get And stored in a v queue of length M.
  • Step S40 Receive the target main gradient value fed back by the coordinator, update the local model parameters in the main participant based on the target main gradient value, and continue to execute the encrypted value set of the acquired data in the main participant Until the longitudinal logistic regression model corresponding to the coordinator converges.
  • the main participant receives the target main gradient value fed back by the coordinator, and updates the local model parameters in the main participant according to this target main gradient value.
  • the sub-participant also receives the target sub-gradient value corresponding to the sub-participant fed back by the coordinator To update the local model parameters in the sub-participants.
  • continue model training that is, randomly obtain new data again, and calculate its encrypted value set through the corresponding algorithm, until the coordinator sends the vertical logistic regression model to converge, and sends the iteration stop signal to the main participant and the secondary participant.
  • an example is given below.
  • Party A calculates and transmits data locally to Party B, that is, encrypting [[u A ]], It is transmitted to Party B.
  • Party B performs local calculations based on the encrypted data transmitted by Party A to obtain the encryption loss function value and encryption value, and transmits [[d]][[h]] to Party A, and both parties AB also encrypt Calculate the respective gradient values and transmit them to the C party, that is, the two parties AB send [[g A ]],[[g B ]][[v A ]],[[v B ]] to C Party B also transmits [[loss]] to Party C.
  • Party C decrypts the received [[v A ]], [[v B ]], [[loss]] to obtain the decrypted g A , g B , loss ⁇ A , ⁇ B , and judge whether the algorithm has converged according to loss, if not, update H according to the received gradient value, calculate and transmit, that is, when k is not greater than 2L, calculate the pre-selection The product of a fixed step size and the gradient And transmit them to A and B respectively; when k is greater than 2L, merge the two gradients into a long vector g, calculate the product of step length, H and g, and split them into corresponding parts A and B respectively Transmitted to A and B, namely: And both parties AB update the local model parameters according to the unencrypted vector passed by party C, namely:
  • the encrypted value set of the data in the main participant is obtained, and the new encrypted value set of the new data in the main participant is obtained based on a preset judgment condition; the encrypted value set and the The new encrypted value set is sent to the secondary participant as a target value set to obtain the intermediate result value fed back by the secondary participant based on the target value set; the primary encrypted data in the primary participant is calculated according to the intermediate result value , And send the main encrypted data to the coordinator, where the coordinator is used to update the second order according to the main encrypted data and the secondary encrypted data sent by the secondary participant in response to the longitudinal logistic regression model not converging Derivative matrix, and calculate the target main gradient value according to the updated second-order derivative matrix, and send the target main gradient value to the main participant; receive the target main gradient value fed back by the coordinator, based on the The target main gradient value updates the local model parameters in the main participant, and continues to execute the step of obtaining the encrypted value set of the data in the main participant until the longitudinal logistic regression model corresponding to the coordinator converges.
  • the sending volume of the target value set is increased, and the intermediate results fed back by the sub-participants are increased.
  • the main encrypted data is calculated based on the value, and then the main encrypted data is sent to the coordinator, and the local model parameters in the main participant are updated according to the target main gradient value fed back by the coordinator, and because the coordinator calculates the target main
  • the gradient value avoids the phenomenon that the prior art adopts a first-order algorithm for longitudinal federated learning, which makes the convergence speed slow and requires a large number of rounds of data interaction, and reduces the amount of communication for longitudinal federated learning.
  • a second embodiment of the vertical federated learning optimization method of the present application is proposed.
  • This embodiment is a refinement of the step S40 of the first embodiment of the present application.
  • the step of receiving the target main gradient value fed back by the coordinator includes:
  • Step a receiving the target main gradient value fed back by the coordinator, wherein the target main gradient value is updated by the coordinator according to the second derivative updated in response to the longitudinal logistic regression model satisfying the preset judgment condition Obtained by the matrix.
  • the target main gradient value is when the coordinator determines that the longitudinal logistic regression model does not converge and satisfies
  • the judgment condition is preset, it is calculated according to the updated second derivative matrix, and it is judged whether the longitudinal logistic regression model meets the preset judgment condition, for example, it is judged whether the new iteration number of the longitudinal logistic regression model meets the preset number condition ( For example, it is determined whether the new iteration number is an integer multiple of the iteration step interval, and whether it is greater than twice or greater than the preset number). And perform different operations according to different judgment results.
  • the accuracy of the obtained target main gradient value is guaranteed.
  • step of receiving the target main gradient value fed back by the coordinator includes:
  • Step b receiving the target main gradient value fed back by the coordinator, wherein the target main gradient value is obtained by the second derivative matrix updated by the coordinator according to the target data, and the target data is in response to longitudinal logistic regression
  • the model does not converge and meets a preset judgment condition, and is obtained by decrypting and combining the primary encrypted data and the secondary encrypted data sent by the secondary participant.
  • the target main gradient value is when the coordinator determines that the longitudinal logistic regression model does not converge and satisfies
  • the second derivative matrix is updated according to the target data, and calculated according to the updated second derivative matrix, where the target data does not converge in the longitudinal logistic regression model and meets the preset judgment
  • it is obtained by decrypting and combining the main encrypted data sent by the main participant and the secondary encrypted data sent by the sub-participants.
  • the obtained target main gradient is guaranteed The accuracy of the value.
  • step of receiving the target main gradient value fed back by the coordinator includes:
  • Step c Receive the target main gradient value fed back by the coordinator, where the target main gradient value is obtained by splitting the first target product by the coordinator, and the first target product is based on the response to the The second-order derivative matrix updated by the longitudinal logistic regression model to satisfy the preset judgment condition, the long vector of the combination of the main gradient value sent by the main participant and the auxiliary gradient value sent by the auxiliary participant, and the preset step The product between the lengths.
  • the main participant When the main participant receives the target main gradient value fed back by the coordinator, it can update its own local model parameters according to the target main gradient value, where the target main gradient value is obtained by the coordinator splitting the first target product , And the first target product is when the longitudinal logistic regression model does not converge and meets the preset judgment conditions, according to the updated second-order derivative matrix, the main gradient value sent by the main participant and the sub-gradient value sent by the sub-participant combined long The product of the vector and the preset step size.
  • the target main gradient value is obtained by the coordinator splitting the first target product, and the first target product is the product of the long vector, the preset step size and the updated second derivative matrix, thus The accuracy of the obtained target main gradient value is guaranteed.
  • step of receiving the target main gradient value fed back by the coordinator includes:
  • Step d receiving the target main gradient value fed back by the coordinator, wherein the target main gradient value is a second target product, and the second target product is that the coordinator does not converge in response to the longitudinal logistic regression model, and The product of the calculated main gradient value sent by the main participant and the preset step length is not satisfied.
  • the main participant When the main participant receives the target main gradient value fed back by the coordinator, it can update its own local model parameters according to the target main gradient value.
  • the target main gradient value is the second product, and the second product is the longitudinal logic of the coordinator.
  • the regression model does not converge and does not meet the preset judgment conditions, the main gradient value sent by the main participant and the preset step length are calculated to obtain its product.
  • This product is the second product, which is the target main The gradient value.
  • the product of the main gradient value and the preset step length is calculated, thereby ensuring the obtained target The accuracy of the main gradient value.
  • a third embodiment of the longitudinal federated learning optimization method of the present application is proposed.
  • This embodiment is step S10 of the first embodiment of the present application.
  • the step of acquiring the new encrypted value set of new data in the main participant based on a preset judgment condition includes: step e, detecting the corresponding data of the main participant Whether the longitudinal logistic regression model meets the preset judgment conditions;
  • the main participant After the main participant obtains the encrypted value set, it is also necessary to check whether the longitudinal logistic regression model corresponding to the main participant meets the preset judgment condition, for example, to determine whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining the new Whether the number of iterations is an integer multiple of the interval of iteration steps, and whether it is greater than two times greater than the preset number). Perform different operations according to different judgment results.
  • step f if it is satisfied, the new encrypted value set of the new data in the main participant is obtained according to the preset logistic regression algorithm.
  • the logistic regression score of the new data in the main participant is calculated through a preset logistic regression algorithm, and the new data is encrypted to obtain a new encrypted value set.
  • the encrypted value set can be directly sent to the secondary participant as the target value set.
  • the new encrypted value set of the new data in the main participant is obtained, thereby ensuring the efficiency of model training.
  • the refinement of the steps of obtaining the encrypted value set of the data in the main participant includes:
  • Step g Obtain a preset logistic regression algorithm in the main participant, and calculate the logistic regression score of the data in the main participant according to the preset logistic regression algorithm, and encrypt the logistic regression score to obtain encryption Numerical collection.
  • model parameters of the main participant's own model that is, the local model parameters
  • randomly select a small batch of training data to be trained for model training Then get the preset logistic regression algorithm in the main participant, such as That is, it represents the logistic regression score of each sample on the A side, where w represents the linear model parameter of the logistic regression, x represents the data, and T represents the quantity.
  • Each training data is calculated according to the preset logistic regression algorithm and local model parameters to obtain the logistic regression score corresponding to each training data in the main participant, and all logistic regression scores are encrypted to obtain an encrypted value set.
  • each training data is calculated according to the preset logistic regression formula and the local model parameters to obtain the encrypted value set, thereby ensuring the accuracy of the obtained encrypted value set.
  • Figure 3 is a schematic flowchart of another embodiment of the vertical federated learning optimization method of this application, including: step S100, receiving the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participant, so The main encrypted data is calculated according to the intermediate result value of the secondary participant based on the feedback of the target value set sent by the main participant, the target value set includes the encrypted value set corresponding to the data and the new encrypted value set corresponding to the new data ;
  • the coordinator when it is determined that the longitudinal logistic regression model has not converged according to the loss function value sent by the secondary participant and meets the preset judgment condition, for example, it is judged whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining the new number of iterations). Whether the number of iterations is an integer multiple of the iteration step interval, and whether it is greater than twice the preset number), if the preset number condition is met, it is determined that the longitudinal logistic regression model satisfies the preset judgment condition, when the main participant sends After the primary encrypted data and the secondary encrypted data sent by the secondary participant, the second derivative matrix is updated according to the primary encrypted data and the secondary encrypted data.
  • the preset number condition such as determining the new number of iterations
  • the primary encrypted data is calculated by the secondary participant based on the intermediate result value of the target value set sent by the primary participant, that is, the primary participant sends the encrypted value set to the secondary participant, and the secondary participant calculates it based on the encrypted value set.
  • the intermediate result value and the loss function value, and the loss function value is sent to the coordinator, the secondary encrypted data is calculated based on the intermediate result value, and the secondary encrypted data is sent to the coordinator, and the intermediate result value is fed back to the main participant. Participants perform calculations based on this intermediate result value to obtain the master encrypted data, and send the master encrypted data to the coordinator.
  • the target value set may include the encrypted value set corresponding to the data and the new encrypted value set corresponding to the new data, that is, whether the current iteration number corresponding to the main participant meets the preset condition (such as whether the current iteration number has passed the preset number) If it is not satisfied, the encrypted value set is taken as the target value set; if it is satisfied, the encrypted value set and the new encrypted value set are taken as the target value set.
  • the method of data encryption in this application can be homomorphic encryption.
  • Step S200 in response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the secondary encrypted data, and calculate a target main gradient value according to the updated second derivative matrix;
  • the coordinator When the coordinator detects that the longitudinal logistic regression model has not converged, it can update the second derivative matrix based on the main encrypted data sent by the main participant and the auxiliary encrypted data sent by the sub-participants, that is, the main encrypted data and the sub-encrypted data Decrypt and merge them and store them in a queue with a preset length to obtain the target queue, and update the second derivative matrix H according to the target queue.
  • the method of calculating H is to initialize with the value at the end of the memory queue, that is, to calculate H ⁇ p[m]I, where I is the identity matrix.
  • Judgment that is, to determine whether the longitudinal logistic regression model converges, and if it converges, the iterative stop signal is sent to the A and B parties, and the training of the longitudinal logistic regression model is stopped. If it does not converge, execute again Until the longitudinal logistic regression model converges.
  • k is greater than 2L
  • the two gradients are merged into a long vector g
  • the product of the step length, H and g is calculated, and split into the corresponding A and B parts (that is, the target main gradient corresponding to the A side Value and the target sub-gradient value corresponding to party B) are transmitted to A and B respectively, namely:
  • Step S300 Send the target master gradient value to the master participant, and the master participant is used to update the local model parameters in the master participant based on the target master gradient value, and continue to perform the acquisition The step of collecting the encrypted values of the data in the main participant until the longitudinal logistic regression model corresponding to the coordinator converges.
  • the coordinator After the coordinator calculates the target main gradient value, it will send the target main gradient value to the main participant, and the main participant will update the local model parameters in the main participant according to the target main gradient value, and continue to perform the acquisition
  • the sub-participants also receive the target sub-gradient values corresponding to the sub-participants fed back by the coordinator to update the local model parameters in the sub-participants.
  • the coordinator updates the second-order derivative matrix according to the main encrypted data and the auxiliary encrypted data, and calculates the target main gradient value according to the updated second-order derivative matrix, and sends the target main gradient value to the main participant to update
  • the local model parameters in the main participant thus avoiding the phenomenon that the prior art adopts the first-order algorithm for longitudinal federated learning, which makes the convergence speed slow, requires a large number of rounds of data interaction, and reduces the communication for longitudinal federated learning. the amount.
  • step of updating the second derivative matrix according to the primary encrypted data and the secondary encrypted data includes step x of judging whether the longitudinal logistic regression model satisfies the preset judgment condition;
  • the coordinator After the coordinator receives the main gradient value sent by the main participant and the secondary gradient value and loss value sent by the deputy coordinator, and determines that the longitudinal logistic regression model does not converge, it needs to determine whether the longitudinal logistic regression model meets the preset determination conditions, for example Determine whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining whether the new iteration number is an integer multiple of the iteration step interval, and whether it is more than twice greater than the preset number). And perform different operations according to different judgment results.
  • the preset determination conditions for example Determine whether the new iteration number of the longitudinal logistic regression model meets the preset number condition (such as determining whether the new iteration number is an integer multiple of the iteration step interval, and whether it is more than twice greater than the preset number).
  • Step y if it is satisfied, update the second derivative matrix according to the primary encrypted data and the secondary encrypted data.
  • the longitudinal logistic regression model when the longitudinal logistic regression model does not converge, it is judged whether the longitudinal logistic regression model satisfies the preset judgment condition, and only when it is satisfied, the second derivative matrix is updated, thereby ensuring the model training of the longitudinal logistic regression model. effectiveness.
  • step of updating a second derivative matrix according to the primary encrypted data and the secondary encrypted data includes: step z, the coordinator decrypts and merges the primary encrypted data and the secondary encrypted data to obtain Target data
  • the coordinator After the coordinator receives the main encrypted data sent by the main participant and the secondary encrypted data sent by the sub-participants, it decrypts and merges to obtain the target data, that is, the encrypted data [[v A ]], [[v B ]] Get the target data after decryption
  • Step m Store the target data in a queue with a preset length to obtain the target queue, and update the second derivative matrix through the target queue.
  • the coordinator stores the target data in a v queue with a length of M (ie, a preset length). At the same time, calculate the current (t) and the last (t-1) Difference between Store it in the s queue of length M. If the current memory has reached the maximum storage length M, delete the first one in the queue and put the latest v and s at the end of the queue. Use m (m not greater than M) v and s in the current memory to calculate H (second derivative matrix). The calculation method is as follows:
  • the target data is obtained by decrypting and combining the primary encrypted data and the secondary encrypted data, and then the second derivative matrix is updated according to the target data, thereby ensuring the effectiveness of the update of the second derivative matrix.
  • step n if it is not met, the coordinator obtains the main gradient value and the preset value sent by the main participant And send the first product as the target main gradient value to the main participant.
  • the coordinator calculates the first product of the pre-selected preset step size and the main gradient value, and the preset step size and the sub-gradient corresponding to the sub-participants
  • the third product of the values, and the first product is sent to the main participant as the target main gradient value to update the local model parameters in the main participant, and the third product is sent to the secondary participants to update the model parameters in the secondary participant , And then re-train the model according to the updated model parameters to obtain the new loss function value, and send it to the coordinator through the deputy participant.
  • the first product between the main gradient value and the preset step size is calculated, and the first product is used as the target main gradient value, thereby The accuracy of the obtained target main gradient value is guaranteed.
  • the vertical federated learning optimization device includes: an acquisition module for acquiring an encrypted value set of data in the main participant, and based on a preset judgment condition Obtain a new encrypted value set of new data in the main participant; a sending module for sending the encrypted value set and the new encrypted value set as a target value set to the secondary participant to obtain the secondary participant The intermediate result value fed back based on the target value set; the first calculation module is used to calculate the main encrypted data in the main participant according to the intermediate result value, and send the main encrypted data to the coordinator, Wherein, the coordinator is used to update the second derivative matrix according to the primary encrypted data and the secondary encrypted data sent by the secondary participant in response to the longitudinal logistic regression model not converging, and according to the updated second-order derivative matrix The derivative matrix calculates the target main gradient value; the first update module is used to receive the target main gradient value fed back by the coordinator, update the local model parameters in the main participant based on the target main gradient value,
  • the first update module is further configured to: receive the target main gradient value fed back by the coordinator, wherein the target main gradient value is satisfied by the coordinator according to the response to the longitudinal logistic regression model.
  • the second derivative matrix updated according to the preset judgment conditions is obtained.
  • the first update module is further configured to: receive the target main gradient value fed back by the coordinator, wherein the target main gradient value is obtained by the second derivative matrix updated by the coordinator according to the target data
  • the target data is obtained by decrypting and merging the primary encrypted data and the secondary encrypted data sent by the secondary participant in response to the failure of the longitudinal logistic regression model to converge and meet a preset determination condition.
  • the first update module is further configured to: receive the target main gradient value fed back by the coordinator, wherein the target main gradient value is obtained by splitting the first target product by the coordinator, The first target product is based on the second derivative matrix updated in response to the longitudinal logistic regression model meeting the preset determination condition, the primary gradient value sent by the primary participant, and the secondary derivative sent by the secondary participant.
  • the product of the long vector of the merged gradient values and the preset step size is further configured to: receive the target main gradient value fed back by the coordinator, wherein the target main gradient value is obtained by splitting the first target product by the coordinator, The first target product is based on the second derivative matrix updated in response to the longitudinal logistic regression model meeting the preset determination condition, the primary gradient value sent by the primary participant, and the secondary derivative sent by the secondary participant.
  • the product of the long vector of the merged gradient values and the preset step size is further configured to: receive the target main gradient value fed back by the coordinator, wherein the target main gradient value is obtained by splitting the first target product by the coordinator, The first target product is
  • the first update module is further configured to: receive a target main gradient value fed back by the coordinator, wherein the target main gradient value is a second target product, and the second target product is the coordination In response to that the longitudinal logistic regression model does not converge and does not meet the preset determination condition, the calculated product between the main gradient value sent by the main participant and the preset step length.
  • the acquisition module is further configured to: detect whether the longitudinal logistic regression model corresponding to the main participant meets a preset judgment condition; if so, obtain new data in the main participant according to the preset logistic regression algorithm. The new set of encrypted values among the participants.
  • the acquiring module is further configured to: acquire a preset logistic regression algorithm in the main participant, and calculate the logistic regression score of the data in the main participant according to the preset logistic regression algorithm, and compare the The logistic regression score is encrypted to obtain a collection of encrypted values.
  • An embodiment of the present application also provides a vertical federated learning optimization device, the vertical federated learning optimization device includes: a receiving module for receiving the primary encrypted data sent by the primary participant and the secondary encrypted data sent by the secondary participants, the primary The encrypted data is calculated according to the intermediate result value of the secondary participant based on the feedback of the target value set sent by the main participant, the target value set includes the encrypted value set corresponding to the data and the new encrypted value set corresponding to the new data; A second calculation module, in response to the failure of the longitudinal logistic regression model to converge, update a second derivative matrix according to the main encrypted data and the secondary encrypted data, and calculate a target main gradient value according to the updated second derivative matrix;
  • the second update module is configured to send the target main gradient value to the main participant respectively, and the main participant is used to update the local model parameters in the main participant based on the target main gradient value, and Continue to perform the step of obtaining the encrypted value set of the data in the main participant until the longitudinal logistic regression model corresponding to the coordinator converges.
  • the second calculation module is further configured to: determine whether the longitudinal logistic regression model satisfies the preset determination condition; if so, update the second derivative according to the primary encrypted data and the secondary encrypted data matrix.
  • the second calculation module is further configured to: the coordinator decrypts and merges the primary encrypted data and the secondary encrypted data to obtain target data; and stores the target data to a preset length In the queue, the target queue is obtained, and the second derivative matrix is updated through the target queue.
  • the second calculation module is further configured to: if it is not satisfied, the coordinator obtains the first product between the main gradient value sent by the main participant and the preset step length, and compares the result The first product is sent to the main participant as a target main gradient value.
  • the embodiment of the present application also provides a storage medium, and the storage medium may be a non-volatile readable storage medium.
  • the storage medium of the present application stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the vertical federated learning optimization method described above are realized.
  • the method implemented when the computer-readable instructions running on the processor are executed please refer to the respective embodiments of the vertical federated learning optimization method of this application, which will not be repeated here.
  • the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disks, optical disks), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.
  • a terminal device which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Theoretical Computer Science (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Operations Research (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Storage Device Security (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种纵向联邦学习优化方法、装置、设备和存储介质，涉及金融科技技术领域，该方法包括：获取数据在主参与者中的加密数值集合，基于预设判定条件获取新数据在主参与者中的新加密数值集合；将加密数值集合和新加密数值集合作为目标数值集合发送至副参与者，以获取副参与者基于目标数值集合反馈的中间结果值；根据中间结果值计算主参与者中的主加密数据，并将所述主加密数据发送至协调者；接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。

Description

纵向联邦学习优化方法、装置、设备及存储介质 技术领域
本申请涉及金融科技(Fintech)技术领域，尤其涉及纵向联邦学习优化方法、装置、设备及存储介质。
背景技术
随着计算机技术的发展，越来越多的技术(大数据、分布式、区块链Blockchain、人工智能等)应用在金融领域，传统金融业正在逐步向金融科技(Fintech)转变，但由于金融行业的安全性、实时性要求，也对技术提出了更高的要求。例如，联邦学习的纵向逻辑回归方法，但现有的纵向逻辑回归方案是基于一阶梯度信息的随机梯度下降方法。该方案的原理是先将逻辑回归的损失(loss)函数用在零点的二阶泰勒展开，然后该近似损失函数值和梯度值可以通过AB两方的数据联邦得到，利用同态加密将计算中间结果加密，由C方进行解密后传输回AB两方进行模型参数更新。该方案采用的路线为随机梯度下降方法，即随机选取小批数据，AB两方通过计算、加密和数据交互得到各自参数分量的加密梯度值，发送给C方解密后乘以相应的步长即得到下降方向。随机梯度下降为一阶算法，收敛速度较慢，需要进行大量轮次的数据交互，使得目前纵向联邦学习所需要的通信量高，耗费网络带宽大，计算成本高。
发明内容
本申请的主要目的在于提出一种纵向联邦学习优化方法、装置、设备及存储介质，旨在解决目前进行纵向联邦学习的通信量较高的技术问题。
为实现上述目的，本申请提供一种纵向联邦学习优化方法，所述纵向联邦学习优化方法包括如下步骤：
获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
此外，本申请提供一种纵向联邦学习优化方法包括如下步骤：
接收主参与者发送的主加密数据和副参与者发送的副加密数据，所述主 加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算得，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
将所述目标主梯度值分别发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
此外，为实现上述目的，本申请还提供一种纵向联邦学习优化装置，所述纵向联邦学习优化装置包括：
获取模块，用于获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
发送模块，用于将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
第一计算模块，用于根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
第一更新模块，用于接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
可选地，所述纵向联邦学习优化装置包括：
接收模块，用于接收主参与者发送的主加密数据和副参与者发送的副加密数据，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算得，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
第二计算模块，用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
第二更新模块，用于将所述目标主梯度值分别发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
此外，为实现上述目的，本申请还提供一种纵向联邦学习优化设备，所述纵向联邦学习优化设备包括：存储器、处理器及存储在所述存储器上并可 在所述处理器上运行的计算机可读指令，所述计算机可读指令被所述处理器执行时实现如上所述的纵向联邦学习优化方法的步骤。
此外，为实现上述目的，本申请还提供一种存储介质，所述存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如上所述的纵向联邦学习优化方法的步骤。
附图说明
图1是本申请实施例方案涉及的硬件运行环境的设备结构示意图；
图2为本申请纵向联邦学习优化方法第一实施例的流程示意图；
图3为本申请纵向联邦学习优化方法另一实施例的流程示意图；
图4为本申请纵向联邦学习优化装置的装置模块示意图；
图5为本申请纵向联邦学习优化方法计算与交互的流程示意图。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。如图1所示，图1是本申请实施例方案涉及的硬件运行环境的设备结构示意图。本申请实施例纵向联邦学习优化设备可以是PC机或服务器设备，其上运行有Java虚拟机。如图1所示，该纵向联邦学习优化设备可以包括：处理器1001，例如CPU，网络接口1004，用户接口1003，存储器1005，通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)，可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。本领域技术人员可以理解，图1中示出的设备结构并不构成对设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。如图1所示，作为一种存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及计算机可读指令。在图1所示的设备中，网络接口1004主要用于连接后台服务器，与后台服务器进行数据通信；用户接口1003主要用于连接客户端(用户端)，与客户端进行数据通信；而处理器1001可以用于调用存储器1005中存储的计算机可读指令，并执行下述纵向联邦学习优化方法中的操作。
基于上述硬件结构，提出本申请纵向联邦学习优化方法实施例。参照图2，图2为本申请纵向联邦学习优化方法第一实施例的流程示意图，所述方法包括：
步骤S10，获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
主参与者拥有数据的部分特征，并且在本实施例中仅以主参与者作为A方，副参与者作为B方，协调者作为C方进行解释说明。逻辑回归是一种基于概率模型的二分类方法。逻辑回归的模型训练是在给定数据特征与标签(x i,y i)下通过最小化损失函数l(w)＝log(1+exp(-yw Tx))来得到模型参数w的过程。纵向联邦 学习是指不同方各自拥有不同的特征数据，相当于把每一条完整的数据纵向切分成了多个部分，各方希望在保护数据隐私的情况下实现逻辑回归模型的训练，从而利用模型参数对新数据进行预测打分。由于逻辑回归损失函数l(w)＝log(1+exp(-yw Tx))的复杂性(为了简化符号，省略下标i)。现有加密计算方案无法在加密状态下计算指数函数和对数函数，在本实施例中是采用纵向联邦逻辑回归中的现有加密计算方案，使用满足加法同态的加密方法，即[[ax]]＝a[[x]],[[x]]+[[y]]＝[[x+y]]。这里[[·]]代表同态加密操作。纵向联邦场景中只有一方持有数据标签，以两方为例，A方持有数据x A，维护对应的模型参数w A，B方持有x B,y B，拥有并维护对应的模型参数w B。为了实现纵向联邦逻辑回归，本方案采用现有的对逻辑回归函数的近似方案，在零点进行二阶泰勒展开，即：
Figure PCTCN2019119409-appb-000001
近似后其梯度为：
Figure PCTCN2019119409-appb-000002
损失函数和梯度可以表示为双方同态加密数据的运算，即：
Figure PCTCN2019119409-appb-000003
Figure PCTCN2019119409-appb-000004
Figure PCTCN2019119409-appb-000005
本方案利用二阶信息提出了一种快速收敛的技术方案，近似损失函数的二阶导数矩阵(即海森矩阵)为
Figure PCTCN2019119409-appb-000006
在本实施例中是利用拟牛顿法，利用二阶信息估计一个逆海森矩阵H，在算法中不用梯度g而采用H g作为下降方向，以此来加快算法收敛速度。由于逆海森矩阵H的维度比梯度要大很多，设计的核心要点是如何降低各方的数据通信量。本方案提出在C端维护逆海森矩阵H，每L步AB方除了计算梯度以外额外随机选择一小批数据，计算出前L步模型的平均值
Figure PCTCN2019119409-appb-000007
与上一个L步模型的平均值
Figure PCTCN2019119409-appb-000008
之差
Figure PCTCN2019119409-appb-000009
然后计算出一个包含了该批数据二阶信息的向量
Figure PCTCN2019119409-appb-000010
发送给C端，其维度同梯度相同。C端利用前M个向量v的信息更新逆海森矩阵一次。因此，可以在本实施例中，将主参与者作为A方，将副参与者作为B方进行阐述，计算数据在A方的逻辑回归得分可以通过公式
Figure PCTCN2019119409-appb-000011
来进行计算，即表示A侧各样本的逻辑回归得分，其中，w表示逻辑回归的线性模型参数，x代表数据，y代表数据标签，[[·]]代表同态加密操作，在随机选取一小批数据ID为S后，在主参与者(即A方)中计算出对应数据ID在S里的数值集合u A,
Figure PCTCN2019119409-appb-000012
并采用同态加密计算对所有u A,
Figure PCTCN2019119409-appb-000013
值进行加密，得到加密数值集合[[u A]]，
Figure PCTCN2019119409-appb-000014
并将其传输给B方，然后更新
Figure PCTCN2019119409-appb-000015
并确定主参与者对应的纵向逻辑回归模型的当前迭代次数是否大于预设次数(也就是确定纵向逻辑回归模型是否满足预设判定条件)。在确定当前迭代次数是否大于预设次数时，需要先确定当前迭代次数k与L的关系，并在当前迭代次数k是L的整数倍时，再判断迭代次 数是否大于2L，此时2L就是预设次数，当经过判断发现迭代次数k大于2L时，A端(即主参与者)令
Figure PCTCN2019119409-appb-000016
也就是计算
Figure PCTCN2019119409-appb-000017
的平均值，并计算本次(t)与上次(t-1)的
Figure PCTCN2019119409-appb-000018
之差，即
Figure PCTCN2019119409-appb-000019
另外随机选择一小批数据ID为S H，A端计算出S H上的
Figure PCTCN2019119409-appb-000020
并将同态加密后的数据
Figure PCTCN2019119409-appb-000021
传输给B端。也就是获取新加密数值集合，并将新加密数值集合和加密数值集合一起作为目标数值集合发送到副参与者。但是当迭代次数k不大于2L时，A端仅更新
Figure PCTCN2019119409-appb-000022
并且主参与者只发送加密数值集合到副参与者。其中，在进行纵向联邦学习优化之前，还需要先进行初始化操作，也就是A、B两侧初始化模型参数w A,w B，以及同一大小的全零向量
Figure PCTCN2019119409-appb-000023
并在C端(即协调者)中初始化矩阵H，用于估计H的记忆长度M，以及更新H的迭代步数间隔L，以及维度为A与B模型参数之和的全零向量
Figure PCTCN2019119409-appb-000024
步骤S20，将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
主参与者将加密数值集合和新加密数值集合作为目标数值集合发送至副参与者，也就是在副参与者(即B方)接收到主参与者(A方)发送的目标数值集合后，B方也会同样先计算出对应数据ID在S里的数值集合u B,
Figure PCTCN2019119409-appb-000025
并根据目标数值集合计算出损失函数值和加密系数。即B方利用同态加密的性质，计算出加密的loss(损失函数)值，
Figure PCTCN2019119409-appb-000026
同时计算得到每条对应数据的加密值
Figure PCTCN2019119409-appb-000027
也就是加密系数，并将其传输给A端。然后更新
Figure PCTCN2019119409-appb-000028
并再次判断当前迭代次数k与L的关系，如果当前迭代次数k是L的整数倍，且迭代次数k大于2L：B端更新
Figure PCTCN2019119409-appb-000029
并计算本次(t)与上次(t-1)的
Figure PCTCN2019119409-appb-000030
之差，即
Figure PCTCN2019119409-appb-000031
另外B方计算出S H上的
Figure PCTCN2019119409-appb-000032
从而计算出中间结果值
Figure PCTCN2019119409-appb-000033
并传输给A方。如果当前迭代次数k是L的整数倍，且迭代次数k不大于2L；B端则仅更新
Figure PCTCN2019119409-appb-000034
并且，当通过B方(即副参与者)计算得到每条数据对应的加密系数后，B方会将各个加密系数返回给A方，并且AB两方各自利用同态加密的性质对每个[[d]]值(即同态系数)乘以对应数据x A,x B，然后对得到的向量集合求和，计算出加密的梯度值[[g A]]＝∑[[d]]x A,[[g B]]＝∑[[d]]x B。
步骤S30，根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
AB两方会分别将[[g A]],[[g B]]传输给C方(即协调者)，并且B方会将加密的loss值传给C方，也就是此时协调者已接收到损失函数值、梯度值(主梯度值和副梯度值)。此时还是需要继续判断当前迭代次数k与L的关系，并在当前迭 代次数k是L的整数倍，且迭代次数k大于2L时，则A、B两端分别根据[[h]]计算出主加密数据和副加密数据，即[[υ A]]＝∑[[h]]x A,[[υ B]]＝∑[[h]]x B，并传输给C方，也就是协调者。C方对收到的数据进行解密，得到g A,g B,loss，并根据loss(损失函数值)判断纵向逻辑回归模型是否收敛，若纵向逻辑回归模型已收敛，则发送迭代停止信号到A、B两方(即主参与者和副参与者)，并且主参与者会根据迭代停止信号停止获取主参与者中具有逻辑回归得分的加密数值集合。若纵向逻辑回归模型未收敛，则更新
Figure PCTCN2019119409-appb-000035
并需要判断当前迭代次数k与迭代步数间隔L的关系，如果不大于2L，则计算出提前设置的步长与梯度的乘积
Figure PCTCN2019119409-appb-000036
并将各自的乘积分别发送到各自对应的A方和B方中，再让A方根据获取到的乘积(即目标主梯度值)更新A方本地的模型参数，并进行下一次的数据模型训练，同理也让B方根据获取到的乘积更新B方本地的模型参数，再进行下一次的模型训练，直到获取到新的损失函数值，并将其传递给到C方(协调者)进行判定，即确定纵向逻辑回归模型是否收敛，若收敛，则发送迭代停止信号给A方和B方，并停止纵向逻辑回归模型的训练。若未收敛，则再次执行
Figure PCTCN2019119409-appb-000037
的操作，直至纵向逻辑回归模型收敛。并且在k大于2L时，则将两个梯度合并成一个长向量g,计算出步长、H与g的乘积，并拆分成对应的A、B两部分(即A方对应的目标主梯度值和B方对应的目标副梯度值)分别传输给A和B，即：
Figure PCTCN2019119409-appb-000038
进一步，如果同时k不是L的整数倍则结束步骤4；如果k是L的整数倍，则C还收到了加密数据[[v A]],[[v B]],将其进行解密后合并可以得到
Figure PCTCN2019119409-appb-000039
并存储在一个长度为M的v队列中。同时，计算本次(t)与上次(t-1)的w～之差，即
Figure PCTCN2019119409-appb-000040
将其存在长度为M的s队列中。如果目前的存储器已经达到最大存储长度M，则将队列首个删掉并将最新得到v和s放在队列末尾。利用当前存储器里的m(m不大于M)个v和s，来计算H。其中，计算H的方法为利用存储器队列末尾的值初始化，即计算
Figure PCTCN2019119409-appb-000041
H←p[m]I，其中I为单位矩阵。然后从队列首到队列尾(j＝1,..,m)迭代计算得到更新的H：p[j]＝1/(v[j] Ts[j]),H←(I-p[j]s[j]v[j] T)H(I-p[j]v[j]s[j] T)+p[j]s[j]s[j] T。
步骤S40，接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
主参与者接收协调者反馈的目标主梯度值，并根据此目标主梯度值更新主参与者中的本地模型参数，同理副参与者也接收协调者反馈的副参与者对应的目标副梯度值以更新副参与者中的本地模型参数。并继续进行模型训练，也就是再次随机获取新数据，并通过相应的算法计算其加密数值集合，直至协调者发送纵向逻辑回归模型收敛，并发送迭代停止信号到主参与者和副参与者。另外，为辅助理解本实施例中进行纵向联邦学习优化的流程，下面进行举例说明。例如，如图5所示，存在有A、B、C三方进行模型训练，其中A 方为主参与者，B方为副参与者，C方为协调者。A方本地计算与传输数据给B方，也就是将加密的[[u A]]，
Figure PCTCN2019119409-appb-000042
传输给B方，B方根据A方传输的加密数据进行本地计算得到加密损失函数值和加密值，并将[[d]][[h]]传输给A方，并且AB两方同时根据加密值计算各自的梯度值，并将其传输给C方，也就是AB两方将[[g A]],[[g B]][[v A]],[[v B]]发送给C方，并且B方还将[[loss]]传输给C方，C方对接收到的[[v A]],[[v B]]，[[loss]]进行解密，得到解密后的g A,g B,lossμ A,μ B，并根据loss判断算法是否收敛，若未收敛，则根据接收到的梯度值更新H，并计算与传输，即在k不大于2L时，计算出事先选定的步长与梯度的乘积
Figure PCTCN2019119409-appb-000043
并将其分别传输给A和B；在k大于2L时，将两个梯度合并成一个长向量g,计算出步长、H与g的乘积，并拆分成对应的A、B两部分分别传输给A和B，即：
Figure PCTCN2019119409-appb-000044
并且AB两方根据C方传递的未加密的向量来更新本地的模型参数，即
Figure PCTCN2019119409-appb-000045
在本实施例中，通过获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值，将所述目标主梯度值发送至所述主参与者；接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。通过根据预设判定条件将主参与者中的加密数值集合和新加密数值集合作为目标数值集合发送至副参与者，从而提高了发送目标数值集合的发送量，并根据副参与者反馈的中间结果值计算主加密数据，再将主加密数据发送至协调者，根据协调者反馈的目标主梯度值对主参与者中的本地模型参数进行更新，并且由于协调者是根据二阶导数矩阵计算目标主梯度值，避免了现有技术进行纵向联邦学习采用一阶算法而使得收敛速度较慢，需要进行大量轮次的数据交互的现象发生，减少了进行纵向联邦学习的通信量。
进一步地，基于本申请纵向联邦学习优化方法第一实施例，提出本申请纵向联邦学习优化方法第二实施例。本实施例是本申请第一实施例的步骤S40，接收所述协调者反馈的目标主梯度值的步骤的细化，包括：
步骤a，接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵获取的。
在主参与者接收到协调者反馈的目标主梯度值时，可以根据此目标主梯 度值更新自身的本地模型参数，其中，目标主梯度值是协调者在确定纵向逻辑回归模型不收敛，且满足预设判定条件时，根据已进行更新的二阶导数矩阵进行计算获取的，而判断纵向逻辑回归模型是否满足预设判定条件，例如判断纵向逻辑回归模型的新迭代次数是否满足预设次数条件(如确定新迭代次数是否为迭代步数间隔的整数倍，且是否大于两倍大于预设次数)。并根据不同的判断结果执行不同的操作。
在本实施例中，通过确定目标主梯度值是由纵向逻辑回归模型满足预设判定条件而更新的二阶导数矩阵获取的，从而保障了获取的目标主梯度值的准确性。
进一步地，接收所述协调者反馈的目标主梯度值的步骤，包括：
步骤b，接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据目标数据更新的二阶导数矩阵获取的，所述目标数据为响应于纵向逻辑回归模型未收敛，且满足预设判定条件，将所述主加密数据和所述副参与者发送的副加密数据进行解密合并得到的。
在主参与者接收到协调者反馈的目标主梯度值时，可以根据此目标主梯度值更新自身的本地模型参数，其中，目标主梯度值是协调者在确定纵向逻辑回归模型不收敛，且满足预设判定条件时，根据目标数据对二阶导数矩阵进行更新，并根据已进行更新的二阶导数矩阵进行计算获取的，其中，目标数据是在纵向逻辑回归模型未收敛，且满足预设判定条件时，将主参与者发送的主加密数据和副参与者发送的副加密数据进行解密合并得到的。
在本实施例中，通过确定目标主梯度值是根据目标数据和更新的二阶导数矩阵获取的，并且目标数据是主加密数据和副加密数据进行合并得到的，从而保障了获取的目标主梯度值的准确性。
进一步地，接收所述协调者反馈的目标主梯度值的步骤，包括：
步骤c，接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者对第一目标乘积进行拆分得到的，所述第一目标乘积为根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵、所述主参与者发送的主梯度值和所述副参与者发送的副梯度值合并的长向量，和预设步长之间的乘积。
在主参与者接收到协调者反馈的目标主梯度值时，可以根据此目标主梯度值更新自身的本地模型参数，其中，目标主梯度值是由协调者对第一目标乘积进行拆分得到的，而第一目标乘积是纵向逻辑回归模型未收敛且满足预设判定条件时，根据已更新的二阶导数矩阵、主参与者发送的主梯度值和副参与者发送的副梯度值合并的长向量、预设的步长进行计算的乘积。
在本实施例中，通过确定目标主梯度值是协调者对第一目标乘积进行拆分得到的，而第一目标乘积是长向量、预设步长和更新的二阶导数矩阵的乘积，从而保障了获取到的目标主梯度值的准确性。
进一步地，接收所述协调者反馈的目标主梯度值的步骤，包括：
步骤d，接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值 为第二目标乘积，所述第二目标乘积为所述协调者响应于纵向逻辑回归模型未收敛，且不满足预设判定条件，计算的所述主参与者发送的主梯度值和预设的步长之间的乘积。
在主参与者接收到协调者反馈的目标主梯度值时，可以根据此目标主梯度值更新自身的本地模型参数，其中，目标主梯度值是第二乘积，第二乘积是协调者在纵向逻辑回归模型未收敛，且不满足预设判定条件时，对主参与者发送的主梯度值和预设的步长之间进行计算，以获取其乘积，该乘积就是第二乘积，也就是目标主梯度值。
在本实施例中，通过确定目标主梯度值时在纵向逻辑回归模型未收敛，且不满足预设判定条件时，计算主梯度值和预设的步长的乘积，从而保障了获取到的目标主梯度值的准确性。
进一步地，在第一至第二任意一个实施例的基础上，提出了本申请纵向联邦学习优化方法第三实施例。本实施例是本申请第一实施例的步骤S10，基于预设判定条件获取新数据在所述主参与者中的新加密数值集合的步骤，包括：步骤e，检测所述主参与者对应的纵向逻辑回归模型是否满足预设判定条件；
在主参与者获取到加密数值集合后，还需要检测主参与者对应的纵向逻辑回归模型是否满足预设判定条件，例如判断纵向逻辑回归模型的新迭代次数是否满足预设次数条件(如确定新迭代次数是否为迭代步数间隔的整数倍，且是否大于两倍大于预设次数)。根据不同的判断结果执行不同的操作。
步骤f，若满足，则根据所述预设逻辑回归算法获取新数据在所述主参与者中的新加密数值集合。
若满足，则随机获取新的数据，通过预设逻辑回归算法来计算新的数据在主参与者中的逻辑回归得分，对其进行加密，得到新加密数值集合。但是若不满足，则可以直接将加密数值集合作为目标数值集合发送到副参与者。
在本实施例中，通过检测纵向逻辑回归模型是否满足预设判定条件，若满足，则获取新数据在主参与者中的新加密数值集合，从而保障了模型训练的效率。
进一步地，获取数据在主参与者中的加密数值集合的步骤的细化，包括：
步骤g，获取主参与者中的预设逻辑回归算法，并根据所述预设逻辑回归算法计算数据在所述主参与者中的逻辑回归得分，对所述逻辑回归得分进行加密，以获取加密数值集合。
获取主参与者自身模型的模型参数，即本地模型参数，并随机选择一小批待进行模型训练的训练数据，以便进行模型训练。再获取主参与者中的预设逻辑回归算法，如
Figure PCTCN2019119409-appb-000046
即表示A侧各样本的逻辑回归得分，其中，w表示逻辑回归的线性模型参数，x代表数据，T表示数量。并根据预设逻辑回归算法和本地模型参数对各个训练数据进行计算，以获取主参与者中各个训练数据对应的逻辑回归得分，并对所有的逻辑回归得分进行加密，得到加密数值集合。
在本实施例中，通过根据预设逻辑回归公式和本地模型参数对各个训练数据进行计算，以获取加密数值集合，从而保障了获取到的加密数值集合的准确性。
进一步地，参照图3，图3为本申请纵向联邦学习优化方法另一实施例的流程示意图，包括：步骤S100，接收主参与者发送的主加密数据和副参与者发送的副加密数据，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算得，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
在协调者中当根据副参与者发送的损失函数值确定纵向逻辑回归模型未收敛，且满足预设判定条件时，例如判断纵向逻辑回归模型的新迭代次数是否满足预设次数条件(如确定新迭代次数是否为迭代步数间隔的整数倍，且是否大于两倍大于预设次数)，若满足预设次数条件，则确定纵向逻辑回归模型满足预设判定条件时，在接收到主参与者发送的主加密数据和副参与者发送的副加密数据后，根据主加密数据和副加密数据更新二阶导数矩阵。其中，主加密数据是副参与者基于主参与者发送的目标数值集合反馈的中间结果值计算得到的，也就是主参与者发送加密数值集合到副参与者，副参与者根据加密数值集合计算出中间结果值和损失函数值，并将损失函数值发送至协调者，根据中间结果值计算出副加密数据，并将副加密数据发送至协调者，同时将中间结果值反馈至主参与者，主参与者根据此中间结果值进行计算获取到主加密数据，将此主加密数据发送至协调者。其中，目标数值集合可以包括数据对应的加密数值集合和新数据对应的新加密数值集合，也就是在主参与者对应的当前迭代次数是否满足预设条件(如当前迭代次数是否经过预设次数)，若不满足，则将加密数值集合作为目标数值集合，若满足，则将加密数值集合和新加密数值集合作为目标数值集合。并且在本申请中数据加密的方式可以是采用同态加密的方式。
步骤S200，响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
在协调者检测到纵向逻辑回归模型未收敛时，可以根据主参与者发送的主加密数据和副参与者发送的副加密数据来更新二阶导数矩阵，也就是将主加密数据和副加密数据进行解密合并，并将其存储在一个预设长度的队列中，得到目标队列，并根据此目标队列来更新二阶导数矩阵H，其中，计算H的方法为利用存储器队列末尾的值初始化，即计算
Figure PCTCN2019119409-appb-000047
H←p[m]I，其中I为单位矩阵。然后从队列首到队列尾(j＝1,..,m)迭代计算得到更新的H：p[j]＝1/(v[j] Ts[j]),H←(I-p[j]s[j]v[j] T)H(I-p[j]v[j]s[j] T)+p[j]s[j]s[j] T。
并且在本实施例中，若纵向逻辑回归模型未收敛，则更新
Figure PCTCN2019119409-appb-000048
并需要判断当前迭代次数k与迭代步数间隔L的关系，如果不大于2L，则计算出提前设置的步长与梯度的乘积
Figure PCTCN2019119409-appb-000049
并将各自的乘积分别发送到各自对应的A方和B方中，再让A方根据获取到的乘积(即目标主梯度值)更 新A方本地的模型参数，并进行下一次的数据模型训练，同理也让B方根据获取到的乘积更新B方本地的模型参数，再进行下一次的模型训练，直到获取到新的损失函数值，并将其传递给到C方(协调者)进行判定，即确定纵向逻辑回归模型是否收敛，若收敛，则发送迭代停止信号给A方和B方，并停止纵向逻辑回归模型的训练。若未收敛，则再次执行
Figure PCTCN2019119409-appb-000050
的操作，直至纵向逻辑回归模型收敛。并且在k大于2L时，则将两个梯度合并成一个长向量g,计算出步长、H与g的乘积，并拆分成对应的A、B两部分(即A方对应的目标主梯度值和B方对应的目标副梯度值)分别传输给A和B，即：
Figure PCTCN2019119409-appb-000051
步骤S300，将所述目标主梯度值发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
在协调者计算得到目标主梯度值后，会将此目标主梯度值发送给主参与者，主参与者会根据此目标主梯度值来更新主参与者中的本地模型参数，并会继续执行获取数据在主参与者中的加密数值集合，直至协调者对应的纵向逻辑回归模型收敛，并发送迭代停止信号到主参与者和副参与者。同理副参与者也接收协调者反馈的副参与者对应的目标副梯度值以更新副参与者中的本地模型参数。
在本实施例中，通过协调者根据主加密数据和副加密数据更新二阶导数矩阵，并根据更新的二阶导数矩阵计算目标主梯度值，将目标主梯度值发送至主参与者，以更新主参与者中的本地模型参数，从而避免了现有技术进行纵向联邦学习采用一阶算法而使得收敛速度较慢，需要进行大量轮次的数据交互的现象发生，减少了进行纵向联邦学习的通信量。
进一步地，根据所述主加密数据和所述副加密数据更新二阶导数矩阵的步骤，包括步骤x，判断所述纵向逻辑回归模型是否满足所述预设判定条件；
在协调者接收到主参与者发送的主梯度值和副协调者发送的副梯度值、损失值，并确定纵向逻辑回归模型不收敛后，需要判断纵向逻辑回归模型是否满足预设判定条件，例如判断纵向逻辑回归模型的新迭代次数是否满足预设次数条件(如确定新迭代次数是否为迭代步数间隔的整数倍，且是否大于两倍大于预设次数)。并根据不同的判断结果执行不同的操作。
步骤y，若满足，则根据所述主加密数据和所述副加密数据更新二阶导数矩阵。
当经过判断发现纵向逻辑回归模型满足预设判定条件时，则需要根据协调者接收到的主参与者发送的主加密数据和副参与者发送的副加密数据来更新协调者中的二阶导数矩阵。
在本实施例中，通过在纵向逻辑回归模型未收敛时，判断纵向逻辑回归模型是否满足预设判定条件，并在满足时，才更新二阶导数矩阵，从而保障 了纵向逻辑回归模型的模型训练效率。
进一步地，根据所述主加密数据和所述副加密数据更新二阶导数矩阵的步骤，包括：步骤z，所述协调者将所述主加密数据和所述副加密数据进行解密合并，以获取目标数据；
协调者在接收到主参与者发送的主加密数据和副参与者发送的副加密数据后，进行解密合并得到目标数据，也就是将加密数据[[v A]],[[v B]]进行解密后合得到目标数据
Figure PCTCN2019119409-appb-000052
步骤m，将所述目标数据存储至预设长度的队列中，以获取目标队列，并通过所述目标队列对二阶导数矩阵进行更新。
协调者将目标数据存储在一个长度为M(即预设长度)的v队列中。同时，计算本次(t)与上次(t-1)的
Figure PCTCN2019119409-appb-000053
之差，即
Figure PCTCN2019119409-appb-000054
将其存在长度为M的s队列中。如果目前的存储器已经达到最大存储长度M，则将队列首个删掉并将最新得到v和s放在队列末尾。利用当前存储器里的m(m不大于M)个v和s，来计算H(二阶导数矩阵)。计算方法如下：
利用存储器队列末尾的值初始化，即计算
Figure PCTCN2019119409-appb-000055
H←p[m]I，其中I为单位矩阵。然后从队列首到队列尾(j＝1,..,m)迭代计算得到更新的H：p[j]＝1/(v[j] Ts[j]),H←(I-p[j]s[j]v[j] T)H(I-p[j]v[j]s[j] T)+p[j]s[j]s[j] T。
在本实施例中，通过将主加密数据和副加密数据进行解密合并得到目标数据，再根据目标数据对二阶导数矩阵进行更新，从而保障了二阶导数矩阵更新的有效性。
进一步地，判断所述纵向逻辑回归模型是否满足所述预设判定条件的步骤之后，包括：步骤n，若不满足，则所述协调者获取所述主参与者发送的主梯度值和预设的步长之间的第一乘积，并将所述第一乘积作为目标主梯度值发送至所述主参与者。
当经过判断发现纵向逻辑回归模型不满足预设判定条件，则协调者计算出事先选定的预设步长与主梯度值的第一乘积，和预设步长与副参与者对应的副梯度值的第三乘积，并将第一乘积作为目标主梯度值发送到主参与者中更新主参与者中的本地模型参数，将第三乘积发送到副参与者中更新副参与者中的模型参数，再根据更新后的各个模型参数重新进行模型训练，以获取新的损失函数值，并通过副参与者发送到协调者中。在本实施例中，通过在确定纵向逻辑回归模型不满足预设判定条件时，计算主梯度值和预设的步长之间的第一乘积，并将第一乘积作为目标主梯度值，从而保障了获取到的目标主梯度值的准确性。
本申请实施例还提供一种纵向联邦学习优化装置，参照图4，所述纵向联邦学习优化装置包括：获取模块，用于获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；发送模块，用于将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间 结果值；第一计算模块，用于根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛时，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；第一更新模块，用于接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。可选地，所述第一更新模块还用于：接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵获取的。可选地，所述第一更新模块还用于：接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据目标数据更新的二阶导数矩阵获取的，所述目标数据为响应于纵向逻辑回归模型未收敛，且满足预设判定条件，将所述主加密数据和所述副参与者发送的副加密数据进行解密合并得到的。可选地，所述第一更新模块还用于：接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者对第一目标乘积进行拆分得到的，所述第一目标乘积为根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵、所述主参与者发送的主梯度值和所述副参与者发送的副梯度值合并的长向量，和预设步长之间的乘积。可选地，所述第一更新模块还用于：接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值为第二目标乘积，所述第二目标乘积为所述协调者响应于纵向逻辑回归模型未收敛，且不满足预设判定条件，计算的所述主参与者发送的主梯度值和预设的步长之间的乘积。可选地，所述获取模块还用于：检测所述主参与者对应的纵向逻辑回归模型是否满足预设判定条件；若满足，则根据所述预设逻辑回归算法获取新数据在所述主参与者中的新加密数值集合。可选地，所述获取模块还用于：获取主参与者中的预设逻辑回归算法，并根据所述预设逻辑回归算法计算数据在所述主参与者中的逻辑回归得分，对所述逻辑回归得分进行加密，以获取加密数值集合。本申请实施例还提供一种纵向联邦学习优化装置，所述纵向联邦学习优化装置包括：接收模块，用于接收主参与者发送的主加密数据和副参与者发送的副加密数据，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算得，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；第二计算模块，用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；第二更新模块，用于将所述目标主梯度值分别发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。可选地，所述第二计算模块还用于：判断所述纵向逻辑回归模型是否满足所述预设判定条件； 若满足，则根据所述主加密数据和所述副加密数据更新二阶导数矩阵。可选地，所述第二计算模块还用于：所述协调者将所述主加密数据和所述副加密数据进行解密合并，以获取目标数据；将所述目标数据存储至预设长度的队列中，以获取目标队列，并通过所述目标队列对二阶导数矩阵进行更新。可选地，所述第二计算模块还用于：若不满足，则所述协调者获取所述主参与者发送的主梯度值和预设的步长之间的第一乘积，并将所述第一乘积作为目标主梯度值发送至所述主参与者。上述各程序模块所执行的方法可参照本申请纵向联邦学习优化方法各个实施例，此处不再赘述。
本申请实施例还提供一种存储介质，所述存储介质可以为非易失性可读存储介质。本申请存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如上所述的纵向联邦学习优化方法的步骤。其中，在所述处理器上运行的计算机可读指令被执行时所实现的方法可参照本申请纵向联邦学习优化方法各个实施例，此处不再赘述。需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种纵向联邦学习优化方法，其中，所述纵向联邦学习优化方法包括如下步骤：
    获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
    将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
    根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  2. 如权利要求1所述的纵向联邦学习优化方法，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵获取的。
  3. 如权利要求2所述的纵向联邦学习优化方法，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据目标数据更新的二阶导数矩阵获取的，所述目标数据为响应于纵向逻辑回归模型未收敛，且满足预设判定条件，将所述主加密数据和所述副参与者发送的副加密数据进行解密合并得到的。
  4. 如权利要求2所述的纵向联邦学习优化方法，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者对第一目标乘积进行拆分得到的，所述第一目标乘积为根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵、所述主 参与者发送的主梯度值和所述副参与者发送的副梯度值合并的长向量，和预设步长之间的乘积。
  5. 如权利要求1所述的纵向联邦学习优化方法，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值为第二目标乘积，所述第二目标乘积为所述协调者响应于纵向逻辑回归模型未收敛，且不满足预设判定条件，计算的所述主参与者发送的主梯度值和预设的步长之间的乘积。
  6. 如权利要求1所述的纵向联邦学习优化方法，其中，所述基于预设判定条件获取新数据在所述主参与者中的新加密数值集合的步骤，包括：
    检测所述主参与者对应的纵向逻辑回归模型满足预设判定条件，则根据所述预设逻辑回归算法获取新数据在所述主参与者中的新加密数值集合。
  7. 如权利要求1所述的纵向联邦学习优化方法，其中，所述获取数据在主参与者中的加密数值集合的步骤，包括：
    获取主参与者中的预设逻辑回归算法，并根据所述预设逻辑回归算法计算数据在所述主参与者中的逻辑回归得分，对所述逻辑回归得分进行加密，以获取加密数值集合。
  8. 一种纵向联邦学习优化方法，其中，所述纵向联邦学习优化方法包括如下步骤：
    接收主参与者发送的主加密数据和副参与者发送的副加密数据，其中，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算的，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
    响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    将所述目标主梯度值发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  9. 如权利要求8所述的纵向联邦学习优化方法，其中，所述根据所述主加密数据和所述副加密数据更新二阶导数矩阵的步骤，包括：
    确定所述纵向逻辑回归模型满足所述预设判定条件，则根据所述主加密数据和所述副加密数据更新二阶导数矩阵。
  10. 如权利要求9所述的纵向联邦学习优化方法，其中，所述根据所述主加密数据和所述副加密数据更新二阶导数矩阵的步骤，包括：
    所述协调者将所述主加密数据和所述副加密数据进行解密合并，以获取目标数据；
    将所述目标数据存储至预设长度的队列中，以获取目标队列，并通过所述目标队列对二阶导数矩阵进行更新。
  11. 如权利要求8所述纵向联邦学习优化方法，其中，所述纵向联邦学习优化方法，包括：
    确定纵向逻辑回归模型不满足所述预设判定条件，则所述协调者获取所述主参与者发送的主梯度值和预设的步长之间的第一乘积，并将所述第一乘积作为目标主梯度值发送至所述主参与者。
  12. 一种纵向联邦学习优化装置，其中，所述纵向联邦学习优化装置包括：
    获取模块，用于获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
    发送模块，用于将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
    第一计算模块，用于根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于确定纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    第一更新模块，用于接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  13. 一种纵向联邦学习优化装置，其中，所述纵向联邦学习优化装置还包括：
    接收模块，用于接收主参与者发送的主加密数据和副参与者发送的副加密数据，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算得，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
    第二计算模块，用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    第二更新模块，用于将所述目标主梯度值分别发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  14. 一种纵向联邦学习优化设备，其中，所述纵向联邦学习优化设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机可读指令，所述计算机可读指令被所述处理器执行时实现如下步骤：
    获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
    将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
    根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  15. 如权利要求14所述的纵向联邦学习优化设备，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵获取的。
  16. 一种纵向联邦学习优化设备，其中，所述纵向联邦学习优化设备包 括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机可读指令，所述计算机可读指令被所述处理器执行时实现如下步骤：
    接收主参与者发送的主加密数据和副参与者发送的副加密数据，其中，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算的，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
    响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    将所述目标主梯度值发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  17. 如权利要求16所述的纵向联邦学习优化设备，其中，所述根据所述主加密数据和所述副加密数据更新二阶导数矩阵的步骤，包括：
    确定所述纵向逻辑回归模型满足所述预设判定条件，则根据所述主加密数据和所述副加密数据更新二阶导数矩阵。
  18. 一种存储介质，其中，所述存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如下步骤：
    获取数据在主参与者中的加密数值集合，并基于预设判定条件获取新数据在所述主参与者中的新加密数值集合；
    将所述加密数值集合和所述新加密数值集合作为目标数值集合发送至副参与者，以获取所述副参与者基于所述目标数值集合反馈的中间结果值；
    根据所述中间结果值计算所述主参与者中的主加密数据，并将所述主加密数据发送至协调者，其中，所述协调者用于响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副参与者发送的副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    接收所述协调者反馈的目标主梯度值，基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
  19. 如权利要求18所述的存储介质，其中，所述接收所述协调者反馈的目标主梯度值的步骤，包括：
    接收所述协调者反馈的目标主梯度值，其中，所述目标主梯度值由所述协调者根据响应于所述纵向逻辑回归模型满足所述预设判定条件而更新的二阶导数矩阵获取的。
  20. 一种存储介质，其中，所述存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如下步骤：
    接收主参与者发送的主加密数据和副参与者发送的副加密数据，其中，所述主加密数据为根据所述副参与者基于所述主参与者发送目标数值集合反馈的中间结果值计算的，所述目标数值集包括数据对应的加密数值集合和新数据对应的新加密数值集合；
    响应于纵向逻辑回归模型未收敛，根据所述主加密数据和所述副加密数据更新二阶导数矩阵，并根据所述更新后的二阶导数矩阵计算目标主梯度值；
    将所述目标主梯度值发送给所述主参与者，所述主参与者用于基于所述目标主梯度值更新所述主参与者中的本地模型参数，并继续执行所述获取数据在主参与者中的加密数值集合的步骤，直至所述协调者对应的纵向逻辑回归模型收敛。
PCT/CN2019/119409 2019-11-14 2019-11-19 纵向联邦学习优化方法、装置、设备及存储介质 WO2021092977A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911124536.3A CN110851785B (zh) 2019-11-14 2019-11-14 纵向联邦学习优化方法、装置、设备及存储介质
CN201911124536.3 2019-11-14

Publications (1)

Publication Number Publication Date
WO2021092977A1 true WO2021092977A1 (zh) 2021-05-20

Family

ID=69600633

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/119409 WO2021092977A1 (zh) 2019-11-14 2019-11-19 纵向联邦学习优化方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN110851785B (zh)
WO (1) WO2021092977A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114003939A (zh) * 2021-11-16 2022-02-01 蓝象智联(杭州)科技有限公司 一种用于纵向联邦场景的多重共线性分析方法
CN114338144A (zh) * 2021-12-27 2022-04-12 杭州趣链科技有限公司 数据防泄露的方法、电子设备和计算机可读存储介质
CN116567652A (zh) * 2023-05-19 2023-08-08 上海科技大学 全向超表面辅助的空中计算赋能垂直联邦学习方法

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111401277A (zh) * 2020-03-20 2020-07-10 深圳前海微众银行股份有限公司 人脸识别模型更新方法、装置、设备和介质
CN111160573B (zh) * 2020-04-01 2020-06-30 支付宝(杭州)信息技术有限公司 保护数据隐私的双方联合训练业务预测模型的方法和装置
CN112668038A (zh) * 2020-06-02 2021-04-16 华控清交信息科技(北京)有限公司 一种模型训练方法、装置和电子设备
CN111783139A (zh) * 2020-06-29 2020-10-16 京东数字科技控股有限公司 联邦学习分类树构建方法、模型构建方法，及终端设备
CN112001500B (zh) * 2020-08-13 2021-08-03 星环信息科技(上海)股份有限公司 基于纵向联邦学习系统的模型训练方法、设备及存储介质
WO2022094888A1 (zh) * 2020-11-05 2022-05-12 浙江大学 一种面向决策树的纵向联邦学习方法
CN112651170B (zh) * 2020-12-14 2024-02-27 德清阿尔法创新研究院 一种纵向联邦学习场景中高效的特征贡献评估方法
CN112733967B (zh) * 2021-03-30 2021-06-29 腾讯科技（深圳）有限公司 联邦学习的模型训练方法、装置、设备及存储介质
CN113536667B (zh) * 2021-06-22 2024-03-01 同盾科技有限公司 联邦模型训练方法、装置、可读存储介质及设备
CN113487042B (zh) * 2021-06-28 2023-10-10 海光信息技术股份有限公司 一种联邦学习方法、装置及联邦学习系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109165515A (zh) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 基于联邦学习的模型参数获取方法、系统及可读存储介质
CN109711529A (zh) * 2018-11-13 2019-05-03 中山大学 一种基于值迭代网络的跨领域联邦学习模型及方法
US20190220253A1 (en) * 2018-01-15 2019-07-18 Cognizant Technology Solutions India Pvt. Ltd. System and method for improving software code quality using artificial intelligence techniques
CN110414688A (zh) * 2019-07-29 2019-11-05 卓尔智联(武汉)研究院有限公司 信息分析方法、装置、服务器及存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109299728B (zh) * 2018-08-10 2023-06-27 深圳前海微众银行股份有限公司 基于构建梯度树模型的样本联合预测方法、系统及介质
CN110263936B (zh) * 2019-06-14 2023-04-07 深圳前海微众银行股份有限公司 横向联邦学习方法、装置、设备及计算机存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190220253A1 (en) * 2018-01-15 2019-07-18 Cognizant Technology Solutions India Pvt. Ltd. System and method for improving software code quality using artificial intelligence techniques
CN109165515A (zh) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 基于联邦学习的模型参数获取方法、系统及可读存储介质
CN109711529A (zh) * 2018-11-13 2019-05-03 中山大学 一种基于值迭代网络的跨领域联邦学习模型及方法
CN110414688A (zh) * 2019-07-29 2019-11-05 卓尔智联(武汉)研究院有限公司 信息分析方法、装置、服务器及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CS ZHENG YANG: "An Introduction to Federated Learning", 21 July 2019 (2019-07-21), XP055812285, Retrieved from the Internet <URL:https://blog.csdn.net/sunyaowu315/article/details/96744126> *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114003939A (zh) * 2021-11-16 2022-02-01 蓝象智联(杭州)科技有限公司 一种用于纵向联邦场景的多重共线性分析方法
CN114003939B (zh) * 2021-11-16 2024-03-15 蓝象智联(杭州)科技有限公司 一种用于纵向联邦场景的多重共线性分析方法
CN114338144A (zh) * 2021-12-27 2022-04-12 杭州趣链科技有限公司 数据防泄露的方法、电子设备和计算机可读存储介质
CN116567652A (zh) * 2023-05-19 2023-08-08 上海科技大学 全向超表面辅助的空中计算赋能垂直联邦学习方法
CN116567652B (zh) * 2023-05-19 2024-02-23 上海科技大学 全向超表面辅助的空中计算赋能垂直联邦学习方法

Also Published As

Publication number Publication date
CN110851785A (zh) 2020-02-28
CN110851785B (zh) 2023-06-06

Similar Documents

Publication Publication Date Title
WO2021092977A1 (zh) 纵向联邦学习优化方法、装置、设备及存储介质
WO2021092980A1 (zh) 纵向联邦学习优化方法、装置、设备及存储介质
CN112733967B (zh) 联邦学习的模型训练方法、装置、设备及存储介质
Feng et al. Privacy preserving high-order bi-lanczos in cloud–fog computing for industrial applications
CN113239404B (zh) 一种基于差分隐私和混沌加密的联邦学习方法
CN113033828B (zh) 模型训练方法、使用方法、系统、可信节点及设备
WO2015155896A1 (ja) サポートベクトルマシン学習システムおよびサポートベクトルマシン学習方法
CN111984960B (zh) 一种基于同态加密的隐私保护设备识别模型设计及使用方法
CN113505882B (zh) 基于联邦神经网络模型的数据处理方法、相关设备及介质
CN114696990B (zh) 基于全同态加密的多方计算方法、系统及相关设备
CN113065660B (zh) 量子态测量方法、装置、设备、存储介质及系统
WO2022142366A1 (zh) 机器学习模型更新的方法和装置
US20230186049A1 (en) Training method and apparatus for a neural network model, device and storage medium
Feng et al. Privacy-preserving tensor analysis and processing models for wireless internet of things
CN111767411A (zh) 知识图谱表示学习优化方法、设备及可读存储介质
CN112016698A (zh) 因子分解机模型构建方法、设备及可读存储介质
CN113051586B (zh) 联邦建模系统及方法、联邦模型预测方法、介质、设备
CN114492850A (zh) 基于联邦学习的模型训练方法、设备、介质及程序产品
CN116502732B (zh) 基于可信执行环境的联邦学习方法以及系统
CN115965093A (zh) 模型训练方法、装置、存储介质及电子设备
Xiong et al. Decentralized privacy-preserving truth discovery for crowd sensing
US20210266383A1 (en) Conversion system, method and program
CN112615852A (zh) 数据的处理方法、相关装置及计算机程序产品
CN116681141A (zh) 隐私保护的联邦学习方法、终端及存储介质
CN110149199B (zh) 一种基于属性感知的隐私保护方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19952253

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19952253

Country of ref document: EP

Kind code of ref document: A1