CN113222180A - Federal learning modeling optimization method, apparatus, medium, and computer program product - Google Patents

Federal learning modeling optimization method, apparatus, medium, and computer program product Download PDF

Info

Publication number
CN113222180A
CN113222180A CN202110460157.2A CN202110460157A CN113222180A CN 113222180 A CN113222180 A CN 113222180A CN 202110460157 A CN202110460157 A CN 202110460157A CN 113222180 A CN113222180 A CN 113222180A
Authority
CN
China
Prior art keywords
party
random number
encryption
model
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110460157.2A
Other languages
Chinese (zh)
Inventor
谭明超
马国强
范涛
陈天健
杨强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN202110460157.2A priority Critical patent/CN113222180A/en
Publication of CN113222180A publication Critical patent/CN113222180A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses a federated learning modeling optimization method, equipment, a medium and a computer program product, wherein the federated learning modeling optimization method is applied to first equipment, and comprises the following steps: obtaining a first party local model predicted value corresponding to a first party local model to be trained; exchanging an encrypted value with second equipment based on a first party random number encrypted value generated by encrypting a random number on the first party local model predicted value to obtain a second party random number encrypted value; the second party random number encryption intermediate parameter generated based on the second party random number encryption value is exchanged with the encryption intermediate parameter of the second equipment to obtain a first party random number encryption intermediate parameter; and optimizing the local model to be trained of the first party based on the first party random number encryption intermediate parameters to obtain a first party federal model. The method and the device solve the technical problem of low safety when the model is built based on federal learning.

Description

Federal learning modeling optimization method, apparatus, medium, and computer program product
Technical Field
The present application relates to the field of artificial intelligence in financial technology (Fintech), and in particular, to a method, apparatus, medium, and computer program product for optimizing federated learning modeling.
Background
With the continuous development of financial science and technology, especially internet science and technology, more and more technologies (such as distributed technology, artificial intelligence and the like) are applied to the financial field, but the financial industry also puts higher requirements on the technologies, for example, higher requirements on the distribution of backlog in the financial industry are also put forward.
With the continuous development of computer technology, the application of federal learning is more and more extensive, at present, when a model is constructed through federal learning, a label owner participating in federal learning generally needs to send a homomorphic encrypted label to a non-label feature owner, then the feature owner calculates a homomorphic encrypted gradient and a homomorphic encryption loss based on the homomorphic encrypted label, sends the homomorphic encrypted gradient and the homomorphic encryption loss to the label owner, and after the label owner decrypts the homomorphic encrypted gradient and the homomorphic encryption loss, sends a plaintext model gradient and a plaintext model loss to the feature owner, the feature owner can update a local model, but if the feature owner receives the homomorphic encrypted label, ciphertext content (homomorphic encrypted label) which the owner wants to know is sent to the label owner instead of the homomorphic encrypted gradient and the homomorphic encryption loss, and then the label owner carries out the deciphering back, feeds back the deciphering result to the characteristic owner, and the sample label that the label owner can be taken over to the characteristic owner, promptly, has the deciphering interface in the whole federal learning process who constructs the model, and then the privacy data that the characteristic owner can take over the label owner through the deciphering interface, and then will lead to the sample label of the privacy data that is the label owner to reveal, so, at present there is still the potential safety hazard in the method based on federal learning construction model.
Disclosure of Invention
The application mainly aims to provide a method, equipment, a medium and a computer program product for optimizing modeling of federated learning, and aims to solve the technical problem of low safety when a model is built based on federated learning in the prior art.
In order to achieve the above object, the present application provides a federated learning modeling optimization method, where the federated learning modeling optimization method is applied to a first device, and the federated learning modeling optimization method includes:
obtaining a first party local model predicted value corresponding to a first party local model to be trained;
performing encryption value exchange with second equipment based on a first party random number encryption value generated by performing random number encryption on the first party local model predicted value to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by performing random number encryption on a second party local model predicted value and a sample tag corresponding to a second party local model to be trained by the second equipment to be used for the second equipment to obtain the first party random number encryption value;
exchanging encryption intermediate parameters with the second device to obtain first-party random number encryption intermediate parameters, wherein the first-party random number encryption intermediate parameters are generated by the second device based on the first-party random number encryption value, so that the second device optimizes the local model to be trained by the second party based on the second-party random number encryption intermediate parameters to obtain a second-party federated model;
and optimizing the local model to be trained of the first party based on the first party random number encryption intermediate parameters to obtain a first party federal model.
In order to achieve the above object, the present application provides a federated learning modeling optimization method, where the federated learning modeling optimization method is applied to a second device, and the federated learning modeling optimization method includes:
obtaining a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value;
exchanging an encrypted value with first equipment based on a second party random number encrypted value generated by encrypting a second party local model predicted value and the sample tag by a random number to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by encrypting the random number of the first party local model predicted value corresponding to the first party local model to be trained by the first equipment to obtain the second party random number encrypted value by the first equipment;
exchanging encryption intermediate parameters with the first device based on the first party random number encryption intermediate parameters generated by the first party random number encryption value to obtain second party random number encryption intermediate parameters, wherein the second party random number encryption intermediate parameters are generated by the first device based on the second party random number encryption value, so that the first device optimizes the local model to be trained by the first party based on the first party random number encryption intermediate parameters to obtain a first party federated model;
and optimizing the local model to be trained of the second party based on the encrypted intermediate parameter of the random number of the second party to obtain a federal model of the second party.
The application also provides a federal learning optimization device that models, federal learning optimization device that models is virtual device, just federal learning optimization device that models is applied to first equipment, federal learning optimization device that models includes:
the acquisition module is used for acquiring a first party local model predicted value corresponding to a first party local model to be trained;
the encrypted value exchange module is used for exchanging an encrypted value with a second device based on a first party random number encrypted value generated by encrypting a random number on the first party local model predicted value to obtain a second party random number encrypted value, wherein the second party random number encrypted value is obtained by encrypting the random number on a second party local model predicted value and a sample tag corresponding to a second party local model to be trained by the second device to obtain the first party random number encrypted value by the second device;
an encrypted intermediate parameter exchange module, configured to exchange an encrypted intermediate parameter with the second device based on a second-party random number encrypted intermediate parameter generated by the second-party random number encrypted value, to obtain a first-party random number encrypted intermediate parameter, where the first-party random number encrypted intermediate parameter is generated by the second device based on the first-party random number encrypted value, so that the second device optimizes the second-party local model to be trained based on the second-party random number encrypted intermediate parameter, and obtains a second-party federated model;
and the optimization module is used for optimizing the local model to be trained of the first party based on the encrypted intermediate parameters of the random number of the first party to obtain the federal model of the first party.
The application also provides a federal learning optimization device that models, federal learning optimization device that models is virtual device, just federal learning optimization device that models is applied to the second equipment, federal learning optimization device that models includes:
the acquisition module is used for acquiring a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value;
the encrypted value exchange module is used for exchanging an encrypted value with first equipment based on a second party random number encrypted value generated by encrypting a random number on the second party local model predicted value and the sample tag to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by encrypting the random number on the first party local model predicted value corresponding to the local model to be trained of the first party by the first equipment so as to be used for the first equipment to obtain the second party random number encrypted value;
the encryption intermediate parameter exchange module is used for exchanging encryption intermediate parameters with the first equipment based on a first party random number encryption intermediate parameter generated by the first party random number encryption value to obtain a second party random number encryption intermediate parameter, wherein the second party random number encryption intermediate parameter is generated by the first equipment based on the second party random number encryption value, so that the first equipment optimizes the local model to be trained by the first party based on the first party random number encryption intermediate parameter to obtain a first party federal model;
and the optimization module is used for optimizing the local model to be trained of the second party based on the encrypted intermediate parameters of the random number of the second party to obtain the federal model of the second party.
The application also provides a federal learning modeling optimization device, the federal learning modeling optimization device is an entity device, the federal learning modeling optimization device includes: a memory, a processor, and a program of the federated learning modeling optimization method stored on the memory and executable on the processor, the program of the federated learning modeling optimization method when executed by the processor may implement the steps of the federated learning modeling optimization method as described above.
The present application also provides a medium, which is a readable storage medium, on which a program for implementing the federal learning modeling optimization method is stored, and the program for implementing the federal learning modeling optimization method implements the steps of the federal learning modeling optimization method as described above when executed by a processor.
The present application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the method of federated learning modeling optimization as described above.
The application provides a federated learning modeling optimization method, equipment, a medium and a computer program product, compared with the technical means that a label owner involved in federated learning generally needs to send a homomorphic encrypted label to a label-free feature owner, the feature owner calculates homomorphic encrypted gradient and homomorphic encryption loss based on the homomorphic encrypted label, sends the homomorphic encrypted gradient and the homomorphic encryption loss to the label owner, and after the label owner decrypts the homomorphic encrypted gradient and the homomorphic encryption loss, feeds back the model gradient and the model loss in a plaintext state to the feature owner for model updating, the method firstly obtains a first party local model predicted value corresponding to a local model to be trained by a first party, and then generates a first party random number encrypted value based on random number encryption of the first party local model predicted value, exchanging encryption values with a second device to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by the second device through random number encryption on a second party local model prediction value and a sample label corresponding to a second party local model to be trained, so that the second device can obtain the first party random number encryption value, further perform encryption intermediate parameter exchange with the second device based on a second party random number encryption intermediate parameter generated by the second party random number encryption value to obtain a first party random number encryption intermediate parameter, wherein the first party random number encryption intermediate parameter is generated by the second device based on the first party random number encryption value, so that the second device can optimize the second party local model to be trained based on the second party random number encryption intermediate parameter to obtain a second party federated model, and then, based on the first party random number encryption intermediate parameter, optimizing the first party local model to be trained to obtain a first party federal model, wherein in the whole interaction process of constructing the federal model, the data mutually transmitted between the first equipment and the second equipment is encrypted data, so that a decryption interface does not exist in the process of constructing the federal model, and the technical defect that in the prior art, as the decryption interface exists in the whole federal learning process of constructing the model, the characteristic owner can collect the private data of the label owner through the decryption interface, and then the sample label serving as the private data of the label owner is leaked is overcome, so that the safety when the model is constructed based on the federal learning is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart diagram of a first embodiment of a federated learning modeling optimization method of the present application;
FIG. 2 is a schematic flow chart of iterative training of a local model to be trained by a first party on a first device side in the federated learning modeling optimization method according to the present application;
fig. 3 is a schematic flow chart of iterative training of a local model to be trained of a second party on a second device side in the federal learning modeling optimization method of the application;
FIG. 4 is a schematic flow chart diagram of a second embodiment of the federated learning modeling optimization method of the present application;
FIG. 5 is a schematic flow chart diagram illustrating a third embodiment of the federated learning modeling optimization method of the present application;
FIG. 6 is a schematic flow chart diagram illustrating a fourth embodiment of the federated learning modeling optimization method of the present application;
FIG. 7 is a schematic flow chart of a second device generating a federated prediction result in the federated learning modeling optimization method of the present application;
FIG. 8 is a schematic device structure diagram of a hardware operating environment related to a federated learning modeling optimization method in an embodiment of the present application;
fig. 9 is a hardware architecture diagram of federal learning according to an embodiment of the present application.
The objectives, features, and advantages of the present application will be further described with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In a first embodiment of the federal learning modeling optimization method of the present application, referring to fig. 1, the federal learning modeling optimization method is applied to a first device, and the federal learning modeling optimization method includes:
step S10, obtaining a first party local model predicted value corresponding to a first party local model to be trained;
in this embodiment, it should be noted that the federal learning modeling optimization method is applied to a longitudinal federal learning scenario, and both a first device and a second device are longitudinal federal learning participants, where the first device has a local sample without a sample label, the second device has a local sample with a sample label, the first-party to-be-trained local model is an untrained first-party local model in the first device, the first-party local model may be a logistic regression model, and a predicted value of the first-party local model is a logistic regression value output by the first-party to-be-trained local model in the first device.
Additionally, it should be noted that, in the first round of iterative process of federate iterative training of the local model to be trained by the first party, the model parameters of the local model to be trained of the first party are in a plaintext state, a second round of iterative process of the local model to be trained of the first party by federal iterative training is started, the model parameters of the local model to be trained of the first party are all in a homomorphic encrypted ciphertext state, therefore, the predicted value of the first-party local model may be represented as a product of a model parameter of the first-party local model to be trained and a sample, or may be represented as a product of a model parameter of the first-party local model to be trained and a sample, which are homomorphically encrypted, for example, assuming that the sample a is X, the model parameter of the first-party local model to be trained is W, the first-party local model prediction value is WX or [ [ WX ] ], wherein [ ] ] is a homomorphic encryption symbol.
The method comprises the steps of obtaining a first party local model predicted value corresponding to a first party local model to be trained, specifically, obtaining a public sample ID, wherein the public sample ID is a sample ID determined by sample alignment between first equipment and second equipment, further obtaining a first party local sample corresponding to the public sample ID and a first party local model parameter corresponding to a first party training local model parameter, further calculating the product of the first party local model parameter and the first party local sample, and obtaining a first party local model predicted value.
Step S20, exchanging encryption values with a second device based on a first party random number encryption value generated by encrypting a random number on the first party local model prediction value to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by encrypting the random number on a second party local model prediction value and a sample label corresponding to a second party local model to be trained by the second device to be used for the second device to obtain the first party random number encryption value;
in this embodiment, it should be noted that, in a scenario of longitudinal federal learning, a longitudinal federal model is divided into two parts, one part is in a first device, that is, a local model to be trained by a first party, the other part is in a second device, that is, a local model to be trained by a second party, and the local model to be trained by the second party is an untrained local model of the second party in the second device, where the local model of the second party may be a logistic regression model.
Performing encryption value exchange with a second device based on a first party random number encryption value generated by performing random number encryption on the first party local model predicted value to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by performing random number encryption on a second party local model predicted value and a sample tag corresponding to a second party local model to be trained by the second device, so that the second device can obtain the first party random number encryption value, specifically, mixing a homomorphic encrypted first party local model predicted value with a preset first party random number to obtain a first party random number encryption value based on a preset first party random number mixing mode, wherein the preset first party random number mixing mode comprises modes of summation, summation and the like, and further performing encryption value exchange with the second device based on the first party random number encryption value, sending the first party random number encrypted value to second equipment, and receiving a second party random number encrypted value sent by the second equipment, wherein the second equipment obtains a public sample ID consistent with that in the first equipment, further obtains a second party local sample corresponding to the public sample ID and second party local model parameters corresponding to the second party training local model parameters, further calculates the product of the second party local model parameters and the second party local sample, and obtains a second party local model predicted value, a base value and the likeMixing the second party local model predicted value, the preset second party random number and the sample label corresponding to the second party local sample, which are encrypted in a homomorphic way, to obtain a second party random number encrypted value, wherein the second party random number mixing way includes summation, multiplication and the like, for example, assuming that the first party local model predicted value is [ [ WX ]]]The preset first party random number mixing mode is summation, and the preset first party random number is r1And further said first party random number secret is [ [ WX ]]]+r1And assuming a preset second-party random number mixing mode as summation and difference calculation, and the predicted value of the second local model is [ [ WX ]2]]The sample label is y, and the preset second-party random number is r2And further said second party random number secret is 1/4[ WX [ [2]]-1/2y+r2。
The step of exchanging the encrypted value with the second device based on the first party random number encrypted value generated by encrypting the random number of the first party local model predicted value to obtain the second party random number encrypted value comprises:
step S21, based on the preset first party random number and the public key in the preset second party key pair, the random number encryption is carried out on the first party local model predicted value to obtain a first party random number encrypted value;
in this embodiment, it should be noted that the second device generates a preset second party key pair, and shares a public key in the preset second party key pair with the first device, and the first device generates a preset first party key pair, and shares a public key in the preset first party key pair with the second device.
Performing random number encryption on the predicted value of the local model of the first party based on a preset first party random number and a public key in a preset second party key pair to obtain a first party random number encrypted value, specifically, if the predicted value of the local model of the first party is in a homomorphic encryption state, wherein the predicted value of the local model of the first party is encrypted based on the public key in the preset second party key pair, and the iteration round of federal learning is greater than 1, mixing the predicted value of the local model of the first party with a preset first party random number based on a preset first random number mixing mode to obtain a first party random number encrypted value, and if the predicted value of the local model of the first party is not in the homomorphic encryption state and the iteration round of federal learning is equal to 1, performing homomorphic encryption on the predicted value of the local model of the first party based on the public key in the preset second party key pair, and obtaining a first party local model predicted value of homomorphic encryption, and further mixing the first party local model predicted value of homomorphic encryption with a preset first party random number based on a preset first random number mixing mode to obtain a first party random number encryption value.
The step of encrypting the random number of the first party local model prediction value based on the preset first party random number and the public key in the preset second party key pair to obtain the first party random number encryption value comprises the following steps:
step S211, homomorphic encryption is carried out on the predicted value of the local model of the first party based on the public key in the preset second party key pair, and the predicted value of the homomorphic encryption model of the first party is obtained;
in this embodiment, it should be noted that, at this time, the iteration round of the federal learning is 1, and the first-party homomorphic encryption model predicted value is the first-party local model predicted value in the homomorphic encryption state.
Step S212, mixing the predicted value of the homomorphic encryption model of the first party with the preset random number of the first party to generate the encrypted value of the random number of the first party.
In this embodiment, the predicted value of the first-party homomorphic encryption model is mixed with the preset first-party random number to generate the encrypted value of the first-party random number, and specifically, the sum of the predicted value of the first-party homomorphic encryption model and the preset first-party random number is calculated to mix the predicted value of the first-party homomorphic encryption model with the preset first-party random number to obtain the encrypted value of the first-party random number.
And step S22, exchanging the encrypted value with the second device based on the encrypted value of the first party random number to obtain the encrypted value of the second party random number.
In this embodiment, based on the first party random number encrypted value, exchanging an encrypted value with the second device to obtain the second party random number encrypted value, specifically, the first device sends the first party random number encrypted value to the second device and receives the second party random number encrypted value sent by the second device, and the second device sends the second party random number encrypted value to the first device and receives the first party random number encrypted value sent by the first device.
Step S30, exchanging encryption intermediate parameters with the second device based on the second party random number encryption intermediate parameters generated by the second party random number encryption value to obtain first party random number encryption intermediate parameters, wherein the first party random number encryption intermediate parameters are generated by the second device based on the first party random number encryption value, so that the second device optimizes the local model to be trained by the second party based on the second party random number encryption intermediate parameters to obtain a second party federation model;
in this embodiment, a second-party random number encryption intermediate parameter generated based on the second-party random number encryption value is exchanged with the second device to obtain a first-party random number encryption intermediate parameter, wherein the first-party random number encryption intermediate parameter is generated by the second device based on the first-party random number encryption value, so that the second device optimizes the second-party local model to be trained based on the second-party random number encryption intermediate parameter to obtain a second-party federated model, specifically, decrypts the second-party random number encryption value based on a private key in a preset first-party key pair to obtain a second-party random number hybrid value, and further calculates the second-party random number encryption intermediate parameter based on the second-party random number hybrid value and the predicted value of the first-party local model, and based on a public key in the preset first-party key pair, homomorphic encryption is carried out on the second party random number mixed intermediate parameter to obtain a second party random number encrypted intermediate parameter, encryption intermediate parameter exchange is carried out with second equipment based on the second party random number encrypted intermediate parameter to send the second party random number encrypted intermediate parameter to the second equipment, and a first party random number encrypted intermediate parameter sent by the second equipment is received to be supplied to the second equipment to receive the second party random number encrypted intermediate parameter sent by the first equipment, random number elimination is carried out on the second party random number intermediate parameter to obtain a second party encrypted intermediate parameter, a second party encryption model gradient is generated by calculating the product of the second party encrypted intermediate parameter and a second party local sample, and then the second party to be trained local model is optimized based on the second party encryption model gradient to obtain a second party federal model, the second device decrypts the first party random number encrypted value based on a private key in a preset second party key pair to obtain a first party random number mixed value, calculates a first party random number mixed intermediate parameter based on the first party random number mixed value and the second party local model predicted value, and homomorphically encrypts the first party random number mixed intermediate parameter based on a public key in the preset second party key pair to obtain a first party random number encrypted intermediate parameter.
The second party random number encryption intermediate parameter generated based on the second party random number encryption value exchanges encryption intermediate parameters with the second device, and the step of obtaining the first party random number encryption intermediate parameter comprises the following steps:
step S31, generating a second party random number encryption intermediate parameter based on a private key and a public key of a preset first party key pair, the second party random number encryption value and the first party local model prediction value;
in this embodiment, a second-party random number encryption intermediate parameter is generated based on a private key and a public key of a preset first-party key pair, the second-party random number encryption value, and the predicted value of the first-party local model, specifically, the second-party random number encryption value is decrypted based on the private key of the preset first-party key pair to obtain a second-party random number mixed value, and then the second-party random number mixing intermediate parameter is calculated based on the second-party random number mixed value and a first-party local sample corresponding to the predicted value of the first-party local model, and the second-party random number mixing intermediate parameter is homomorphically encrypted based on the public key of the preset first-party key pair to generate a second-party random number encryption intermediate parameter, where the process of calculating the second-party random number mixing intermediate parameter is as follows:
Figure BDA0003041960050000111
wherein d is the second square random number mixed intermediate parameter, WX1For the first party local model prediction value, WX2Is the second-party local model prediction value, y is the sample label, r2The preset second-party random number is the second-party random number, and the mixed value of the second-party random number is 1/4WX2-1/2y+r2。
The step of generating a second party random number encryption intermediate parameter based on the private key and the public key of the preset first party key pair, the second party random number encryption value and the first party local model prediction value comprises:
step S311, based on the private key in the preset first party key pair, decrypting the second party random number encrypted value to obtain a second party random number mixed value;
in this embodiment, it should be noted that the second-party random number encrypted value is a homomorphic encrypted second-party random number mixed value.
Step S312, generating a second party random number mixing intermediate parameter based on the first party local model predicted value and the second party random number mixing value;
in this embodiment, a second-party random number mixture intermediate parameter is generated based on the first-party local model predicted value and the second-party random number mixture value, and specifically, the second-party random number mixture intermediate parameter is generated by calculating a sum of the first-party local model predicted value and the second-party random number mixture value.
Step 313, encrypting the second party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the second party random number encrypted intermediate parameter.
In this embodiment, it should be noted that the private key in the preset first party key pair is only held by the first device, and both the first device and the second device hold the public key in the preset first party key pair.
Step S32, performing encryption intermediate parameter exchange with the second device based on the second party random number encryption intermediate parameter, to obtain the first party random number encryption intermediate parameter.
In this embodiment, the first party random number encryption intermediate parameter is obtained by exchanging encryption intermediate parameters with the second device based on the second party random number encryption intermediate parameter, specifically, the first device sends the second party random number encryption intermediate parameter to the second device, so that the second device receives the second party random number encryption intermediate parameter, the second device sends the first party random number encryption intermediate parameter to the first device, so that the first device receives the first party random number encryption intermediate parameter, where it needs to be noted that, since the first party random number encryption intermediate parameter is encrypted based on a public key in a preset second party secret key, and a private key in the preset second party secret key is only held by the second device, the first device cannot decrypt the first party random number encryption intermediate parameter, and the second party random number encryption intermediate parameter is encrypted based on the public key in the preset first party secret key, and the private key in the preset first party secret key is only held by the first equipment, so that the second equipment cannot decrypt the second party random number encryption intermediate parameter.
And step S40, optimizing the local model to be trained of the first party based on the encrypted intermediate parameters of the random number of the first party to obtain a federal model of the first party.
In this embodiment, the local model to be trained of the first party is optimized based on the first party random number encryption intermediate parameter to obtain a first party federation model, specifically, the first party random number encryption intermediate parameter is subjected to random number elimination to obtain a first party encryption intermediate parameter, and then the model parameter of the local model to be trained of the first party is updated based on a first party encryption model gradient generated by the first party encryption intermediate parameter to obtain the first party federation model.
The step of optimizing the local model to be trained of the first party based on the first party random number encryption intermediate parameter to obtain the first party federal model comprises the following steps:
step S41, random number elimination is carried out on the first party random number encryption intermediate parameter to obtain a first party encryption intermediate parameter;
in this embodiment, the first party nonce encryption intermediate parameter is subjected to nonce elimination to obtain a first party nonce encryption intermediate parameter, and specifically, the first party nonce encryption intermediate parameter is subjected to nonce elimination by calculating a difference between the first party nonce encryption intermediate parameter and a preset first party nonce to obtain a first party nonce encryption intermediate parameter, where a process of performing nonce elimination on the first party nonce encryption intermediate parameter is as follows:
[[d]]1＝[[d1]]-r1
Figure BDA0003041960050000121
wherein [ [ d ]]]1Encrypting an intermediate parameter, [ [ d ] for the first party1]]Encrypting an intermediate parameter, WX, for a first party random number1For the first party local model prediction value, WX2Is the second-party local model prediction value, y is the sample label, r1And presetting a first party random number.
Step S42, generating a first party encryption gradient based on the first party encryption intermediate parameter and a first party local sample corresponding to the first party local model prediction value;
in this embodiment, a first party encryption gradient is generated based on the first party encryption intermediate parameter and the first party local sample corresponding to the first party local model prediction value, specifically, the first party encryption gradient is generated by calculating a product between the first party encryption intermediate parameter and the first party local sample corresponding to the first party local model prediction value, where a specific process of generating the first party encryption gradient is as follows:
[[g]]1＝[[d]]1*X
wherein [ [ g ]]]1Encrypting a gradient for the first party [ [ d ]]]1Encrypting an intermediate parameter for the first party, X being the first party local sample.
And step S43, optimizing the local model to be trained of the first party based on the encryption gradient of the first party to obtain the federal model of the first party.
In this embodiment, the local model to be trained of the first party is optimized based on the first party encryption gradient to obtain the first party federation model, and specifically, the model parameters of the local model to be trained of the first party are updated by a preset model parameter updating method based on the first party encryption gradient to obtain the first party federation model, where the preset model parameter updating method includes a gradient descent method, a gradient ascent method, and the like.
The step of optimizing the local model to be trained of the first party based on the encryption gradient of the first party and obtaining the federal model of the first party comprises the following steps:
step S431, based on the first party encryption gradient, updating model parameters of the first party local model to be trained, and judging whether the first party local model to be trained after model parameter updating meets preset training end conditions;
in this embodiment, it should be noted that the preset training end includes loss convergence, reaching the maximum number of iterations, and the like.
Step S432, if the local model is met, taking the first party local model to be trained after the model parameters are updated as the first party federal model;
and S433, if the local model prediction value does not meet the requirement, returning to the step of obtaining the first party local model prediction value corresponding to the first party local model to be trained.
In this embodiment, if the first-party local model to be trained after the model parameter update meets the preset training end condition, the first-party local model to be trained after the model parameter update is directly used as the first-party federal model, and if the first-party local model to be trained after the model parameter update does not meet the preset training end condition, the step of obtaining the first-party local model predicted value corresponding to the first-party local model to be trained is returned to be executed.
In the whole federal learning process of the embodiment of the application, the first party random number encryption value, the second party random number encryption value, the first party random number encryption intermediate parameter and the second party random number encryption intermediate parameter which are exchanged between the first device and the second device are in homomorphic encryption states, and for the first party random number encryption value, the second device can decrypt but still ensure the privacy of data due to the mixing of random numbers, while for the second party random number encryption value, the first device can decrypt but also ensure the privacy of data due to the mixing of random numbers, while the first party random number encryption intermediate parameter held by the first device is encrypted by the public key in the preset second party key pair, the first device cannot decrypt, the second device holds the second party random number encryption intermediate parameter and is encrypted by the public key in the preset first party key pair, the unable deciphering of second equipment, and in whole federal learning process, gradient and model parameter in the model in first equipment and the second equipment all are in homomorphic encryption state, the data of the own side of first equipment and the unable deciphering of second equipment, and then realized guaranteeing the purpose of data privacy nature in the federal learning process, and because the data of transmission is encrypted data between first equipment and the second equipment, first equipment and second equipment need not the assistance of the other side and carry out the data decryption at the federal learning in-process, and then do not have the decryption interface at whole federal learning in-process, therefore, the security in the federal learning process has been promoted.
Further, as shown in fig. 2, a schematic flow diagram of iterative training of a first local model to be trained of a first party on a first device side is shown, where a is the first device, B is the second device, x1, x2, x3, x4, and x5 are features of samples, y is a sample label, r is a sample label1To preset first party randomThe number, wx _ g is the predicted value of the local model of the first party, wx _ h is the predicted value of the local model of the second party, the homomorphic encryption public key and the private key generated by B are the preset key pair of the second party, [ [ wx ]]]+r1Encrypting a value for said first party random number, [ [ d _ b ]]]Encrypting an intermediate parameter for the first party random number [ [ d ]]]Encrypting an intermediate parameter, [ [ g ] for the first party]]Encrypting the gradient for the first party, as shown in fig. 3, a schematic flow diagram of iteratively training a local model to be trained of a second party on a side of a second device is shown, where a is the first device, B is the second device, x1, x2, x3, x4, and x5 are features of samples, y is a sample label, r is a sample label2For presetting a random number of a second party, wx _ g is a predicted value of a local model of the first party, wx _ h is a predicted value of a local model of the second party, and a homomorphic encryption public key and a private key generated by A are a pair of preset first party keys 1/4[ [ wx ]]]-1/2y+r2Encrypting a value for the second party random number, [ [ d _ a ]]]Encrypting an intermediate parameter for the second party random number [ [ d ]]]Encrypting an intermediate parameter, [ [ g ] for the second party]]Encrypting the gradient for the second party.
Further, when constructing the logistic regression model based on federal learning, the tag owner participating in the federal learning generally needs to send the homomorphic encrypted tag to the non-tagged feature owner, and then the feature owner calculates the gradient of homomorphic encryption and the loss of homomorphic encryption based on the homomorphic encrypted tag, and sends the gradient of homomorphic encryption and the loss of homomorphic encryption to the tag owner, and then the tag owner decrypts the gradient of homomorphic encryption and the loss of homomorphic encryption, and sends the model gradient of plaintext state and the model loss to the feature owner, and then the feature owner can update the local logistic regression model, so that in the prior art, there is a decryption interface in the process of constructing the logistic regression model based on the federal learning, and then the tag owner has a risk of data privacy leakage, while in the process of constructing the model based on the federal learning in the embodiment of the present application, a decryption interface does not exist from beginning to end, and then the logistic regression model is built through the federal learning modeling optimization method in the embodiment of the application, so that the risk that a label owner leaks data privacy can be avoided, and the safety of building the logistic regression model based on federal learning is improved.
Compared with the technical means that a label owner who participates in federal learning generally needs to send a homomorphic encrypted label to a label-free feature owner, the feature owner calculates homomorphic encryption gradient and homomorphic encryption loss based on the homomorphic encrypted label, sends the homomorphic encryption gradient and homomorphic encryption loss to the label owner, decrypts the homomorphic encryption gradient and the homomorphic encryption loss, and feeds back a model gradient and a model loss in a plaintext state to the feature owner for model updating, the method provided by the embodiment of the application firstly obtains a first party local model predicted value corresponding to a local model to be trained of a first party, and then generates a first party random number encrypted value based on random number encryption on the first party local model predicted value, exchanging encryption values with a second device to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by the second device through random number encryption on a second party local model prediction value and a sample label corresponding to a second party local model to be trained, so that the second device can obtain the first party random number encryption value, further perform encryption intermediate parameter exchange with the second device based on a second party random number encryption intermediate parameter generated by the second party random number encryption value to obtain a first party random number encryption intermediate parameter, wherein the first party random number encryption intermediate parameter is generated by the second device based on the first party random number encryption value, so that the second device can optimize the second party local model to be trained based on the second party random number encryption intermediate parameter to obtain a second party federated model, and then, based on the first party random number encryption intermediate parameter, optimizing the first party local model to be trained to obtain a first party federal model, wherein in the whole interaction process of constructing the federal model, data mutually transmitted between the first equipment and the second equipment are encrypted data, so that a decryption interface does not exist in the process of constructing the federal model in the embodiment of the application, and the technical defect that in the prior art, due to the fact that the decryption interface exists in the whole federal learning process of constructing the model, a characteristic owner can collect the private data of a label owner through the decryption interface, and then a sample label serving as the private data of the label owner is leaked is overcome, and therefore the safety when the model is constructed based on the federal learning is improved.
Further, referring to fig. 4, based on the first embodiment of the present application, in another embodiment of the present application, the first party federal model includes first party cryptographic model parameters encrypted by a public key of a preset second party key pair, the second party federal model includes second party cryptographic model parameters encrypted by a public key of a preset first party key pair,
after the step of optimizing the local model to be trained of the first party and obtaining the federal model of the first party based on the encrypted intermediate parameter of the random number of the first party, the federal learning modeling optimization method further includes:
step A10, obtaining a first party sample to be predicted, and generating a first party encryption model output value based on the first party sample to be predicted and the first party encryption model parameter;
in this embodiment, it should be noted that the first party encryption model parameter is a model parameter encrypted based on a public key in a preset second party key pair.
The method comprises the steps of obtaining a first party to-be-predicted sample, generating a first party encryption model output value based on the first party to-be-predicted sample and a first party encryption model parameter, specifically obtaining the first party to-be-predicted sample, and obtaining the first party encryption model output value by calculating the product between the first party to-be-predicted sample and the first party encryption model parameter.
Step A20, sending the first party encryption model output value and the target sample ID of the first party to-be-predicted sample to the second device, so that the second device generates a second party encryption model output value based on the second party to-be-predicted sample corresponding to the target sample ID and the second party encryption model parameter, and decrypts the first party encryption model output value based on a private key in the preset second party key pair to obtain a first party model output value;
in this embodiment, the first party cryptographic model output value and the target sample ID of the first party sample to be predicted are sent to the second device, so that the second device generates a second party cryptographic model output value based on the second party sample to be predicted corresponding to the target sample ID and the second party cryptographic model parameter, decrypts the first party cryptographic model output value based on a private key in the preset second party key pair, and obtains a first party model output value, and specifically, the first party cryptographic model output value and the target sample ID of the first party sample to be predicted are sent to the second device, so that the second device obtains a second party predicted sample corresponding to the target sample ID, and obtains a second party cryptographic model output value by calculating a product between the second party predicted sample and the second party cryptographic model parameter, and decrypting the first party encryption model output value based on a private key in a preset second party key pair to obtain a first party model output value, and then sending the first party model output value and the second party encryption model output value to the first equipment by the second equipment.
Step a30, receiving a second party encryption model output value and a first party model output value sent by the second device, and decrypting the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party model output value;
in this embodiment, it should be noted that, because the second-party encryption model parameter is encrypted based on the public key in the preset first key pair, and further based on the private key in the preset first-party key pair, the second-party encryption model output value can be decrypted to obtain the second-party model output value.
And A40, generating a federal prediction result based on the first-party model output value and the second-party model output value.
In this embodiment, a federal prediction result is generated based on the first-party model output value and the second-party model output value, and specifically, the first-party model output value and the second-party model output value are aggregated based on a preset aggregation manner to obtain a federal prediction result, where the preset aggregation manner includes weighted summation and weighted averaging.
Furthermore, the model constructed based on the federal learning can be a logistic regression model, and when the logistic regression model is classified, the logistic regression model is based on the federal model without a decryption interface, so that the purpose of classifying the logistic regression model constructed based on the federal learning with higher safety can be achieved, and the safety of classifying the logistic regression model constructed based on the federal learning is improved.
The embodiment of the application provides a federated prediction method, wherein, because a first federated model and a second federated model are constructed based on a federated learning process without decryption interfaces, and then model parameters in first equipment and second equipment are all in a homomorphic encryption state, so as to obtain a sample to be predicted by a first party, and based on the sample to be predicted by the first party and the first party encryption model parameters, a first party encryption model output value is generated, and further the first party encryption model output value and a target sample ID of the sample to be predicted by the first party are sent to the second equipment, so that the second equipment generates a second party encryption model output value based on the sample to be predicted by the second party corresponding to the target sample ID and the second party encryption model parameters, and decrypts the first party encryption model output value based on a private key in a preset second party key pair, and obtaining a first party model output value, further receiving a second party encryption model output value and a first party model output value sent by the second equipment, decrypting the second party encryption model output value based on a private key in the preset first party key pair, obtaining a second party model output value, further generating a federal prediction result based on the first party model output value and the second party model output value, achieving the purpose of carrying out federal prediction based on a federal model without a decryption interface, further achieving the purpose of carrying out federal prediction based on a federal model with higher safety, and improving the safety of the federal prediction.
Further, in the first embodiment of the federal learning modeling optimization method of the present application, referring to fig. 5, the federal learning modeling optimization method is applied to a second device, and the federal learning modeling optimization method includes:
step B10, obtaining a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value;
in this embodiment, it should be noted that the federal learning modeling optimization method is applied to a longitudinal federal learning scenario, and both the first device and the second device are longitudinal federal learning participants, where the first device has a local sample without a sample label, the second device has a local sample with a sample label, the local model to be trained by the second party is an untrained local model of the second device, the local model of the second party may be a logistic regression model, and a predicted value of the local model of the second party is a logistic regression value output by the local model to be trained by the second party in the second device.
Additionally, it should be noted that, in the first iteration process of the federal iterative training of the local model to be trained of the second party, model parameters of the local model to be trained of the second party are in a plaintext state, a second round of iteration process of the local model to be trained of the second party is started by federal iteration training, the model parameters of the local model to be trained of the second party are all in a homomorphic encrypted ciphertext state, therefore, the predicted value of the second-party local model may be represented as a product of the model parameter of the second-party local model to be trained and the sample, or may be represented as a product of the model parameter of the homomorphic encrypted second-party local model to be trained and the sample, for example, assuming that the sample a is X, the model parameter of the second-party local model to be trained is W, the second-party local model prediction value is WX or [ [ WX ] ], wherein [ ] ] is a homomorphic encryption symbol.
Obtaining a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value, specifically, obtaining a common sample ID, wherein the common sample ID is a sample ID of a sample alignment determination made by the first device and the second device, further obtaining a second party local sample corresponding to the common sample ID and a second party local model parameter corresponding to the second party training local model parameter, further calculating the product of the second local model parameter and the second local sample to obtain the second local model prediction value, wherein the first party local sample and the second party local sample correspond to the same common sample ID, thereby obtaining a sample label corresponding to the second party local sample, the sample label is an identifier of the second-party local sample, and is used for identifying a category or an attribute of the second-party local sample.
Step B20, exchanging encryption values with first equipment based on a second party random number encryption value generated by encrypting the random number of the second party local model prediction value and the sample label to obtain a first party random number encryption value, wherein the first party random number encryption value is obtained by the first equipment through encrypting the random number of the first party local model prediction value corresponding to the first party local model to be trained, so that the first equipment can obtain the second party random number encryption value;
in this embodiment, it should be noted that, in a scenario of longitudinal federal learning, a longitudinal federal model is divided into two parts, one part is in a first device, that is, a local model to be trained by a first party, and the other part is in a second device, that is, a local model to be trained by a second party, where the local model to be trained by the first party is an untrained local model of the first party in the first device, and the local model of the first party may be a logistic regression model.
Exchanging an encrypted value with a first device based on a second party random number encrypted value generated by encrypting a random number for the second party local model predicted value and the sample tag to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by encrypting a random number for the first party local model predicted value corresponding to the first party local model to be trained by the first device to obtain the second party random number encrypted value by the first device, and specifically, based on a preset second party random number mixing mode, the second party random number encrypted value is obtained by the first devicePresetting a second party random number, a sample label and a second party local model predicted value encrypted in a homomorphic way to obtain a second party random number encrypted value, wherein the preset second party random number mixing way comprises a summing way, a quadrature way and the like, and further carrying out encryption value exchange with first equipment based on the second party random number encrypted value so as to send the second party random number encrypted value to the first equipment and receive the first party random number encrypted value sent by the first equipment, wherein the first equipment obtains a public sample ID consistent with that in the second equipment, further obtains a first party local sample corresponding to the public sample ID and a first party local model parameter corresponding to a first party training local model parameter, further calculates the product of the first party local model parameter and the first party local model parameter to obtain a first party local model predicted value, and mixing a preset first-party random number and a homomorphically encrypted first-party local model predicted value based on a preset first-party random number mixing mode to obtain a first-party random number encrypted value, wherein the preset first-party random number mixing mode comprises summation, product solving and the like, for example, the first-party local model predicted value is assumed to be [ [ WX ]]]The preset first party random number mixing mode is summation, and the preset first party random number is r1And further said first party random number secret is [ [ WX ]]]+r1And assuming a preset second-party random number mixing mode as summation and difference calculation, and the predicted value of the second local model is [ [ WX ]2]]The sample label is y, and the preset second-party random number is r2And further said second party random number secret is 1/4[ WX [ [2]]-1/2y+r2。
Wherein, the step of exchanging the encrypted value with the first device based on the encrypted value of the second party random number generated by encrypting the random number of the predicted value of the second party local model and the sample label to obtain the encrypted value of the first party random number comprises:
step B21, based on the preset second party random number and the public key in the preset first party key pair, encrypting the random number of the second party local model predicted value and the sample label to obtain a second party random number encrypted value;
in this embodiment, it should be noted that the second device generates a preset second party key pair, and shares a public key in the preset second party key pair with the first device, and the first device generates a preset first party key pair, and shares a public key in the preset first party key pair with the second device.
Encrypting the random number of the second party local model predicted value and the sample label based on a preset second party random number and a public key in a preset first party key pair to obtain a second party random number encrypted value, specifically, if the second party local model predicted value is in a homomorphic encryption state, wherein the second party local model predicted value is encrypted based on the public key in the preset first party key pair, and the iteration round of federal learning is greater than 1, mixing the second party local model predicted value, the preset second party random number and the sample label based on a preset second random number mixing mode to obtain a second party random number encrypted value, and if the second party local model predicted value is not in the homomorphic encryption state and the iteration round of federal learning is equal to 1, homomorphic encrypting the second party local model predicted value based on the public key in the preset first party key pair, and obtaining a second party local model predicted value of homomorphic encryption, and further mixing the preset first party random number, the sample tag and the first party local model predicted value of homomorphic encryption based on a preset second random number mixing mode to obtain a second party random number encryption value.
The step of encrypting the random number of the second party local model predicted value and the sample tag based on the preset second party random number and the public key of the preset first party key pair to obtain the second party random number encrypted value comprises:
step B211, homomorphic encryption is carried out on the second party local model predicted value based on the public key in the preset first party key pair, and a second party homomorphic encryption model predicted value is obtained;
in this embodiment, it should be noted that, at this time, the iteration round of the federal learning is 1, and the second-party homomorphic encryption model predicted value is a second-party local model predicted value in a homomorphic encryption state.
Step B212, mixing the predicted value of the homomorphic encryption model of the second party, the sample label and the preset random number of the second party to generate the encrypted value of the random number of the second party.
In this embodiment, the predicted value of the homomorphic encryption model of the second party, the sample tag, and the preset second party random number are mixed to generate the encrypted value of the second party random number, specifically, a difference between the predicted value of the homomorphic encryption model of the second party at a first preset multiple and the sample tag at a preset second multiple is calculated to obtain a difference calculation result, and then the encrypted value of the second party random number is generated by calculating a sum between the difference calculation result and the preset second party random number, where the specific process of generating the encrypted value of the second party random number is as follows:
Figure BDA0003041960050000211
wherein N is the second party random number encryption value, [ [ WX ]2]]For the second party homomorphic cryptographic model prediction value, y is the sample label, r2For the preset second party random number,
Figure BDA0003041960050000212
is the first preset multiple, and is the first preset multiple,
Figure BDA0003041960050000213
is the second preset multiple.
And step B22, exchanging the encrypted value with the first device based on the encrypted value of the second party random number to obtain the encrypted value of the first party random number.
In this embodiment, based on the second party random number encrypted value, exchanging an encrypted value with the first device to obtain the first party random number encrypted value, specifically, the second device sends the second party random number encrypted value to the first device and receives the first party random number encrypted value sent by the first device, and the first device sends the first party random number encrypted value to the second device and receives the second party random number encrypted value sent by the second device.
Step B30, exchanging encryption intermediate parameters with the first device based on the first party random number encryption intermediate parameters generated by the first party random number encryption value to obtain second party random number encryption intermediate parameters, wherein the second party random number encryption intermediate parameters are generated by the first device based on the second party random number encryption value, so that the first device optimizes the local model to be trained by the first party based on the first party random number encryption intermediate parameters to obtain a first party federated model;
in this embodiment, a first-party random number encryption intermediate parameter generated based on the first-party random number encryption value is exchanged with the first device to obtain a second-party random number encryption intermediate parameter, where the second-party random number encryption intermediate parameter is generated by the first device based on the second-party random number encryption value, so that the first device optimizes the local model to be trained by the first party based on the first-party random number encryption intermediate parameter to obtain a first-party federated model, specifically, decrypts the first-party random number encryption value based on a private key in a preset second-party key pair to obtain a first-party random number mixed value, and further calculates a first-party random number mixing intermediate parameter based on the first-party random number mixed value and the predicted value of the second-party local model, and based on a public key in a preset second-party key pair, performing homomorphic encryption on the first party random number mixed intermediate parameter to obtain a first party random number encrypted intermediate parameter, further performing encryption intermediate parameter exchange with first equipment based on the first party random number encrypted intermediate parameter to send the first party random number encrypted intermediate parameter to the first equipment, and receiving a second party random number encrypted intermediate parameter sent by the first equipment, so that the first equipment receives the first party random number encrypted intermediate parameter sent by the second equipment, further performing random number elimination on the first party random number intermediate parameter to obtain a first party encrypted intermediate parameter, further generating a first party encryption model gradient by calculating the product of the first party encrypted intermediate parameter and a first party local sample, further optimizing the first party local model to be trained based on the first party encryption model gradient to obtain a first party federation model, the first device decrypts the second party random number encryption value based on a private key in a preset first party key pair to obtain a second party random number mixed value, calculates a second party random number mixed intermediate parameter based on the second party random number mixed value and the first party local model predicted value, and homomorphically encrypts the second party random number mixed intermediate parameter based on a public key in the preset first party key pair to obtain a second party random number encryption intermediate parameter.
The step of obtaining the second party random number encryption intermediate parameter includes:
step B31, generating a first party random number encryption intermediate parameter based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model prediction value and the sample label;
in this embodiment, a first party random number encryption intermediate parameter is generated based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model predicted value and the sample label, specifically, the first party random number encryption value is decrypted based on the private key of the preset second party key pair to obtain a first party random number mixed value, and then a first party random number mixing intermediate parameter is calculated based on the first party random number mixed value, a second party local sample corresponding to the second party local model predicted value and a sample label corresponding to the second party local sample, and the first party random number mixing intermediate parameter is homomorphically encrypted based on the public key of the preset second party key pair to generate a first party random number encryption intermediate parameter.
The step of generating a first party random number encryption intermediate parameter based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model prediction value and the sample label comprises:
step B311, based on the private key in the preset second party key pair, decrypting the first party random number encrypted value to obtain a first party random number mixed value;
in this embodiment, it should be noted that the first party random number encrypted value is a homomorphic encrypted first party random number mixed value.
Step B312, generating a first party random number mixed intermediate parameter based on the second party local model predicted value, the first party random number mixed value and the sample label;
in this embodiment, a first party random number mixed intermediate parameter is generated based on the second party local model predicted value, the first party random number mixed value and the sample label, specifically, a sum of the second party local model predicted value and the first party random number mixed value is calculated to obtain a summation result, and then the first party random number mixed intermediate parameter is generated by calculating a difference between the summation result of a first preset multiple and the sample label of a preset second multiple, wherein a specific process of generating the first party random number mixed intermediate parameter is as follows:
Figure BDA0003041960050000231
wherein d is the first party random number mixed intermediate parameter, WX1For the first party local model prediction value, WX2Is the second-party local model prediction value, y is the sample label, r1The preset first party random number is the first party random number, and the mixed value of the first party random numbers is WX1+r1。
And step B313, encrypting the first party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the first party random number encrypted intermediate parameter.
In this embodiment, it should be noted that the private key in the preset second party key pair is only held by the second device, and both the first device and the second device hold the public key in the preset second party key pair.
Step B32, based on the first party random number encryption intermediate parameter, exchanging encryption intermediate parameter with the first device to obtain the second party random number encryption intermediate parameter.
In this embodiment, based on the first party nonce encryption intermediate parameter, perform encryption intermediate parameter exchange with the first device to obtain the second party nonce encryption intermediate parameter, specifically, the second device sends the first party nonce encryption intermediate parameter to the first device, so that the first device receives the first party nonce encryption intermediate parameter, the first device sends the second party nonce encryption intermediate parameter to the second device, so that the second device receives the second party nonce encryption intermediate parameter, where it needs to be noted that, since the second party nonce encryption intermediate parameter is encrypted based on the public key in the preset first party secret key, and the private key in the preset first party secret key is only held by the first device, the second device cannot decrypt the second party nonce encryption intermediate parameter, and because the first party random number encryption intermediate parameter is encrypted based on the public key in the preset second party secret key, and the private key in the preset second party secret key is only held by the second device, the first device cannot decrypt the first party random number encryption intermediate parameter.
And step B40, encrypting the intermediate parameters based on the second party random number, optimizing the local model to be trained of the second party, and obtaining a second party federal model.
In this embodiment, the local model to be trained of the second party is optimized based on the second party random number encryption intermediate parameter to obtain a second party federation model, specifically, the second party random number encryption intermediate parameter is subjected to random number elimination to obtain a second party encryption intermediate parameter, and then the model parameter of the local model to be trained of the second party is updated based on a second party encryption model gradient generated by the second party encryption intermediate parameter to obtain a second party federation model.
The step of optimizing the local model to be trained of the second party based on the second party random number encryption intermediate parameter to obtain the second party federal model comprises the following steps:
step B41, carrying out random number elimination on the second party random number encryption intermediate parameter to obtain a second party encryption intermediate parameter;
in this embodiment, the second party random number encryption intermediate parameter is subjected to random number elimination to obtain a second party encryption intermediate parameter, specifically, the second party random number encryption intermediate parameter is subjected to random number elimination by calculating a difference between the second party random number encryption intermediate parameter and a preset second party random number to obtain a second party encryption intermediate parameter, where the process of performing random number elimination on the second party random number encryption intermediate parameter is as follows:
[[d]]2＝[[d2]]-r2
Figure BDA0003041960050000241
wherein [ [ d ]]]2Encrypting an intermediate parameter, [ [ d ] for the second party2]]Encrypting an intermediate parameter, WX, for a second party random number1For the first party local model prediction value, WX2Is the second-party local model prediction value, y is the sample label, r2And setting the preset second party random number.
Step B42, generating a second party encryption gradient based on the second party encryption intermediate parameter and a second party local sample corresponding to the second party local model prediction value;
in this embodiment, a second-party encryption gradient is generated based on the second-party encryption intermediate parameter and a second-party local sample corresponding to the second-party local model prediction value, specifically, the second-party encryption gradient is generated by calculating a product between the second-party encryption intermediate parameter and the second-party local sample corresponding to the second-party local model prediction value, where a specific process of generating the second-party encryption gradient is as follows:
[[g]]2＝[[d]]2*X
wherein [ [ g ]]]2Encrypting a gradient for the second party [ [ d ]]]2Encrypting intermediate parameters for the second party, X being the second party local sample.
And step B43, optimizing the local model to be trained of the second party based on the encryption gradient of the second party to obtain the federal model of the second party.
In this embodiment, the local model to be trained of the second party is optimized based on the encryption gradient of the second party to obtain the federal model of the second party, and specifically, the model parameters of the local model to be trained of the second party are updated by a preset model parameter updating method based on the encryption gradient of the second party to obtain the federal model of the second party, where the preset model parameter updating method includes a gradient descent method, a gradient ascent method, and the like.
The step of optimizing the local model to be trained of the second party based on the encryption gradient of the second party and obtaining the federated model of the second party comprises the following steps:
step B431, based on the second party encryption gradient, updating model parameters of the second party local model to be trained, and judging whether the second party local model to be trained after model parameter updating meets preset training end conditions;
in this embodiment, it should be noted that the preset training end includes loss convergence, reaching the maximum number of iterations, and the like.
Step B432, if the local model is satisfied, taking the second-party local model to be trained after the model parameters are updated as the second-party federal model;
and B433, if the local model prediction value does not meet the requirement, returning to execute the step of obtaining the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.
In this embodiment, if the second-party to-be-trained local model after the model parameter update meets a preset training end condition, directly using the second-party to-be-trained local model after the model parameter update as the second-party federation model, and if the second-party to-be-trained local model after the model parameter update does not meet the preset training end condition, returning to execute the step of obtaining the second-party local model predicted value corresponding to the second-party to-be-trained local model and the sample label corresponding to the second-party local model predicted value.
In the whole federal learning process of the embodiment of the application, the first party random number encryption value, the second party random number encryption value, the first party random number encryption intermediate parameter and the second party random number encryption intermediate parameter which are exchanged between the first device and the second device are in homomorphic encryption states, and for the first party random number encryption value, the second device can decrypt but still ensure the privacy of data due to the mixing of random numbers, while for the second party random number encryption value, the first device can decrypt but also ensure the privacy of data due to the mixing of random numbers, while the first party random number encryption intermediate parameter held by the first device is encrypted by the public key in the preset second party key pair, the first device cannot decrypt, the second device holds the second party random number encryption intermediate parameter and is encrypted by the public key in the preset first party key pair, the unable deciphering of second equipment, and in whole federal learning process, gradient and model parameter in the model in first equipment and the second equipment all are in homomorphic encryption state, the data of the own side of first equipment and the unable deciphering of second equipment, and then realized guaranteeing the purpose of data privacy nature in the federal learning process, and because the data of transmission is encrypted data between first equipment and the second equipment, first equipment and second equipment need not the assistance of the other side and carry out the data decryption at the federal learning in-process, and then do not have the decryption interface at whole federal learning in-process, therefore, the security in the federal learning process has been promoted.
The embodiment of the application provides a federated learning modeling optimization method, compared with the prior art that a label owner who participates in federated learning generally needs to send homomorphic encrypted labels to a label owner without labels, and then the label owner calculates homomorphic encrypted gradient and homomorphic encrypted loss based on homomorphic encrypted labels, and sends homomorphic encrypted gradient and homomorphic encrypted loss to the label owner, and then the label owner decrypts the homomorphic encrypted gradient and homomorphic encrypted loss, feeds back the model gradient and model loss of a plaintext state to the technical means of the characteristic owner for updating the model, and in the embodiment of the application, firstly, a second party local model predicted value corresponding to a local model to be trained by a second party and a sample label corresponding to the second party local model predicted value are obtained, and then, based on the pair, the second party local model predicted value and the sample label are subjected to random number encryption to generate a second party random number and are added with a random number The secret value is encrypted with first equipment to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by the first equipment through random number encryption on a first party local model predicted value corresponding to a first party local model to be trained so as to be obtained by the first equipment, and then a first party random number encryption intermediate parameter generated based on the first party random number encrypted value is exchanged with the first equipment through encryption intermediate parameter to obtain a second party random number encryption intermediate parameter, wherein the second party random number encryption intermediate parameter is generated by the first equipment based on the second party random number encrypted value so as to be used by the first equipment for optimizing the first party local model to be trained based on the first party random number encryption intermediate parameter to obtain a first party federated model, and then, based on the second party random number encryption intermediate parameter, optimizing the second party local model to be trained to obtain a second party federal model, wherein in the whole interaction process of constructing the federal model, data mutually transmitted between the first equipment and the second equipment are all encrypted data, so that a decryption interface does not exist in the process of constructing the federal model in the embodiment of the application, and the technical defect that in the prior art, due to the fact that the decryption interface exists in the whole federal learning process of constructing the model, private data of a label owner can be collected by a characteristic owner through the decryption interface, and then a sample label serving as the private data of the label owner is leaked is overcome, and therefore, the safety of constructing the model based on the federal learning is improved.
Further, referring to fig. 6, based on the third embodiment of the present application, in another embodiment of the present application, the first party federal model includes first party cryptographic model parameters encrypted by a public key of a preset second party key pair, the second party federal model includes second party cryptographic model parameters encrypted by a public key of a preset first party key pair,
after the step of optimizing the local model to be trained of the second party and obtaining the federal model of the second party based on the encrypted intermediate parameter of the random number of the second party, the federal learning modeling optimization method further includes:
step C10, obtaining a second party sample to be predicted, and generating a second party encryption model output value based on the second party sample to be predicted and the second party encryption model parameter;
in this embodiment, it should be noted that the second party encryption model parameter is a model parameter encrypted based on a public key in a preset first party key pair.
And acquiring a sample to be predicted of a second party, generating an output value of an encryption model of the second party based on the sample to be predicted of the second party and the encryption model parameter of the second party, specifically acquiring the sample to be predicted of the second party, and calculating a product between the sample to be predicted of the second party and the encryption model parameter of the second party to obtain the output value of the encryption model of the second party.
Step C20, sending the second party encryption model output value and the target sample ID of the second party to-be-predicted sample to the first device, so that the first device generates a first party encryption model output value based on the first party to-be-predicted sample corresponding to the target sample ID and the first party encryption model parameter, and decrypts the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party encryption model output value;
in this embodiment, the second party cryptographic model output value and the target sample ID of the second party sample to be predicted are sent to the first device, so that the first device generates a first party cryptographic model output value based on the first party sample to be predicted corresponding to the target sample ID and the first party cryptographic model parameter, decrypts the second party cryptographic model output value based on a private key in the preset first party key pair, and obtains a second party model output value, specifically, the second party cryptographic model output value and the target sample ID of the second party sample to be predicted are sent to the first device, so that the first device obtains the first party predicted sample corresponding to the target sample ID, and obtains the first party cryptographic model output value by calculating a product between the first party predicted sample and the first party cryptographic model parameter, and decrypting the output value of the second party encryption model based on a private key in a preset first party key pair to obtain the output value of the second party model, and then sending the output value of the second party model and the output value of the first party encryption model to the second equipment by the first equipment.
Step C30, receiving the first party encryption model output value and the second party encryption model output value sent by the first device, and decrypting the first party encryption model output value based on the private key in the preset second party key pair to obtain a first party model output value;
in this embodiment, it should be noted that, because the first-party encryption model parameter is encrypted based on the public key in the preset second key pair, and further based on the private key in the preset second-party key pair, the first-party encryption model output value can be decrypted to obtain the first-party model output value.
And step C40, generating a federal prediction result based on the first-party model output value and the second-party model output value.
In this embodiment, a federal prediction result is generated based on the first-party model output value and the second-party model output value, and specifically, a federal prediction result is obtained by aggregating the first-party model output value and the second-party model output value based on a preset aggregation manner, where the preset aggregation manner includes weighted summation and weighted averaging, and fig. 7 is a schematic flow chart of generating the federal prediction result by the second device in the embodiment of the present application, where a is the first device, B is the second device, X1, X2, X3, X4 and X5 are characteristics of samples, y is sample labels, and [ [ w ] ] in a is the first-party cryptographic model parameter, and [ [ w ] ] in a is the first-party cryptographic model output value, that is, [ [ w ] ] [ × a, and [ w ] ] in B is the second-party cryptographic model parameter, and the [ w ] ]xin the B is the output value of the second party encryption model, namely the [ w ] ]x _ B, the ID is the target sample ID, the complete wx is the federal prediction result, and the federal prediction result is converted into a prediction score.
The embodiment of the application provides a federated prediction method, wherein, because a first federated model and a second federated model are constructed based on a federated learning process without decryption interfaces, and then model parameters in first equipment and second equipment are all in a homomorphic encryption state, so as to obtain a sample to be predicted by a second party, and based on the sample to be predicted by the second party and the second party encryption model parameters, a second party encryption model output value is generated, and the second party encryption model output value and a target sample ID of the sample to be predicted by the second party are sent to the first equipment, so that the first equipment generates a first party encryption model output value based on the first party sample to be predicted corresponding to the target sample ID and the first party encryption model parameters, and decrypts the second party encryption model output value based on a private key in a preset first party key pair, and obtaining a second-party model output value, further receiving a first-party encryption model output value and a second-party model output value sent by the first equipment, decrypting the first-party encryption model output value based on a private key in the preset second-party key pair, obtaining a first-party model output value, further generating a federal prediction result based on the first-party model output value and the second-party model output value, achieving the purpose of carrying out federal prediction based on a federal model without a decryption interface, further achieving the purpose of carrying out federal prediction based on a higher-security federal model, and improving the security of the federal prediction.
Referring to fig. 8, fig. 8 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present application.
As shown in fig. 8, the federal learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used for realizing connection communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Optionally, the federal learning modeling optimization device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The rectangular user interface may comprise a Display screen (Display), an input sub-module such as a Keyboard (Keyboard), and the optional rectangular user interface may also comprise a standard wired interface, a wireless interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface).
Those skilled in the art will appreciate that the federated learning modeling optimization facility architecture illustrated in FIG. 8 does not constitute a limitation of the federated learning modeling optimization facility, and may include more or fewer components than those illustrated, or some components in combination, or a different arrangement of components.
As shown in fig. 8, the memory 1005, which is a type of computer storage medium, may include an operating system, a network communication module, and a federal learning modeling optimization program. The operating system is a program for managing and controlling hardware and software resources of the Federal learning modeling optimization equipment and supports the operation of the Federal learning modeling optimization program and other software and/or programs. The network communication module is used for realizing communication among components in the memory 1005 and communication with other hardware and software in the federal learning modeling optimization system.
In the federated learning modeling optimization apparatus shown in fig. 8, the processor 1001 is configured to execute a federated learning modeling optimization program stored in the memory 1005 to implement the steps of any of the federated learning modeling optimization methods described above.
The specific implementation of the federal learning modeling optimization device of the application is basically the same as that of each embodiment of the federal learning modeling optimization method, and details are not repeated herein.
The embodiment of the present application further provides a federal learning modeling optimization device, which is applied to the first device, and includes:
the acquisition module is used for acquiring a first party local model predicted value corresponding to a first party local model to be trained;
the encrypted value exchange module is used for exchanging an encrypted value with a second device based on a first party random number encrypted value generated by encrypting a random number on the first party local model predicted value to obtain a second party random number encrypted value, wherein the second party random number encrypted value is obtained by encrypting the random number on a second party local model predicted value and a sample tag corresponding to a second party local model to be trained by the second device to obtain the first party random number encrypted value by the second device;
an encrypted intermediate parameter exchange module, configured to exchange an encrypted intermediate parameter with the second device based on a second-party random number encrypted intermediate parameter generated by the second-party random number encrypted value, to obtain a first-party random number encrypted intermediate parameter, where the first-party random number encrypted intermediate parameter is generated by the second device based on the first-party random number encrypted value, so that the second device optimizes the second-party local model to be trained based on the second-party random number encrypted intermediate parameter, and obtains a second-party federated model;
and the optimization module is used for optimizing the local model to be trained of the first party based on the encrypted intermediate parameters of the random number of the first party to obtain the federal model of the first party.
Optionally, the cryptographic value exchange module is further configured to:
based on a preset first party random number and a public key in a preset second party key pair, carrying out random number encryption on the first party local model predicted value to obtain a first party random number encrypted value;
and exchanging encryption values with the second equipment based on the first party random number encryption value to obtain the second party random number encryption value.
Optionally, the predicted value of the local model of the first party is homomorphic encrypted based on the public key in the preset second party key pair, so as to obtain the predicted value of the homomorphic encryption model of the first party;
and mixing the predicted value of the homomorphic encryption model of the first party with the preset random number of the first party to generate the encrypted value of the random number of the first party.
Optionally, the encrypted intermediate parameter exchange module is further configured to:
generating a second party random number encryption intermediate parameter based on a private key and a public key of a preset first party key pair, the second party random number encryption value and the first party local model prediction value;
and exchanging encryption intermediate parameters with the second equipment based on the second party random number encryption intermediate parameters to obtain the first party random number encryption intermediate parameters.
Optionally, the encrypted intermediate parameter exchange module is further configured to:
decrypting the second party random number encrypted value based on a private key in the preset first party key pair to obtain a second party random number mixed value;
generating a second party random number mixing intermediate parameter based on the first party local model predicted value and the second party random number mixing value;
and encrypting the second party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the second party random number encrypted intermediate parameter.
Optionally, the optimization module is further configured to:
carrying out random number elimination on the first party random number encryption intermediate parameter to obtain a first party encryption intermediate parameter;
generating a first party encryption gradient based on the first party encryption intermediate parameter and a first party local sample corresponding to the first party local model prediction value;
and optimizing the local model to be trained of the first party based on the encryption gradient of the first party to obtain the federal model of the first party.
Optionally, the optimization module is further configured to:
updating model parameters of the first party local model to be trained based on the first party encryption gradient, and judging whether the first party local model to be trained after model parameter updating meets preset training end conditions;
if so, taking the local model to be trained of the first party after the model parameters are updated as the federal model of the first party;
and if not, returning to execute the step of obtaining the first party local model predicted value corresponding to the first party local model to be trained.
Optionally, the federal learning modeling optimization device is further configured to:
obtaining a first party to-be-predicted sample, and generating a first party encryption model output value based on the first party to-be-predicted sample and the first party encryption model parameter;
sending the first party encryption model output value and the target sample ID of the first party to-be-predicted sample to the second device, so that the second device generates a second party encryption model output value based on a second party to-be-predicted sample corresponding to the target sample ID and the second party encryption model parameter, and decrypts the first party encryption model output value based on a private key in the preset second party key pair to obtain a first party model output value;
receiving a second party encryption model output value and a first party model output value sent by the second device, and decrypting the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party model output value;
and generating a federal prediction result based on the first-party model output value and the second-party model output value.
The specific implementation of the federal learning modeling optimization device of the application is basically the same as that of each embodiment of the federal learning modeling optimization method, and details are not repeated herein.
The embodiment of the present application further provides a federal learning modeling optimization device, the federal learning modeling optimization device is applied to the second device, the federal learning modeling optimization device includes:
the acquisition module is used for acquiring a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value;
the encrypted value exchange module is used for exchanging an encrypted value with first equipment based on a second party random number encrypted value generated by encrypting a random number on the second party local model predicted value and the sample tag to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by encrypting the random number on the first party local model predicted value corresponding to the local model to be trained of the first party by the first equipment so as to be used for the first equipment to obtain the second party random number encrypted value;
the encryption intermediate parameter exchange module is used for exchanging encryption intermediate parameters with the first equipment based on a first party random number encryption intermediate parameter generated by the first party random number encryption value to obtain a second party random number encryption intermediate parameter, wherein the second party random number encryption intermediate parameter is generated by the first equipment based on the second party random number encryption value, so that the first equipment optimizes the local model to be trained by the first party based on the first party random number encryption intermediate parameter to obtain a first party federal model;
and the optimization module is used for optimizing the local model to be trained of the second party based on the encrypted intermediate parameters of the random number of the second party to obtain the federal model of the second party.
Optionally, the cryptographic value exchange module is further configured to:
based on a preset second party random number and a public key in a preset first party key pair, carrying out random number encryption on the second party local model predicted value and the sample tag to obtain a second party random number encrypted value;
and exchanging encryption values with the first equipment based on the second party random number encryption value to obtain the first party random number encryption value.
Optionally, the cryptographic value exchange module is further configured to:
homomorphic encryption is carried out on the second party local model predicted value based on the public key in the preset first party key pair, and a second party homomorphic encryption model predicted value is obtained;
and mixing the predicted value of the homomorphic encryption model of the second party, the sample tag and the preset random number of the second party to generate the encrypted value of the random number of the second party.
Optionally, the encrypted intermediate parameter exchange module is further configured to:
generating a first party random number encryption intermediate parameter based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model prediction value and the sample label;
and exchanging encryption intermediate parameters with the first equipment based on the first party random number encryption intermediate parameters to obtain second party random number encryption intermediate parameters.
Optionally, the encrypted intermediate parameter exchange module is further configured to:
decrypting the first party random number encrypted value based on a private key in the preset second party key pair to obtain a first party random number mixed value;
generating a first party random number hybrid intermediate parameter based on the second party local model prediction value, the first party random number hybrid value and the sample label;
and encrypting the first party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the first party random number encrypted intermediate parameter.
Optionally, the optimization module is further configured to:
carrying out random number elimination on the second party random number encryption intermediate parameter to obtain a second party encryption intermediate parameter;
generating a second party encryption gradient based on the second party encryption intermediate parameters and second party local samples corresponding to the second party local model prediction values;
and optimizing the local model to be trained of the second party based on the encryption gradient of the second party to obtain the federal model of the second party.
Optionally, the optimization module is further configured to:
updating model parameters of the local model to be trained of the second party based on the encryption gradient of the second party, and judging whether the local model to be trained of the second party after the model parameters are updated meets a preset training end condition;
if so, taking the second party local model to be trained after the model parameters are updated as the second party federal model;
and if not, returning to execute the step of obtaining the second party local model predicted value corresponding to the second party local model to be trained and the sample label corresponding to the second party local model predicted value.
Optionally, the federal learning modeling optimization device is further configured to:
obtaining a second party to-be-predicted sample, and generating a second party encryption model output value based on the second party to-be-predicted sample and the second party encryption model parameter;
sending the second party encryption model output value and the target sample ID of the second party to-be-predicted sample to the first device, so that the first device generates a first party encryption model output value based on the first party to-be-predicted sample corresponding to the target sample ID and the first party encryption model parameter, and decrypts the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party model output value;
receiving a first party encryption model output value and a second party encryption model output value sent by the first device, and decrypting the first party encryption model output value based on a private key in the preset second party key pair to obtain a first party model output value;
and generating a federal prediction result based on the first-party model output value and the second-party model output value.
The specific implementation of the federal learning modeling optimization device of the application is basically the same as that of each embodiment of the federal learning modeling optimization method, and details are not repeated herein.
The present application provides a medium, which is a readable storage medium, and the readable storage medium stores one or more programs, and the one or more programs are further executable by one or more processors for implementing the steps of any one of the above methods for federally learned modeling optimization.
The specific implementation of the readable storage medium of the application is substantially the same as that of each embodiment of the federated learning modeling optimization method, and is not described herein again.
The present application provides a computer program product, and the computer program product includes one or more computer programs, which can also be executed by one or more processors for implementing the steps of any of the above methods for federated learning modeling optimization.
The specific implementation of the computer program product of the present application is substantially the same as the embodiments of the federated learning modeling optimization method described above, and is not described herein again.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims (19)

1. The federated learning modeling optimization method is applied to first equipment, and comprises the following steps:
obtaining a first party local model predicted value corresponding to a first party local model to be trained;
performing encryption value exchange with second equipment based on a first party random number encryption value generated by performing random number encryption on the first party local model predicted value to obtain a second party random number encryption value, wherein the second party random number encryption value is obtained by performing random number encryption on a second party local model predicted value and a sample tag corresponding to a second party local model to be trained by the second equipment to be used for the second equipment to obtain the first party random number encryption value;
exchanging encryption intermediate parameters with the second device to obtain first-party random number encryption intermediate parameters, wherein the first-party random number encryption intermediate parameters are generated by the second device based on the first-party random number encryption value, so that the second device optimizes the local model to be trained by the second party based on the second-party random number encryption intermediate parameters to obtain a second-party federated model;
and optimizing the local model to be trained of the first party based on the first party random number encryption intermediate parameters to obtain a first party federal model.
2. The federal learning modeling optimization method of claim 1, wherein the step of obtaining the encrypted value of the random number of the second party by exchanging the encrypted value with the second device based on the encrypted value of the random number of the first party generated by encrypting the predicted value of the local model of the first party comprises:
based on a preset first party random number and a public key in a preset second party key pair, carrying out random number encryption on the first party local model predicted value to obtain a first party random number encrypted value;
and exchanging encryption values with the second equipment based on the first party random number encryption value to obtain the second party random number encryption value.
3. The federal learning modeling optimization method of claim 2, wherein the step of encrypting the predicted value of the local model of the first party by a random number based on a preset random number of the first party and a public key of a preset key pair of the second party to obtain an encrypted value of the random number of the first party comprises:
homomorphic encryption is carried out on the local model predicted value of the first party on the basis of the public key in the preset second party key pair, and the homomorphic encryption model predicted value of the first party is obtained;
and mixing the predicted value of the homomorphic encryption model of the first party with the preset random number of the first party to generate the encrypted value of the random number of the first party.
4. The federal learning modeling optimization method of claim 1, wherein the step of obtaining the first party nonce encryption intermediate parameter by exchanging the second party nonce encryption intermediate parameter generated based on the second party nonce encryption value with the second device comprises:
generating a second party random number encryption intermediate parameter based on a private key and a public key of a preset first party key pair, the second party random number encryption value and the first party local model prediction value;
and exchanging encryption intermediate parameters with the second equipment based on the second party random number encryption intermediate parameters to obtain the first party random number encryption intermediate parameters.
5. The federal learning modeling optimization method of claim 4, wherein the step of generating second-party random number encryption intermediate parameters based on the private key and the public key of the preset first-party key pair, the second-party random number encryption value, and the first-party local model prediction value comprises:
decrypting the second party random number encrypted value based on a private key in the preset first party key pair to obtain a second party random number mixed value;
generating a second party random number mixing intermediate parameter based on the first party local model predicted value and the second party random number mixing value;
and encrypting the second party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the second party random number encrypted intermediate parameter.
6. The federal learning modeling optimization method of claim 1, wherein the step of optimizing the local model to be trained of the first party based on the first party nonce encryption intermediate parameters to obtain a first party federal model comprises:
carrying out random number elimination on the first party random number encryption intermediate parameter to obtain a first party encryption intermediate parameter;
generating a first party encryption gradient based on the first party encryption intermediate parameter and a first party local sample corresponding to the first party local model prediction value;
and optimizing the local model to be trained of the first party based on the encryption gradient of the first party to obtain the federal model of the first party.
7. The federated learning modeling optimization method of claim 6, wherein the step of optimizing the first-party local model to be trained based on the first-party encryption gradient to obtain the first-party federated model comprises:
updating model parameters of the first party local model to be trained based on the first party encryption gradient, and judging whether the first party local model to be trained after model parameter updating meets preset training end conditions;
if so, taking the local model to be trained of the first party after the model parameters are updated as the federal model of the first party;
and if not, returning to execute the step of obtaining the first party local model predicted value corresponding to the first party local model to be trained.
8. The federated learning modeling optimization method of claim 1, wherein the first party federated model includes first party cryptographic model parameters that are encrypted by a public key of a pre-established second party key pair, the second party federated model includes second party cryptographic model parameters that are encrypted by a public key of a pre-established first party key pair,
after the step of optimizing the local model to be trained of the first party and obtaining the federal model of the first party based on the encrypted intermediate parameter of the random number of the first party, the federal learning modeling optimization method further includes:
obtaining a first party to-be-predicted sample, and generating a first party encryption model output value based on the first party to-be-predicted sample and the first party encryption model parameter;
sending the first party encryption model output value and the target sample ID of the first party to-be-predicted sample to the second device, so that the second device generates a second party encryption model output value based on a second party to-be-predicted sample corresponding to the target sample ID and the second party encryption model parameter, and decrypts the first party encryption model output value based on a private key in the preset second party key pair to obtain a first party model output value;
receiving a second party encryption model output value and a first party model output value sent by the second device, and decrypting the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party model output value;
and generating a federal prediction result based on the first-party model output value and the second-party model output value.
9. The federated learning modeling optimization method is applied to second equipment, and comprises the following steps:
obtaining a second party local model predicted value corresponding to a second party local model to be trained and a sample label corresponding to the second party local model predicted value;
exchanging an encrypted value with first equipment based on a second party random number encrypted value generated by encrypting a second party local model predicted value and the sample tag by a random number to obtain a first party random number encrypted value, wherein the first party random number encrypted value is obtained by encrypting the random number of the first party local model predicted value corresponding to the first party local model to be trained by the first equipment to obtain the second party random number encrypted value by the first equipment;
exchanging encryption intermediate parameters with the first device based on the first party random number encryption intermediate parameters generated by the first party random number encryption value to obtain second party random number encryption intermediate parameters, wherein the second party random number encryption intermediate parameters are generated by the first device based on the second party random number encryption value, so that the first device optimizes the local model to be trained by the first party based on the first party random number encryption intermediate parameters to obtain a first party federated model;
and optimizing the local model to be trained of the second party based on the encrypted intermediate parameter of the random number of the second party to obtain a federal model of the second party.
10. The federal learning modeling optimization method of claim 9, wherein the step of obtaining a first-party random number encrypted value based on a second-party random number encrypted value generated by encrypting a random number of the second-party local model predicted value and the sample tag, exchanging encrypted values with a first device, comprises:
based on a preset second party random number and a public key in a preset first party key pair, carrying out random number encryption on the second party local model predicted value and the sample tag to obtain a second party random number encrypted value;
and exchanging encryption values with the first equipment based on the second party random number encryption value to obtain the first party random number encryption value.
11. The federal learning modeling optimization method of claim 10, wherein the step of encrypting the second-party local model predicted value and the sample tag by a random number based on a preset second-party random number and a public key of a preset first-party key pair to obtain a second-party random number encrypted value comprises:
homomorphic encryption is carried out on the second party local model predicted value based on the public key in the preset first party key pair, and a second party homomorphic encryption model predicted value is obtained;
and mixing the predicted value of the homomorphic encryption model of the second party, the sample tag and the preset random number of the second party to generate the encrypted value of the random number of the second party.
12. The federal learning modeling optimization method of claim 9, wherein the step of obtaining the second party nonce encryption intermediate parameter by exchanging the first party nonce encryption intermediate parameter generated based on the first party nonce encryption value with the first device comprises:
generating a first party random number encryption intermediate parameter based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model prediction value and the sample label;
and exchanging encryption intermediate parameters with the first equipment based on the first party random number encryption intermediate parameters to obtain second party random number encryption intermediate parameters.
13. The federal learning modeling optimization method of claim 12, wherein the step of generating a first party random number encryption intermediate parameter based on a private key and a public key of a preset second party key pair, the first party random number encryption value, the second party local model prediction value, and the sample label comprises:
decrypting the first party random number encrypted value based on a private key in the preset second party key pair to obtain a first party random number mixed value;
generating a first party random number hybrid intermediate parameter based on the second party local model prediction value, the first party random number hybrid value and the sample label;
and encrypting the first party random number mixed intermediate parameter based on the public key in the preset first party key pair to obtain the first party random number encrypted intermediate parameter.
14. The federal learning modeling optimization method of claim 9, wherein the step of optimizing the second-party local model to be trained based on the second-party random number encryption intermediate parameters to obtain a second-party federal model comprises:
carrying out random number elimination on the second party random number encryption intermediate parameter to obtain a second party encryption intermediate parameter;
generating a second party encryption gradient based on the second party encryption intermediate parameters and second party local samples corresponding to the second party local model prediction values;
and optimizing the local model to be trained of the second party based on the encryption gradient of the second party to obtain the federal model of the second party.
15. The federal learning modeling optimization method of claim 14, wherein the step of optimizing the second-party to-be-trained local model based on the second-party encryption gradient to obtain the second-party federal model comprises:
updating model parameters of the local model to be trained of the second party based on the encryption gradient of the second party, and judging whether the local model to be trained of the second party after the model parameters are updated meets a preset training end condition;
if so, taking the second party local model to be trained after the model parameters are updated as the second party federal model;
and if not, returning to execute the step of obtaining the second party local model predicted value corresponding to the second party local model to be trained and the sample label corresponding to the second party local model predicted value.
16. The federated learning modeling optimization method of claim 9, wherein the first party federated model includes first party cryptographic model parameters that are encrypted by a public key of a pre-established second party key pair, the second party federated model includes second party cryptographic model parameters that are encrypted by a public key of a pre-established first party key pair,
after the step of optimizing the local model to be trained of the second party and obtaining the federal model of the second party based on the encrypted intermediate parameter of the random number of the second party, the federal learning modeling optimization method further includes:
obtaining a second party to-be-predicted sample, and generating a second party encryption model output value based on the second party to-be-predicted sample and the second party encryption model parameter;
sending the second party encryption model output value and the target sample ID of the second party to-be-predicted sample to the first device, so that the first device generates a first party encryption model output value based on the first party to-be-predicted sample corresponding to the target sample ID and the first party encryption model parameter, and decrypts the second party encryption model output value based on a private key in the preset first party key pair to obtain a second party model output value;
receiving a first party encryption model output value and a second party encryption model output value sent by the first device, and decrypting the first party encryption model output value based on a private key in the preset second party key pair to obtain a first party model output value;
and generating a federal prediction result based on the first-party model output value and the second-party model output value.
17. The Federal learning modeling optimization apparatus is characterized by comprising: a memory, a processor, and a program stored on the memory for implementing the federated learning modeling optimization method,
the memory is used for storing a program for realizing the Federal learning modeling optimization method;
the processor is configured to execute a program implementing the federal learning modeling optimization methodology to implement the steps of the federal learning modeling optimization methodology of any of claims 1 to 8 or 9 to 16.
18. A medium being a readable storage medium, characterized in that the readable storage medium has stored thereon a program for implementing a federal learning modeling optimization method, the program being executed by a processor to implement the steps of the federal learning modeling optimization method as claimed in any one of claims 1 to 8 or 9 to 16.
19. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the steps of the federal learning modeling optimization method as claimed in any of claims 1 to 8 or 9 to 16.
CN202110460157.2A 2021-04-27 2021-04-27 Federal learning modeling optimization method, apparatus, medium, and computer program product Pending CN113222180A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110460157.2A CN113222180A (en) 2021-04-27 2021-04-27 Federal learning modeling optimization method, apparatus, medium, and computer program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110460157.2A CN113222180A (en) 2021-04-27 2021-04-27 Federal learning modeling optimization method, apparatus, medium, and computer program product

Publications (1)

Publication Number Publication Date
CN113222180A true CN113222180A (en) 2021-08-06

Family

ID=77089287

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110460157.2A Pending CN113222180A (en) 2021-04-27 2021-04-27 Federal learning modeling optimization method, apparatus, medium, and computer program product

Country Status (1)

Country Link
CN (1) CN113222180A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114282692A (en) * 2022-03-08 2022-04-05 富算科技(上海)有限公司 Model training method and system for longitudinal federal learning
CN115169576A (en) * 2022-06-24 2022-10-11 上海富数科技有限公司广州分公司 Model training method and device based on federal learning and electronic equipment
WO2023092792A1 (en) * 2021-11-29 2023-06-01 深圳前海微众银行股份有限公司 Optimization method for modeling based on federated learning, and electronic device, storage medium and program product
WO2023216899A1 (en) * 2022-05-13 2023-11-16 北京字节跳动网络技术有限公司 Model performance evaluation method and apparatus, device and medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023092792A1 (en) * 2021-11-29 2023-06-01 深圳前海微众银行股份有限公司 Optimization method for modeling based on federated learning, and electronic device, storage medium and program product
CN114282692A (en) * 2022-03-08 2022-04-05 富算科技(上海)有限公司 Model training method and system for longitudinal federal learning
WO2023216899A1 (en) * 2022-05-13 2023-11-16 北京字节跳动网络技术有限公司 Model performance evaluation method and apparatus, device and medium
CN115169576A (en) * 2022-06-24 2022-10-11 上海富数科技有限公司广州分公司 Model training method and device based on federal learning and electronic equipment
CN115169576B (en) * 2022-06-24 2024-02-09 上海富数科技有限公司 Model training method and device based on federal learning and electronic equipment

Similar Documents

Publication Publication Date Title
CN113222180A (en) Federal learning modeling optimization method, apparatus, medium, and computer program product
CN109255444B (en) Federal modeling method and device based on transfer learning and readable storage medium
WO2021232754A1 (en) Federated learning modeling method and device, and computer-readable storage medium
CN110851869B (en) Sensitive information processing method, device and readable storage medium
US20170359321A1 (en) Secure Data Exchange
CN110633806A (en) Longitudinal federated learning system optimization method, device, equipment and readable storage medium
JP6946561B2 (en) Data statistics methods and equipment
US10587587B2 (en) Information processing apparatus and information processing method
CN112926073A (en) Federal learning modeling optimization method, apparatus, medium, and computer program product
CN113505894A (en) Longitudinal federated learning linear regression and logistic regression model training method and device
CN107196919B (en) Data matching method and device
CN111914038B (en) Federal computing method, apparatus, device, and storage medium
CN113609781B (en) Method, system, equipment and medium for optimizing automobile production die based on federal learning
CN110751294A (en) Model prediction method, device, equipment and medium combining multi-party characteristic data
US20200259819A1 (en) Systems and methods for managing secure sharing of online data
WO2021135793A1 (en) Multi-party secret sharing method and apparatus, and readable storage medium
CA3143855A1 (en) Systems and methods for federated learning on blockchain
CN114168293A (en) Hybrid architecture system and task scheduling method based on data transmission time consumption
US9229687B2 (en) Private two-party computation using partially homomorphic encryption
CN111010283B (en) Method and apparatus for generating information
CN114462626B (en) Federal model training method and device, terminal equipment and storage medium
CN111368314A (en) Modeling and predicting method, device, equipment and storage medium based on cross features
CN114168295A (en) Hybrid architecture system and task scheduling method based on historical task effect
CN115361196A (en) Service interaction method based on block chain network
CN111008236B (en) Data query method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination