CN111368336B - Secret sharing-based training method and device, electronic equipment and storage medium - Google Patents

Secret sharing-based training method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111368336B
CN111368336B CN202010459211.7A CN202010459211A CN111368336B CN 111368336 B CN111368336 B CN 111368336B CN 202010459211 A CN202010459211 A CN 202010459211A CN 111368336 B CN111368336 B CN 111368336B
Authority
CN
China
Prior art keywords
coding
training parameter
encryption
training
random number
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010459211.7A
Other languages
Chinese (zh)
Other versions
CN111368336A (en
Inventor
谭晋
王磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alipay Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Hangzhou Information Technology Co Ltd filed Critical Alipay Hangzhou Information Technology Co Ltd
Priority to CN202010459211.7A priority Critical patent/CN111368336B/en
Publication of CN111368336A publication Critical patent/CN111368336A/en
Application granted granted Critical
Publication of CN111368336B publication Critical patent/CN111368336B/en
Priority to PCT/CN2021/096166 priority patent/WO2021239006A1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/008—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols involving homomorphic encryption
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
    • H04L9/0618—Block ciphers, i.e. encrypting groups of characters of a plain text message using fixed encryption transformation
    • H04L9/0631—Substitution permutation network [SPN], i.e. cipher composed of a number of stages or rounds each involving linear and nonlinear transformations, e.g. AES algorithms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
    • H04L9/0816—Key establishment, i.e. cryptographic processes or cryptographic protocols whereby a shared secret becomes available to two or more parties, for subsequent use
    • H04L9/085—Secret sharing or secret splitting, e.g. threshold schemes

Abstract

The method carries out fixed point number coding on training parameters, trains the coded training parameters, carries out fixed point number interception through one secret sharing in the training process to finish one round of training, updates the training parameters and carries out the next round of training, thereby realizing the training of large-scale data, avoiding the problem of data overflow caused by homomorphic encryption and fixed point number coding, reducing the communication traffic of data in the training process, improving the training efficiency of a model, and protecting the privacy of the data by the secret sharing training method.

Description

Secret sharing-based training method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a training method and apparatus based on secret sharing, an electronic device, and a storage medium.
Background
Artificial Intelligence (AI) is an emerging scientific technology currently being studied and developed for simulating, extending and extending human Intelligence. Among them, the main research goal of AI is to make a machine competent for some complex tasks that usually require human intelligence to do; in practical applications, it is common to construct a model and then train the model through sample data repeatedly until the model reaches a desired goal.
Disclosure of Invention
The embodiment of the specification provides a training method, a training device, electronic equipment and a storage medium based on secret sharing, the training parameter after coding is trained by conducting fixed point coding on the training parameter, the fixed point is intercepted through secret sharing once in the training process to complete one round of training, the training parameter is updated to conduct the next round of training, therefore, large-scale data training can be achieved, the problem of data overflow caused by homomorphic encryption and fixed point coding is avoided, the communication traffic of data in the training process is reduced, the training efficiency of a model is improved, and the secret sharing training method can protect the privacy of the data.
In a first aspect, an embodiment of the present specification provides a secret sharing-based training method, including:
acquiring a first training parameter, performing fixed point number coding on the first training parameter according to a first target multiplying power to obtain a first coding training parameter, and performing homomorphic encryption on the first coding training parameter according to a public key preset at a receiving end to obtain a first encrypted coding training parameter;
acquiring sample data, performing fixed-point number coding on the sample data according to a second target multiplying power to obtain coding sample data, and training the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result;
secret sharing is carried out on the first encryption coding prediction result to obtain a first coding prediction result, and fixed point interception is carried out on the first coding prediction result according to the first target multiplying power to obtain a second coding prediction result;
obtaining a first coding error according to the second coding prediction result and target label data, and homomorphically encrypting the first coding error according to a preset public key of the receiving end to obtain a first encryption coding error;
and updating the first encryption coding training parameter according to the first encryption coding error, then carrying out iterative training to obtain a trained second encryption coding training parameter, and obtaining a second training parameter according to the second encryption coding training parameter to finish the training of the first training parameter.
In one possible implementation manner, the secret sharing the first encrypted coding prediction result to obtain a first coding prediction result, and performing fixed-point truncation on the first coding prediction result according to the first target magnification to obtain a second coding prediction result includes:
creating an arbitrary random number, and performing fixed-point number coding on the random number to obtain a first coded random number of a sending end, wherein the coding rate of the first coded random number of the sending end corresponds to the coding rate of the first encryption coding prediction result;
obtaining a second encrypted coding random number of the sending end according to the difference value between the first coding random number of the sending end and the first encrypted coding prediction result, and sending the second encrypted coding random number of the sending end to a receiving end, so that the receiving end decrypts the second encrypted coding random number of the sending end to obtain the second coding random number of the sending end;
and receiving a second coded random number of the sending end sent by the receiving end, respectively carrying out fixed point number interception on the first coded random number of the sending end and the second coded random number of the sending end according to the first target multiplying power to obtain a third coded random number of the sending end and a fourth coded random number of the sending end, and obtaining a second coding prediction result according to the sum of the third coded random number of the sending end and the fourth coded random number of the sending end.
In one possible implementation manner, the obtaining a first coding error according to the second coding prediction result and the target tag data includes:
acquiring label data, and performing fixed-point number coding on the label data according to the second target multiplying power to obtain coded label data;
and obtaining a first coding error according to the second coding prediction result and the difference value of the coding label data.
In one possible implementation manner, the updating the first encryption coding training parameter according to the first encryption coding error and then performing iterative training to obtain a trained second encryption coding training parameter includes:
obtaining a first encryption coding gradient according to the product of the first encryption coding error and the coding sample data;
and updating the first encryption coding training parameter according to the first encryption coding gradient, and performing iterative training according to the updated first encryption coding training parameter to obtain a trained second encryption coding training parameter.
In one possible implementation manner, the updating the first encryption coding training parameter according to the first encryption coding gradient, and performing iterative training according to the updated first encryption coding training parameter to obtain a trained second encryption coding training parameter includes:
updating the first encryption coding training parameter according to the difference value between the first encryption coding training parameter and the first encryption coding gradient;
performing iterative training according to the updated first encryption coding training parameter and a preset round number, and updating the first encryption coding training parameter after each round of training;
and if the current round number reaches the preset round number, finishing the training and obtaining a trained second encryption coding training parameter.
In one possible implementation manner, the obtaining a second training parameter according to the second encryption coding training parameter includes:
creating an arbitrary random number, obtaining a third encryption coding training parameter according to the difference value between the second encryption coding training parameter and the random number, and sending the third encryption coding training parameter to a receiving end, so that the receiving end decrypts the third encryption coding training parameter to obtain a third coding training parameter;
receiving a third coding training parameter returned by the receiving end, and obtaining a second coding training parameter according to the sum of the third coding training parameter and the random number;
and intercepting the fixed point number of the second coding training parameter according to the first target multiplying power to obtain a second training parameter.
In the training method based on secret sharing, the training parameters after being coded are trained by carrying out fixed point number coding on the training parameters, and fixed point number interception is carried out through one secret sharing in the training process to complete one round of training, and the training parameters are updated to carry out the next round of training, so that the training of large-scale data can be realized, the problem of data overflow caused by homomorphic encryption and fixed point number coding is avoided, the data communication traffic in the training process is reduced, and the training efficiency of the model is improved.
In a second aspect, an embodiment of the present specification provides a secret sharing-based training device, including:
the encoding module is used for acquiring a first training parameter, performing fixed point number encoding on the first training parameter according to a first target multiplying power to obtain a first encoded training parameter, and performing homomorphic encryption on the first encoded training parameter according to a public key preset at a receiving end to obtain a first encrypted encoded training parameter;
the training module is used for acquiring sample data, performing fixed-point number coding on the sample data according to a second target multiplying power to obtain coding sample data, and training the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result;
the secret sharing module is used for carrying out secret sharing on the first encryption coding prediction result to obtain a first coding prediction result, and carrying out fixed point interception on the first coding prediction result according to the first target multiplying power to obtain a second coding prediction result;
the encryption module is used for obtaining a first coding error according to the second coding prediction result and the target label data, and homomorphically encrypting the first coding error according to a preset public key of the receiving end to obtain a first encryption coding error;
and the output module is used for updating the first encryption coding training parameter according to the first encryption coding error and then carrying out iterative training to obtain a trained second encryption coding training parameter, and obtaining a second training parameter according to the second encryption coding training parameter to finish the training of the first training parameter.
In one possible implementation manner, the secret sharing module includes:
the device comprises a creating unit, a calculating unit and a calculating unit, wherein the creating unit is used for creating an arbitrary random number and carrying out fixed point number coding on the random number to obtain a first coding random number of a sending end, and the coding multiplying power of the first coding random number of the sending end corresponds to the coding multiplying power of the first encryption coding prediction result;
the interactive unit is used for obtaining a second encrypted coding random number of the sending end according to the difference value between the first coding random number of the sending end and the first encrypted coding prediction result, and sending the second encrypted coding random number of the sending end to a receiving end, so that the receiving end decrypts the second encrypted coding random number of the sending end to obtain the second coding random number of the sending end;
and the output unit is used for receiving the second coded random number of the sending end sent by the receiving end, respectively carrying out fixed point interception on the first coded random number of the sending end and the second coded random number of the sending end according to the first target multiplying power to obtain a third coded random number of the sending end and a fourth coded random number of the sending end, and obtaining a second coding prediction result according to the sum of the third coded random number of the sending end and the fourth coded random number of the sending end.
In one possible implementation manner, the encryption module includes:
the acquisition unit is used for acquiring label data and carrying out fixed point number coding on the label data according to the second target multiplying power to obtain coded label data;
and the calculating unit is used for obtaining a first coding error according to the second coding prediction result and the difference value of the coding label data.
In one possible implementation manner, the output module includes:
a calculation unit, configured to obtain a first encryption coding gradient according to a product of the first encryption coding error and the coding sample data;
and the updating unit is used for updating the first encryption coding training parameter according to the first encryption coding gradient and performing iterative training according to the updated first encryption coding training parameter to obtain a trained second encryption coding training parameter.
In one possible implementation manner, the updating unit includes:
the updating subunit is configured to update the first encryption coding training parameter according to a difference between the first encryption coding training parameter and the first encryption coding gradient;
the iteration subunit is used for performing iteration training according to the updated first encryption coding training parameter and a preset number of rounds, and updating the first encryption coding training parameter after each round of training;
and the output subunit is used for finishing the training and obtaining a trained second encryption coding training parameter if the current round number reaches the preset round number.
In one possible implementation manner, the output module includes:
the creating unit is used for creating an arbitrary random number, obtaining a third encryption coding training parameter according to the difference value between the second encryption coding training parameter and the random number, and sending the third encryption coding training parameter to a receiving end, so that the receiving end decrypts the third encryption coding training parameter to obtain a third coding training parameter;
the receiving unit is used for receiving a third coding training parameter returned by the receiving end and obtaining a second coding training parameter according to the sum of the third coding training parameter and the random number;
and the output unit is used for intercepting the fixed point number of the second coding training parameter according to the first target multiplying power to obtain a second training parameter.
In a third aspect, an embodiment of the present specification provides an electronic device, including:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor calling the program instructions to be able to perform the method provided by the first aspect.
In a fourth aspect, embodiments of the present specification provide a non-transitory computer-readable storage medium storing computer instructions which, when executed on a computer, cause the computer to perform the method provided by the first aspect.
It should be understood that the second to fourth aspects of the present description are consistent with the technical solutions of the first aspect of the present description, and similar advantageous effects are obtained in each aspect and the corresponding possible implementation manners, and are not described again.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a flow chart of one embodiment of a secret sharing based training method of the present description;
FIG. 2 is a flow diagram of one embodiment of secret sharing for the present description;
FIG. 3 is a flow diagram of one embodiment of the present description of calculating an encryption coding error;
FIG. 4 is a flow diagram illustrating one embodiment of obtaining second training parameters;
FIG. 5 is a schematic diagram of an embodiment of a secret sharing-based training apparatus according to the present disclosure;
FIG. 6 is a schematic diagram of another embodiment of a secret sharing-based training apparatus according to the present disclosure;
FIG. 7 is a schematic diagram of a secret sharing-based training apparatus according to still another embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a secret sharing-based training apparatus according to still another embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of an embodiment of an electronic device in the present specification.
Detailed Description
For better understanding of the technical solutions in the present specification, the following detailed description of the embodiments of the present specification is provided with reference to the accompanying drawings.
It should be understood that the described embodiments are only a few embodiments of the present specification, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step are within the scope of the present specification.
The terminology used in the embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the specification. As used in the specification examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
At present, when a federated learning model is trained, each party of federated learning needs to keep secret respective data, so Homomorphic Encryption (HE) is used in the communication process of each party, the homomorphic encryption has the effect of processing homomorphic encrypted data to obtain an output, decrypting the output, and the result is consistent with the output result obtained by processing unencrypted original data by the same method; however, in the process of model training, the calculation of decimal is involved, and homomorphic encryption can only be performed for integers, so that homomorphic encryption can be performed after decimal is converted into integer; the decimal conversion into integer can be usually solved by fixed point number coding, that is, the decimal can be multiplied by a fixed-size rate value, so that the decimal is amplified, and an integer part of the amplified number is taken, for example, assuming that one decimal is 0.0003, the fixed point number coding can obtain Encode (0.0003) =0.0003 × 10000=3, wherein the coding rate can be 10000; however, in a model of Linear Regression (LR), a large number of multiplications are usually performed, and these multiplications may cause overflow of values in fixed-point codes, for example, assuming that there are two decimals 0.003 and 0.1, a fixed-point code of 0.003 may be Encode (0.003) =0.003 =10000 =30, and a fixed-point code of 0.1 may be Encode (0.1) =0.1 =10000 =1000, but Encode (0.003 = 0.1) =0.0003 =10000 =3, and Encode (0.003) = Encode (0.1) =30 =1000 =30000, which are different by a factor of 10000; therefore, the fixed point number coded data after multiplication can be processed, and data overflow is avoided; for example, in the above example, the result 30000 of multiplying two fixed-point-number data may be divided by the encoding magnification value, i.e., 30000/10000=3, whereby the two results may be made coincident; in summary, in the training process of the LR model, because multiple rounds of iterative training are performed, each round of iteration requires a large number of multiplication operations, and each multiplication operation requires conversion and processing between fixed point number coding and homomorphic encryption, the operation amount is large, and the data transmission efficiency is low, so that the training efficiency is also reduced.
Fig. 1 is a flowchart of an embodiment of a secret sharing based training method according to the present disclosure, and as shown in fig. 1, the secret sharing based training method may include:
step 101, obtaining a first training parameter, coding the first training parameter according to a first target multiplying power to obtain a first coding training parameter, and homomorphically encrypting the first coding training parameter according to a public key preset at a receiving end to obtain a first encrypted coding training parameter.
Specifically, the first training parameter Wa may be a training parameter of a local federal learning model, and the first training parameter Wa may be optimized through training of the federal learning model, so that a trained second training parameter Wa may be obtained’(ii) a The federate learning model is trained by a plurality of participants together, so that other participants can optimize the training parameters of the participants, and each participant can finally obtain the optimal training parameters; taking training of both parties as an example, the federal learning model may correspond to a transmitting end and a receiving end, and the transmitting end (local part) may train the first training parameter Wa and the receiving end (opposite part) may train its own training parameter Wb.
It should be noted that, for the federal learning model, the local part may be a sending end, and any other participating party may be a receiving end; for the model in which only two parties participate, the local part is the sending end, and the other party is the receiving end.
Then, fixed point coding may be performed on the first training parameter Wa at the transmitting end to obtain a first coded training parameter, where the fixed point coding may be to multiply the first training parameter Wa by a preset multiplying factor scale to obtain an integer part of the first coded training parameter, and discard a fractional part.
In a specific implementation, the first training parameter Wa may be multiplied by 2 same scale values, so that a first encoded training parameter E2 (Wa) may be obtained; wherein, assuming fixed point number code E (X) = X × scale, E2 (X) = X × scale, E3 (X) = X × scale, here, E2 encoding of first training parameter Wa may be performed, i.e., E2 (Wa) = Wa scale × scale, wherein the scale may be a predetermined multiplier value, e.g., scale =100000000 if preset scale = 10000; for convenience of illustration, hereinafter, E encoding refers to E (X), E2 encoding refers to E2 (X), and E3 encoding refers to E3 (X).
Then, the transmitting end can preset the public key Pk according to the receiving endbHomomorphic encryption is performed on the first encrypted training parameter E2 (Wa) to obtain a first encrypted encoded training parameter [ E2 (Wa)]b(ii) a The sending end and the receiving end can respectively maintain a pair of preset keys, and the sending end maintains a public key Pk of the sending endaAnd private key SkaThe receiving end maintains the public key Pk of the receiving endbAnd private key SkbIt should be noted that the public key Pk of the transmitting endaAnd public key Pk of receiving endbCan be known by the other party; the sending end can be based on the public key PkaData is encrypted and can be encrypted by a private key SkaFor the public key PkaThe encrypted data is decrypted, and similarly, the receiving end can decrypt the encrypted data according to the public key PkbData is encrypted and can be encrypted by a private key SkbFor the public key PkbThe encrypted data is decrypted.
Optionally, the receiving end may also perform E2 encoding on its own training parameter Wb to obtain an encoded training parameter E2 (Wb), and then may preset a public key Pk according to the transmitting endaThe encoded training parameter E2 (Wb) is homomorphic encrypted to obtain the encrypted encoded training parameter [ E2 (Wb) ]]a。
102, obtaining sample data, performing fixed point number coding on the sample data according to a second target multiplying power to obtain coding sample data, and training the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result.
Specifically, when the sender obtains the first encryption coding training parameter [ E2 (Wa)]bThereafter, the first cryptographically encoded training parameter [ E2 (Wa)]bTraining with the transmitting end sample data Xa, the training output can be obtained by encoding the first encryption training parameter [ E2 (Wa)]bAnd carrying out coding calculation with the sending end sample data Xa.
In specific implementation, the sample data Xa may be first E-encoded to obtain encoded sample data E (Xa), that is, E (Xa) = Xa × scale, where the magnification value scale may be consistent with that in step 101; and the encoding sample data E (Xa) and the first encryption encoding training parameter [ E2 (Wa)]bMultiplying the first and second images to obtain a first cryptographically encoded prediction [ E3 (Wa Xa) ]]b(ii) a Among these, [ E3 (Wa Xa) was obtained from the prediction result Pa = Wa Xa]b=[E3（Pa）]bI.e., [ E3 (Wa Xa) ]]bIt may be the sender-side encryption coding prediction result.
Alternatively, the receiving end may also perform the same training on its own sample data Xb, thereby obtaining the receiving end cryptographic prediction result [ E3 (Wb x Xb)]aI.e., [ E3 (Pb)]a(ii) a Wherein, [ E3 (Wb Xb)]a=E（Xb）*[E2（Wb）]a。
And 103, carrying out secret sharing on the first encrypted coding prediction result to obtain a first coding prediction result, and carrying out fixed point interception on the first coding prediction result according to the first target multiplying power to obtain a second coding prediction result.
Specifically, when the transmitting end obtains the first encryption encoding prediction result [ E3 (Pa) ]]bAfter that, the air conditioner is started to work,since the sample data Xa and the training parameter Wa are multiplied after fixed point number encoding, the data may overflow, and the first encrypted code predictor [ E3 (Pa)]bSecret sharing is performed, and fixed point number interception can be performed in the secret sharing domain, thereby avoiding data overflow.
Now, the secret sharing and fixed point number interception are described with reference to fig. 2, as shown in fig. 2, a random number Pa1 may be randomly selected at the transmitting end, and the random number Pa1 is subjected to E3 encoding, so as to obtain a first encoded random number E3 (Pa 1) at the transmitting end, that is, E3 (Pa 1) = Pa1 scale, where the scale value scale may be consistent with that in step 101; then according to the principle of homomorphic encryption, the key can be obtained by [ E3 (Pa)]b-E3 (Pa 1) obtaining the sender second cryptographically encoded random number [ E3 (Pa 2)]bAnd encodes the transmitting end second encryption code random number [ E3 (Pa 2)]bSending the data to a receiving end; wherein, E3 (Pa) = E3 (Pa 1) + E3 (Pa 2); when the receiving end receives the second encrypted and encoded random number [ E3 (Pa 2) ]]bThen can be based on the private key Sk preset by the receiving endbEncoding the random number [ E3 (Pa 2) for the second encryption of the sender]bDecryption is performed, whereby the sender-side second encoded random number E3 (Pa 2) can be obtained.
Further, the receiving end may return the sender-side second encoded random number E3 (Pa 2) to the sending end to allow the sending end to perform fixed point number interception, and after the sending end receives the sender-side second encoded random number E3 (Pa 2), the first encoded prediction result E3 (Pa) may be obtained according to a sum of the sender-side first encoded random number E3 (Pa 1) and the sender-side second encoded random number E3 (Pa 2).
Optionally, the sender may further perform fixed point number interception on the sender first encoded random number E3 (Pa 1) to obtain a third encoded random number E (Pa 1); the fixed point number is intercepted in a mode of reducing according to a preset multiplying power; in a specific implementation, since E3 (Pa 1) = Pa1 × scale, the magnification may be selected to be scale × scale, E3 (Pa 1) may be reduced to obtain E (Pa 1), that is, E (Pa 1) = Pa1 × scale, and an integer portion of E (Pa 1) is cut.
Further, after receiving the second encoded random number E3 (Pa 2) from the sender, the sender may perform fixed point number interception on the second encoded random number E3 (Pa 2) from the sender to obtain a fourth encoded random number E (Pa 2); the fixed point number may be truncated according to the above-described E3 (Pa 1).
Optionally, fixed point number interception of the second encoded random number E3 (Pa 2) at the transmitting end may also be completed at the receiving end, that is, the receiving end performs fixed point number interception on the second encoded random number E3 (Pa 2) at the transmitting end to obtain a fourth encoded random number E (Pa 2) at the transmitting end, and returns the fourth encoded random number E (Pa 2) at the transmitting end to the transmitting end.
After the transmitting end obtains the third encoded random number E (Pa 1) and the fourth encoded random number E (Pa 2), a second encoded prediction result E (Pa) can be obtained according to the sum of the third encoded random number E (Pa 1) and the fourth encoded random number E (Pa 2), that is, E (Pa) = E (Pa 1) + E (Pa 2).
Optionally, the sending end may further perform fixed point truncation on the first encoding prediction result E3 (Pa) to obtain a second encoding prediction result E (Pa).
Optionally, a random number Pb1 may also be randomly selected at the receiving end, and E3 encoding is performed on the random number Pb1 to obtain E3 (Pb 1); then through homomorphic encryption, the product [ E3 (Pb 2)]aAnd combining the [ E3 (Pb 2)]aSending the data to a sending end; wherein, E3 (Pb) = E3 (Pb 1) + E3 (Pb 2); when the sender receives [ E3 (Pb 2)]aThen the transmitter can be preset according to a private key Ska pair [ E3 (Pb 2)]aDecrypting to obtain E3 (Pb 2), performing fixed point number interception on E3 (Pb 2) to obtain E (Pb 2), and returning the E (Pb 2) to a receiving end, wherein the receiving end can obtain an encoding prediction result E (Pb) after fixed point number interception according to the sum of E (Pb 1) and E (Pb 2); the sending end can also return E3 (Pb 2) to the receiving end, when the receiving end receives E3 (Pb 2), the sum E3 (Pb) of E3 (Pb 1) and E3 (Pb 2) can be subjected to fixed point number interception to obtain a coded prediction result E (Pb) after fixed point number interception, and therefore the secret is finishedThe number of fixed points within the shared domain is intercepted.
And 104, obtaining a first coding error according to the second coding prediction result and the target label data, and performing homomorphic encryption on the first coding error according to a preset public key of the receiving end to obtain a first encryption coding error.
Specifically, after the sending end obtains the second coding prediction result E (pa), the sending end training coding error E (Err _ a), that is, the first coding error, may be obtained according to the preset tag data Y; in a specific implementation, E encoding may be performed on the tag data Y to obtain encoded tag data E (Y), that is, E (Y) = Y scale, so as to ensure that the encoding magnifications of E (Y) and E (pa) are consistent; then, a first coding error E (Err _ a), i.e., E (Err _ a) = E (pa) -E (y), is obtained according to a difference between the second coding prediction result E (pa) and the coding tag data E (y).
Now referring to fig. 3, as shown in fig. 3, first, a sending end may perform E coding on tag data Y to obtain coded tag data E (Y), and then may subtract the coded tag data E (Y) from a second coding prediction result E (pa), so as to obtain a first coding error E (Err _ a) = E (pa) -E (Y), and may obtain the first coding error E (Err _ a) = E (pa) -E (Y) according to a receiving end public key PkbThe first coding error E (Err _ a) is homomorphically encrypted, whereby a transmitting-end encryption coding error [ E (Err _ a) is obtained]bI.e. the first encryption coding error.
Optionally, the sending end may also obtain the sending end public key PkaEncrypting the encoded tag data E (Y) to obtain encrypted encoded tag data [ E (Y)]aAnd encoding the encrypted tag data [ E (Y) ]]aSending to the receiving end, when the receiving end receives the encrypted coded label data [ E (Y)]aThen, the receiving end encryption coding error [ E (Err _ b) can be obtained according to the difference value of the receiving end coding prediction result E (Pb) and the sending end encryption coding label data]aWherein [ E (Err _ b)]a= E（Pb）-[E（Y）]a。
And 105, updating the first encryption coding training parameter according to the first encryption coding error, performing iterative training to obtain a trained second encryption coding training parameter, and obtaining a second training parameter according to the second encryption coding training parameter to finish training of the first training parameter.
Specifically, the sending end may first transpose the sample data Xa to obtain a transposed matrix Xa of the sample dataTThe transposed matrix Xa can then be alignedTE encoding is carried out to obtain an encoding transposition matrix E (Xa)T) I.e. E (Xa)T）=XaTScale; wherein, the scale multiplying power can be the same as that in the step 101; when the transmitting end gets the first encryption coding error [ E (Err _ a)]bThen, the transpose matrix E (Xa) can be encodedT) Performing a multiplication operation, thereby obtaining a first cryptographic encoding gradient [ E2 (Ga)]bI.e., [ E2 (Ga)]b=E（XaT）*[ E（Err_a）]b(ii) a Wherein Ga may be a training gradient at the transmitting end, i.e. Ga = XaTErr _ a; when the sender obtains the current first encryption encoding gradient [ E2 (Ga)]bThereafter, the gradient [ E2 (Ga) can be encoded according to the current first encryption]bPerforming iterative training to update the training parameter Wa, wherein the iterative training can be performed according to the following formula:
[E2（Wa）]b ’=[E2（Wa）]b-Z* [E2（Ga）]b，
wherein Z may be a preset weight coefficient, [ E2 (Wa)]bMay be the current cryptographically encoded training parameter, [ E2 (Wa)]b ’May be updated encryption coding training parameters; that is, after a round of training, the gradient can be encoded according to the current first encryption [ E2 (Ga)]bFor the current [ E2 (Wa)]bUpdates are made so that the sender can train parameters [ E2 (Wa) according to the new encryption code]b ’The next round of training is performed, whereby repeated iterations are performed and finally the latest cryptographically encoded training parameters are obtained [ E2 (Wa)]b ’I.e. the second cryptographically encoded training parameter; the number of iterative rounds can be preset, and if the current training round number reaches the preset numberFor a given number of rounds, the training may be terminated and a second cryptographically encoded training parameter [ E2 (Wa) output]b ’As final encryption coding training parameters; the second cryptographically encoded training parameter [ E2 (Wa)]b ’And decrypting and intercepting the fixed point number to obtain Wa, wherein the Wa is the training parameter obtained after training, namely the second training parameter.
Wherein, when the transmitting end obtains the second encryption coding training parameter [ E2 (Wa)]b ’Thereafter, a random number Ra may be selected and [ E2 (Wa) calculated]b ’-value of Ra, resulting in a third cryptographically encoded training parameter [ Ya ]]bI.e., [ Ya)]b=[E2（Wa）]b ’-Ra and reacting the [ Ya]bSending the data to a receiving end, and receiving the data by the receiving end]bThen, the [ Ya ] can be paired according to a private key Skb preset by the receiving end]bDecrypting to obtain the third encoded training parameter Ya, namely Ya = E2 (Wa)’The value of Ya is returned to the sending end, and the value of Ya is added with the value of random number Ra after the sending end receives the value of Ya to obtain a second coding training parameter E2 (Wa)’This E2 (Wa) can then be matched’Performing fixed point number interception to obtain a second training parameter Wa’As shown in fig. 4.
Optionally, the receiving end may also perform iterative training according to the above manner, that is, the receiving end may also transpose its own sample data Xb to obtain a transposed matrix Xb of the sample dataTThen, the transposed matrix Xb can be alignedTE encoding is carried out to obtain an encoding transposition matrix E (Xb)T) And the encoding may be transposed into a matrix E (Xb)T) And receiving end encryption coding error [ E (Err _ b)]aPerforming multiplication operation to obtain receiving end encryption coding gradient [ E2 (Gb)]a(ii) a Then can be based on the formula [ E2 (Wb)]a=[E2（Wb）]a-Z* [E2（Gb）]aIterative training is performed to encrypt the training parameters [ E2 (Wb)]aUpdating is carried out, and the latest training parameters Wb are finally obtained.
In the embodiment of the invention, the training parameters are subjected to fixed point number coding, the coded training parameters are trained, and the fixed point number is intercepted through one secret sharing in the training process to complete one round of training, and the training parameters are updated to carry out the next round of training, so that the large-scale data training can be realized, the problem of data overflow caused by homomorphic encryption and fixed point number coding is avoided, the data communication amount in the training process is reduced, the training efficiency of a model is improved, and the privacy of the data can be protected by the secret sharing training method.
Fig. 5 is a schematic structural diagram of an embodiment of a training apparatus 50 based on secret sharing according to the present disclosure, and as shown in fig. 5, the training apparatus 50 based on secret sharing may include: an encoding module 51, a training module 52, a secret sharing module 53, an encryption module 54 and an output module 55;
the encoding module 51 is configured to obtain a first training parameter, perform fixed-point number encoding on the first training parameter according to a first target magnification to obtain a first encoded training parameter, and perform homomorphic encryption on the first encoded training parameter according to a public key preset at a receiving end to obtain a first encrypted encoded training parameter;
the training module 52 is configured to acquire sample data, perform fixed-point number coding on the sample data according to a second target magnification to obtain coding sample data, and train the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result;
the secret sharing module 53 is configured to perform secret sharing on the first encrypted coding prediction result to obtain a first coding prediction result, and perform fixed-point interception on the first coding prediction result according to the first target magnification to obtain a second coding prediction result;
the encryption module 54 is configured to obtain a first coding error according to the second coding prediction result and the target tag data, and perform homomorphic encryption on the first coding error according to a public key preset at the receiving end to obtain a first encryption coding error;
and the output module 55 is configured to perform iterative training after updating the first encryption coding training parameter according to the first encryption coding error to obtain a trained second encryption coding training parameter, and obtain a second training parameter according to the second encryption coding training parameter to complete training of the first training parameter.
The training device based on secret sharing provided by the embodiment shown in fig. 5 can be used for executing the technical solutions of the method embodiments shown in fig. 1 to fig. 4 in this specification, and the implementation principle and technical effects thereof can be further described with reference to the related descriptions in the method embodiments.
Fig. 6 is a schematic structural diagram of another embodiment of the training apparatus 60 based on secret sharing according to the present disclosure, and compared with the training apparatus 50 based on secret sharing shown in fig. 5, in the training apparatus 60 based on secret sharing shown in fig. 6, the secret sharing module 53 may include: a creation unit 6531, an interaction unit 6532, and an output unit 6533;
a creating unit 6531, configured to create an arbitrary random number, and perform fixed-point number coding on the random number to obtain a first coded random number at a sending end, where a coding magnification of the first coded random number at the sending end corresponds to a coding magnification of the first encryption coding prediction result;
an interaction unit 6532, configured to obtain a second encrypted encoded random number of the sending end according to a difference between the first encoded random number of the sending end and the first encrypted encoded prediction result, and send the second encrypted encoded random number of the sending end to a receiving end, so that the receiving end decrypts the second encrypted encoded random number of the sending end to obtain a second encoded random number of the sending end;
an output unit 6533, configured to receive the sending-end second encoded random number sent by the receiving end, perform fixed point number interception on the sending-end first encoded random number and the sending-end second encoded random number according to the first target magnification, respectively, to obtain a sending-end third encoded random number and a sending-end fourth encoded random number, and obtain a second encoding prediction result according to a sum of the sending-end third encoded random number and the sending-end fourth encoded random number.
In one possible implementation manner, the encryption module 54 may include: an acquisition unit 6541 and a calculation unit 6542;
an obtaining unit 6541, configured to obtain tag data, and perform fixed-point number coding on the tag data according to the second target magnification to obtain coded tag data;
a calculating unit 6542, configured to obtain a first coding error according to the second coding prediction result and the difference between the coding label data.
The training device based on secret sharing provided in the embodiment shown in fig. 6 can be used to execute the technical solutions of the method embodiments shown in fig. 1 to fig. 4 of the present application, and the implementation principles and technical effects thereof can be further described with reference to the related descriptions in the method embodiments.
Fig. 7 is a schematic structural diagram of a further embodiment of the training device 70 based on secret sharing according to the present disclosure, and compared with the training device 50 based on secret sharing shown in fig. 5, in the training device 70 based on secret sharing shown in fig. 7, the output module 55 may include: a calculation unit 7551 and an update unit 7552;
a calculating unit 7551 configured to obtain a first encryption coding gradient according to a product of the first encryption coding error and the coding sample data;
an updating unit 7552, configured to update the first encryption coding training parameter according to the first encryption coding gradient, and perform iterative training according to the updated first encryption coding training parameter to obtain a trained second encryption coding training parameter.
In one possible implementation manner, the updating unit 7552 may include: an update subunit 75521, an iteration subunit 75522, and an output subunit 75523;
an updating subunit 75521, configured to update the first encryption coding training parameter according to a difference between the first encryption coding training parameter and the first encryption coding gradient;
an iteration subunit 75522, configured to perform iterative training according to the updated first encryption coding training parameter and a preset number of rounds, and update the first encryption coding training parameter after each round of training;
and the output subunit 75523 is configured to, if the current round number reaches the preset round number, end the training and obtain a trained second encryption coding training parameter.
The training device based on secret sharing provided in the embodiment shown in fig. 7 can be used to execute the technical solutions of the method embodiments shown in fig. 1 to fig. 4 of the present application, and the implementation principles and technical effects thereof can be further described with reference to the related descriptions in the method embodiments.
Fig. 8 is a schematic structural diagram of a further embodiment of the training apparatus 80 based on secret sharing according to the present disclosure, and compared with the training apparatus 50 based on secret sharing shown in fig. 5, in the training apparatus 80 based on secret sharing shown in fig. 8, the output module 55 may include: a creation unit 8551, a reception unit 8552, and an output unit 8553;
a creating unit 8551, configured to create an arbitrary random number, obtain a third encryption coding training parameter according to a difference between the second encryption coding training parameter and the random number, and send the third encryption coding training parameter to a receiving end, so that the receiving end decrypts the third encryption coding training parameter to obtain a third coding training parameter;
a receiving unit 8552, configured to receive a third coding training parameter returned by the receiving end, and obtain a second coding training parameter according to a sum of the third coding training parameter and the random number;
and an output unit 8553, configured to perform fixed-point truncation on the second coding training parameter according to the first target magnification to obtain a second training parameter.
The training device based on secret sharing provided in the embodiment shown in fig. 8 can be used to execute the technical solutions of the method embodiments shown in fig. 1 to fig. 4 of the present application, and the implementation principles and technical effects thereof can be further described with reference to the related descriptions in the method embodiments.
It should be understood that the division of the modules of the secret sharing based training apparatus shown in fig. 5-8 is only a logical division, and the actual implementation can be wholly or partially integrated into one physical entity or physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling by the processing element in software, and part of the modules can be realized in the form of hardware. For example, the detection module may be a separate processing element, or may be integrated into a chip of the electronic device. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors (DSPs), one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, these modules may be integrated together and implemented in the form of a System-On-a-Chip (SOC)
FIG. 9 is a block diagram illustrating an embodiment of an electronic device 900, which may include at least one processor, as shown in FIG. 9; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the secret sharing-based training method provided by the embodiments shown in fig. 1 to 4 of the present specification.
The electronic device may be a training device capable of performing secret sharing, for example: the embodiments of the present specification do not limit the specific form of the electronic device. It is understood that the electronic device herein is a machine as mentioned in the method embodiments.
FIG. 9 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present specification. The electronic device 900 shown in fig. 9 is only an example and should not bring any limitation to the functions and the scope of use of the embodiments of the present specification.
As shown in fig. 9, the electronic device 900 is embodied in the form of a general purpose computing device. Components of electronic device 900 may include, but are not limited to: one or more processors 910, a memory 930, and a communication bus 940 that couples various system components including the memory 930 and the processing unit 910.
Communication bus 940 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. These architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, to name a few.
Electronic devices typically include a variety of computer system readable media. Such media may be any available media that is accessible by the electronic device and includes both volatile and nonvolatile media, removable and non-removable media.
Memory 930 may include computer system readable media in the form of volatile Memory, such as Random Access Memory (RAM) and/or cache Memory. The electronic device may further include other removable/non-removable, volatile/nonvolatile computer system storage media. Memory 930 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the specification.
A program/utility having a set (at least one) of program modules, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in memory 930, each of which examples or some combination may include an implementation of a network environment. The program modules generally perform the functions and/or methodologies of the embodiments described herein.
The processor 910 executes programs stored in the memory 930 to execute various functional applications and data processing, for example, to implement the secret sharing based training method provided in the embodiments shown in fig. 1 to 4 of the present specification.
In the above embodiments, the processors may include, for example, a CPU, a DSP, a microcontroller, or a digital Signal processor, and may further include a GPU, an embedded Neural Network Processor (NPU), and an Image Signal Processing (ISP), and the processors may further include necessary hardware accelerators or logic Processing hardware circuits, such as an ASIC, or one or more integrated circuits for controlling the execution of the program according to the technical solution of the present application. Further, the processor may have the functionality to operate one or more software programs, which may be stored in the storage medium.
The embodiments of the present specification provide a non-transitory computer-readable storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform a secret sharing based training method provided by the embodiments of fig. 1 to 4 of the present specification.
The non-transitory computer readable storage medium described above may take any combination of one or more computer readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a flash Memory, an optical fiber, a portable compact disc Read Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present description may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
In the description of the specification, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present specification, "a plurality" means at least two, e.g., two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present description in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present description.
The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It should be noted that the terminal referred to in the embodiments of the present disclosure may include, but is not limited to, a Personal Computer (Personal Computer; hereinafter, referred to as PC), a Personal Digital Assistant (Personal Digital Assistant; hereinafter, referred to as PDA), a wireless handheld device, a Tablet Computer (Tablet Computer), a mobile phone, an MP3 player, an MP4 player, and the like.
In the several embodiments provided in this specification, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present description may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a Processor (Processor) to execute some steps of the methods described in the embodiments of the present disclosure. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims (12)

1. A secret sharing based training method, comprising:
acquiring a first training parameter, performing fixed point number coding on the first training parameter according to a first target multiplying power to obtain a first coding training parameter, and performing homomorphic encryption on the first coding training parameter according to a public key preset at a receiving end to obtain a first encrypted coding training parameter;
acquiring sample data, performing fixed-point number coding on the sample data according to a second target multiplying power to obtain coding sample data, and training the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result;
secret sharing is carried out on the first encryption coding prediction result to obtain a first coding prediction result, and fixed point interception is carried out on the first coding prediction result according to the first target multiplying power to obtain a second coding prediction result;
obtaining a first coding error according to the second coding prediction result and target label data, and homomorphically encrypting the first coding error according to a preset public key of the receiving end to obtain a first encryption coding error;
obtaining a first encryption coding gradient according to the product of the first encryption coding error and the coding sample data, updating the first encryption coding training parameter according to the first encryption coding gradient to obtain an updated first encryption coding training parameter, performing iterative training according to the updated first encryption coding training parameter after updating to obtain a trained second encryption coding training parameter, and obtaining a second training parameter according to the second encryption coding training parameter to finish the training of the first training parameter; and obtaining the updated first encryption coding training parameter according to the difference value between the first encryption coding training parameter and the first encryption coding gradient.
2. The method of claim 1, wherein the secret sharing the first encrypted encoded prediction result to obtain a first encoded prediction result, and the fixed-point truncation of the first encoded prediction result according to the first target magnification to obtain a second encoded prediction result comprises:
creating an arbitrary random number, and performing fixed-point number coding on the random number to obtain a first coded random number of a sending end, wherein the coding rate of the first coded random number of the sending end corresponds to the coding rate of the first encryption coding prediction result;
obtaining a second encrypted coding random number of the sending end according to the difference value between the first coding random number of the sending end and the first encrypted coding prediction result, and sending the second encrypted coding random number of the sending end to a receiving end, so that the receiving end decrypts the second encrypted coding random number of the sending end to obtain the second coding random number of the sending end;
and receiving a second coded random number of the sending end sent by the receiving end, respectively carrying out fixed point number interception on the first coded random number of the sending end and the second coded random number of the sending end according to the first target multiplying power to obtain a third coded random number of the sending end and a fourth coded random number of the sending end, and obtaining a second coding prediction result according to the sum of the third coded random number of the sending end and the fourth coded random number of the sending end.
3. The method of claim 2, the deriving a first coding error from the second coding predictor and target tag data comprising:
acquiring target label data, and performing fixed-point number coding on the target label data according to the second target multiplying power to obtain coded label data;
and obtaining a first coding error according to the second coding prediction result and the difference value of the coding label data.
4. The method of claim 1, wherein the performing iterative training according to the updated first cipher coding training parameter after updating to obtain a trained second cipher coding training parameter comprises:
performing iterative training according to the updated first encryption coding training parameter and a preset round number, and updating the first encryption coding training parameter after each round of training;
and if the current round number reaches the preset round number, finishing the training and obtaining a trained second encryption coding training parameter.
5. The method of claim 1, the deriving a second training parameter from the second cryptographically encoded training parameter comprising:
creating an arbitrary random number, obtaining a third encryption coding training parameter according to the difference value between the second encryption coding training parameter and the random number, and sending the third encryption coding training parameter to a receiving end, so that the receiving end decrypts the third encryption coding training parameter to obtain a third coding training parameter;
receiving a third coding training parameter returned by the receiving end, and obtaining a second coding training parameter according to the sum of the third coding training parameter and the random number;
and intercepting the fixed point number of the second coding training parameter according to the first target multiplying power to obtain a second training parameter.
6. A secret sharing based training apparatus comprising:
the encoding module is used for acquiring a first training parameter, performing fixed point number encoding on the first training parameter according to a first target multiplying power to obtain a first encoded training parameter, and performing homomorphic encryption on the first encoded training parameter according to a public key preset at a receiving end to obtain a first encrypted encoded training parameter;
the training module is used for acquiring sample data, performing fixed-point number coding on the sample data according to a second target multiplying power to obtain coding sample data, and training the first encryption coding training parameter and the coding sample data to obtain a first encryption coding prediction result;
the secret sharing module is used for carrying out secret sharing on the first encryption coding prediction result to obtain a first coding prediction result, and carrying out fixed point interception on the first coding prediction result according to the first target multiplying power to obtain a second coding prediction result;
the encryption module is used for obtaining a first coding error according to the second coding prediction result and the target label data, and homomorphically encrypting the first coding error according to a preset public key of the receiving end to obtain a first encryption coding error;
the output module is used for obtaining a first encryption coding gradient according to the product of the first encryption coding error and the coding sample data, updating the first encryption coding training parameter according to the first encryption coding gradient to obtain an updated first encryption coding training parameter, performing iterative training according to the updated first encryption coding training parameter after updating to obtain a trained second encryption coding training parameter, and obtaining a second training parameter according to the second encryption coding training parameter to finish the training of the first training parameter; and obtaining the updated first encryption coding training parameter according to the difference value between the first encryption coding training parameter and the first encryption coding gradient.
7. The apparatus of claim 6, the secret sharing module comprising:
the device comprises a creating unit, a calculating unit and a calculating unit, wherein the creating unit is used for creating an arbitrary random number and carrying out fixed point number coding on the random number to obtain a first coding random number of a sending end, and the coding multiplying power of the first coding random number of the sending end corresponds to the coding multiplying power of the first encryption coding prediction result;
the interactive unit is used for obtaining a second encrypted coding random number of the sending end according to the difference value between the first coding random number of the sending end and the first encrypted coding prediction result, and sending the second encrypted coding random number of the sending end to a receiving end, so that the receiving end decrypts the second encrypted coding random number of the sending end to obtain the second coding random number of the sending end;
and the output unit is used for receiving the second coded random number of the sending end sent by the receiving end, respectively carrying out fixed point interception on the first coded random number of the sending end and the second coded random number of the sending end according to the first target multiplying power to obtain a third coded random number of the sending end and a fourth coded random number of the sending end, and obtaining a second coding prediction result according to the sum of the third coded random number of the sending end and the fourth coded random number of the sending end.
8. The apparatus of claim 7, the encryption module comprising:
the acquisition unit is used for acquiring target label data and carrying out fixed point number coding on the target label data according to the second target multiplying power to obtain coded label data;
and the calculating unit is used for obtaining a first coding error according to the second coding prediction result and the difference value of the coding label data.
9. The apparatus of claim 6, the output module comprising:
the iteration unit is used for performing iteration training according to the updated first encryption coding training parameter and a preset number of rounds and updating the first encryption coding training parameter after each round of training;
and the output unit is used for finishing the training and obtaining a trained second encryption coding training parameter if the current round number reaches the preset round number.
10. The apparatus of claim 6, the output module comprising:
the creating unit is used for creating an arbitrary random number, obtaining a third encryption coding training parameter according to the difference value between the second encryption coding training parameter and the random number, and sending the third encryption coding training parameter to a receiving end, so that the receiving end decrypts the third encryption coding training parameter to obtain a third coding training parameter;
the receiving unit is used for receiving a third coding training parameter returned by the receiving end and obtaining a second coding training parameter according to the sum of the third coding training parameter and the random number;
and the output unit is used for intercepting the fixed point number of the second coding training parameter according to the first target multiplying power to obtain a second training parameter.
11. An electronic device, comprising:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 5.
12. A non-transitory computer readable storage medium storing computer instructions which, when executed on a computer, cause the computer to perform the method of any of claims 1 to 5.
CN202010459211.7A 2020-05-27 2020-05-27 Secret sharing-based training method and device, electronic equipment and storage medium Active CN111368336B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010459211.7A CN111368336B (en) 2020-05-27 2020-05-27 Secret sharing-based training method and device, electronic equipment and storage medium
PCT/CN2021/096166 WO2021239006A1 (en) 2020-05-27 2021-05-26 Secret sharing-based training method and apparatus, electronic device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010459211.7A CN111368336B (en) 2020-05-27 2020-05-27 Secret sharing-based training method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111368336A CN111368336A (en) 2020-07-03
CN111368336B true CN111368336B (en) 2020-09-04

Family

ID=71211105

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010459211.7A Active CN111368336B (en) 2020-05-27 2020-05-27 Secret sharing-based training method and device, electronic equipment and storage medium

Country Status (2)

Country Link
CN (1) CN111368336B (en)
WO (1) WO2021239006A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111368336B (en) * 2020-05-27 2020-09-04 支付宝(杭州)信息技术有限公司 Secret sharing-based training method and device, electronic equipment and storage medium
CN111740815A (en) * 2020-07-31 2020-10-02 支付宝(杭州)信息技术有限公司 Ciphertext-based two-party secret sharing method, device, equipment and storage medium
CN112529101A (en) * 2020-12-24 2021-03-19 深圳前海微众银行股份有限公司 Method and device for training classification model, electronic equipment and storage medium
CN114648130B (en) * 2022-02-07 2024-04-16 北京航空航天大学 Longitudinal federal learning method, device, electronic equipment and storage medium
CN117648999B (en) * 2024-01-30 2024-04-23 上海零数众合信息科技有限公司 Federal learning regression model loss function evaluation method and device and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110537191A (en) * 2017-03-22 2019-12-03 维萨国际服务协会 Secret protection machine learning
CN110782883A (en) * 2019-11-12 2020-02-11 百度在线网络技术（北京）有限公司 Model training method and device, electronic equipment and storage medium
CN111105240A (en) * 2019-12-12 2020-05-05 中国科学院深圳先进技术研究院 Resource-sensitive combined financial fraud detection model training method and detection method

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7472105B2 (en) * 2004-10-19 2008-12-30 Palo Alto Research Center Incorporated System and method for providing private inference control
US10601579B2 (en) * 2017-12-20 2020-03-24 Nxp B.V. Privacy preserving comparison
CN108681698B (en) * 2018-04-28 2021-03-16 武汉大学 Large-scale iris recognition method with privacy protection function
CN109669973A (en) * 2018-12-27 2019-04-23 深算科技（重庆）有限公司 One kind being based on distributed dynamic training system
SG11201909946UA (en) * 2019-01-11 2019-11-28 Alibaba Group Holding Ltd Logistic regression modeling scheme using secrete sharing
JP6825138B2 (en) * 2019-01-11 2021-02-03 アドバンスド ニュー テクノロジーズ カンパニー リミテッド Decentralized multi-party security model training framework for privacy protection
CN110210619A (en) * 2019-05-22 2019-09-06 上海商汤智能科技有限公司 The training method and device of neural network, electronic equipment and storage medium
CN110288094B (en) * 2019-06-10 2020-12-18 深圳前海微众银行股份有限公司 Model parameter training method and device based on federal learning
CN110276210B (en) * 2019-06-12 2021-04-23 深圳前海微众银行股份有限公司 Method and device for determining model parameters based on federal learning
CN110399742B (en) * 2019-07-29 2020-12-18 深圳前海微众银行股份有限公司 Method and device for training and predicting federated migration learning model
CN110490128B (en) * 2019-08-16 2022-09-06 南京邮电大学 Handwriting recognition method based on encryption neural network
CN110728375B (en) * 2019-10-16 2021-03-19 支付宝(杭州)信息技术有限公司 Method and device for training logistic regression model by combining multiple computing units
CN110704860A (en) * 2019-11-18 2020-01-17 深圳前海微众银行股份有限公司 Longitudinal federal learning method, device and system for improving safety and storage medium
CN110955907B (en) * 2019-12-13 2022-03-25 支付宝(杭州)信息技术有限公司 Model training method based on federal learning
CN111143878B (en) * 2019-12-20 2021-08-03 支付宝(杭州)信息技术有限公司 Method and system for model training based on private data
CN111368336B (en) * 2020-05-27 2020-09-04 支付宝(杭州)信息技术有限公司 Secret sharing-based training method and device, electronic equipment and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110537191A (en) * 2017-03-22 2019-12-03 维萨国际服务协会 Secret protection machine learning
CN110782883A (en) * 2019-11-12 2020-02-11 百度在线网络技术（北京）有限公司 Model training method and device, electronic equipment and storage medium
CN111105240A (en) * 2019-12-12 2020-05-05 中国科学院深圳先进技术研究院 Resource-sensitive combined financial fraud detection model training method and detection method

Also Published As

Publication number Publication date
WO2021239006A1 (en) 2021-12-02
CN111368336A (en) 2020-07-03

Similar Documents

Publication Publication Date Title
CN111368336B (en) Secret sharing-based training method and device, electronic equipment and storage medium
CN112906044B (en) Multi-party security calculation method, device, equipment and storage medium
CN107196926B (en) Cloud outsourcing privacy set comparison method and device
CN112822005B (en) Secure transfer learning system based on homomorphic encryption
CN111783129A (en) Data processing method and system for protecting privacy
CN111371544B (en) Prediction method and device based on homomorphic encryption, electronic equipment and storage medium
WO2023116144A1 (en) Method and apparatus for joint data processing by two parties for protecting data privacy
CN114726512B (en) Data processing method and device
CN111555880A (en) Data collision method and device, storage medium and electronic equipment
Jalali et al. ARMv8 SIKE: Optimized supersingular isogeny key encapsulation on ARMv8 processors
CN115664747A (en) Encryption method and device
CN112231740B (en) Data encryption method, data decryption method, data encryption device, computer device, and medium
CN114640436A (en) Packet statistical parameter calculation method and device based on privacy protection
CN114726580B (en) Data processing method and device
JP7297131B2 (en) Distributed Machine Learning Model Training Method, Apparatus, Equipment and Medium
CN116861477A (en) Data processing method, system, terminal and storage medium based on privacy protection
CN114726511B (en) Data processing method and device
WO2022137447A1 (en) Confidential information processing system, and confidential information processing method
CN114726611A (en) Multiparty privacy set intersection method, system and device
CN113645022A (en) Method and device for determining privacy set intersection, electronic equipment and storage medium
CN113535121A (en) Safe and efficient mathematical division calculation optimization method based on secret sharing protocol
CN116881950B (en) Processing method and device of privacy data, electronic equipment and readable storage medium
Shen et al. Privacy-preserving multi-party deep learning based on homomorphic proxy re-encryption
CN114095157B (en) Key management method, key management device, computer equipment and readable storage medium
CN117009723B (en) Multiparty computing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40033203

Country of ref document: HK